System

Abstract

A system is provided.SOLUTION: A system comprising: means for detecting a face; means for analyzing an upper region of the detected face and identifying a thin hair portion; means for generating hair by digital processing for the identified thin hair portion and correcting the hair to a natural state; and means for storing the corrected image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many people worry about their thinning hair when taking photos, which can make them feel less confident. This problem is particularly noticeable on the top of the head and at the hairline, and can cause stress and low self-esteem. Traditional photo editing software often manually corrects thinning hair, requiring advanced skills and time to achieve natural results. Therefore, there is a need for a system that can automatically detect thinning hair areas and correct them for a natural look, allowing users to easily take photos with confidence. [Means for solving the problem]

[0005] The present invention provides a system that includes a face detection means, a means for analyzing the detected upper region of the face to identify areas with thinning hair, a means for digitally generating hair in the identified areas with thinning hair and correcting it to a natural look, and a means for saving the corrected image. This system uses a convolutional neural network to identify the upper region of the face, and a Poisson blending technique to digitally generate hair. This allows users to easily take photos with natural hairstyles, allowing them to take photos with confidence.

[0006] "Face detection means" refers to a function or algorithm for recognizing the face of a subject in an image or video and identifying its position.

[0007] The "upper region of the face" refers to the uppermost part of the face, including the top of the head and the hairline, after face detection.

[0008] "Analysis" is the process of examining image data in detail for a specific purpose, breaking it down into components, and understanding them.

[0009] "Thinning areas" refers to areas where hair is thinning, such as the top of the head or hairline.

[0010] "Means of identification" are methods or techniques for recognizing and clarifying specific objects or conditions.

[0011] "Digital processing" is the process of manipulating digital data and performing calculations or transformations to produce a particular result.

[0012] "Generating hair" refers to using digital technology to artificially draw new hair into thinning areas.

[0013] "Correcting to a natural look" refers to adjusting the hair generated through digital processing so that it blends seamlessly with the original image or video.

[0014] "Enhanced images" refer to photographs or videos that have been digitally altered to make areas of thinning hair appear more natural.

[0015] "Storage means" refers to a method or function for storing processed image data in a storage device of a device or the like.

[0016] A "system" is a set of elements, such as hardware, software, and algorithms, that work together to achieve a specific purpose.

[0017] A "convolutional neural network" is a type of deep learning model that is applied to image analysis and is particularly effective for tasks such as pattern recognition and object detection.

[0018] "Poisson blending technology" is an image processing technique that makes the boundaries between different image regions look natural when they are joined together. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] System Overview

[0041] The system of the present invention uses a device equipped with a camera and provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. The system includes a face detection unit, a hair thinning area identification unit, a digital processing unit, and a corrected image storage unit.

[0042] Program processing and explanation

[0043] 1. Launching the app and initializing the camera

[0044] Subject: User, Device

[0045] The user launches the camera app on their smartphone.

[0046] The device initializes the camera module and displays real-time video on the preview screen, activating the device's internal image recognition engine.

[0047] 2. Face detection and head location

[0048] Subject: Terminal

[0049] The device uses image recognition algorithms, such as the Haar detector or deep learning-based FaceNet, to detect human faces from each frame of video.

[0050] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[0051] 3. Analysis and identification of thinning hair areas

[0052] Subject: Terminal

[0053] The device analyzes an image region of the top of the head and runs a convolutional neural network (CNN) algorithm to identify areas of thinning hair based on hair density and texture information.

[0054] A region mask of the identified thinning hair area is generated and its location is recorded.

[0055] 4. Performing digital correction

[0056] Subject: Terminal

[0057] For each identified thinning area, the device selects an appropriate patch from a library of hair patches and digitally processes it to fit the thinning area.

[0058] The digital processing uses Poisson blending techniques to naturally integrate the patches into the original image.

[0059] 5. Saving and displaying the corrected image

[0060] Subject: Terminal, User

[0061] The device stores the corrected image in local storage and displays it to the user.

[0062] Users can view the saved images and share them on social media if desired.

[0063] Specific examples

[0064] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[0065] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[0066] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[0067] 4. The device identifies areas of thinning hair and generates hair in those areas using digital processing means.

[0068] 5. The device saves the digitally enhanced image and displays it to the user, who can then review it and save or share it as desired.

[0069] In this way, the system of the present invention instantly corrects thinning hair on the top of the head or at the hairline in a photograph, allowing you to take photos with confidence. Our invention provides a powerful tool for taking natural-looking photos without worrying about thinning hair.

[0070] The processing flow will be explained below.

[0071] Step 1: Launch the app

[0072] Subject: User

[0073] The user launches the camera app on their smartphone by tapping the icon on the home screen.

[0074] Step 2: Initialize the camera and acquire video

[0075] Subject: Terminal

[0076] The device initializes the camera module, displays the real-time camera image on the preview screen, and starts capturing the video stream.

[0077] Step 3: Face detection

[0078] Subject: Terminal

[0079] The device uses a face detection algorithm (such as a Haar detector or deep learning-based FaceNet) to detect human faces in each frame of video, then locates the detected face and obtains its coordinates.

[0080] Step 4: Parietal region estimation

[0081] Subject: Terminal

[0082] Based on the detected face position, the device estimates the top of the head area, which is located at the top of the face and may contain thinning hair.

[0083] Step 5: Analysis of thinning areas

[0084] Subject: Terminal

[0085] The device analyzes the image area of ​​the top of the head using a convolutional neural network (CNN) to analyze hair density and texture information, thereby identifying areas with thinning hair and generating a region mask.

[0086] Step 6: Prepare for digital correction

[0087] Subject: Terminal

[0088] The device selects a patch of hair from a data library that matches the identified thinning area and prepares to map the selected patch to the thinning area.

[0089] Step 7: Perform digital correction

[0090] Subject: Terminal

[0091] The device uses Poisson blending technology to seamlessly integrate selected hair patches into thinning areas, smoothly connecting different image regions and creating natural-looking boundaries.

[0092] Step 8: Generate the corrected image

[0093] Subject: Terminal

[0094] The device generates an image after the digital correction process is complete and displays it on the preview screen. The correction results are then displayed on the screen for the user to review.

[0095] Step 9: Save the image

[0096] Subject: User, Device

[0097] If the user selects save, the device saves the generated corrected image to local storage and notifies the user that the save is complete.

[0098] Step 10: Share your images

[0099] Subject: User

[0100] Users can view the saved images and share them on social media or other platforms as needed. Images can be easily shared via the share button.

[0101] Through each processing step, images with naturally corrected thinning hair are generated, which can then be saved and shared, helping users take photos with confidence.

[0102] Example 1

[0103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] Conventional camera apps lack the ability to naturally correct thinning hair, preventing many users who are concerned about their thinning hair from taking natural-looking photos. Furthermore, simple image editing software often struggles to correct thinning hair in real time, often resulting in a poor user experience. There is a need for a system that can solve these issues and allow users to enjoy taking photos with confidence.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0106] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas with thinning hair, a means for digitally generating hair in the identified areas with thinning hair and correcting the hair to a natural state, a means for saving the corrected image, a means for displaying the image on a preview screen in real time, a means for estimating the detected upper region of the face and acquiring its position coordinates, a means for identifying areas with thinning hair based on the analysis results and drawing the areas as a mask, a means for selecting an optimal patch from a hair patch library, and a means for applying the selected patch to the areas with thinning hair and digitally processing them. This allows users to take natural-looking photos that have been corrected in real time without worrying about thinning hair.

[0107] A "face detection method" is an algorithm or technology for recognizing a person's face in each frame of video and identifying its location.

[0108] The "means for identifying areas of thinning hair" is a technique for analyzing the detected upper face region and identifying areas of thinning hair based on hair density and texture.

[0109] "Digitally generating hair and natural-looking correction" is a process of applying appropriate hair patches to identified thinning areas and integrating them naturally into the original image using digital image processing techniques.

[0110] The "means for saving corrected images" is a function for saving images that have completed processing in local storage or the cloud.

[0111] "Means for displaying video on a preview screen in real time" is a function that displays video captured by a camera in real time, allowing the user to check it immediately.

[0112] The "means for estimating the upper region of the detected face and acquiring its position coordinates" is a technology for calculating the position coordinates of the top of the head and hairline in particular based on the position information of the face.

[0113] "Means for identifying areas of thinning hair based on the analysis results and drawing those areas as a mask" is a function for identifying areas of thinning hair based on the information obtained by analysis and displaying those areas as a mask superimposed on the image.

[0114] The "means for selecting an optimal patch from a hair patch library" refers to an algorithm or technique for selecting an optimal hair patch from a library based on hair color and texture.

[0115] The "means of applying selected patches to the thinning hair area and digitally processing" refers to the process of applying selected hair patches to the thinning hair area and processing them to create a natural-looking composite.

[0116] This invention provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. The system includes a face detection unit, a hair thinning area identification unit, a digital processing unit, and a corrected image storage unit. These units are realized through a real-time video preview screen display unit, a unit for estimating the upper face area and acquiring its coordinates, a unit for identifying the thinning hair area based on the analysis results and drawing the area as a mask, a unit for selecting an optimal patch from a hair patch library, and a unit for applying the selected patch to the thinning hair area and performing digital processing.

[0117] App launch and camera initialization

[0118] The user launches the camera app on their smartphone.

[0119] The device will initialize the camera module and display real-time video on the preview screen. The internal image recognition engine will be ready.

[0120] Face detection and head location

[0121] The device detects human faces from each frame of video using image recognition algorithms, such as the Haar detector or FaceNet.

[0122] The upper area of ​​the detected face (top of the head and hairline area) is estimated and its position coordinates are obtained.

[0123] Analysis and identification of thinning hair areas

[0124] The device analyzes an image region of the top of the head and uses a convolutional neural network (CNN) to identify areas of thinning hair based on hair density and texture information.

[0125] A region mask of the identified thinning hair area is generated and its location is recorded.

[0126] Performing digital correction

[0127] Based on the mask of the bald area, the device selects the best patch from a hair patch library, based on color and texture match criteria.

[0128] The device uses Poisson blending technology to apply the selected patch to the thinning area, allowing it to blend in naturally.

[0129] Saving and displaying the corrected image

[0130] The device saves the corrected image to local storage in a standard image format such as JPEG or PNG.

[0131] The device displays the saved image to the user, allowing them to check it on a preview screen.

[0132] Users can review the images and use the in-app sharing features to share them on social media or with friends if desired.

[0133] Specific examples

[0134] Example 1: Taking a daily selfie

[0135] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[0136] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[0137] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[0138] 4. The terminal identifies areas of thinning hair and generates hair in those areas using digital processing means.

[0139] 5. The device saves the digitally enhanced image and displays it to the user, who can then review it and save or share it as desired.

[0140] Example 2: When taking photos for social media

[0141] 1. The user launches the app to take a photo for social media.

[0142] 2. The device applies filters to the image for social media, while also detecting and correcting faces and areas of baldness using the process described above.

[0143] 3. The device saves the corrected image to local storage and displays it to the user.

[0144] 4. The user uploads this image to a social networking site.

[0145] Example prompts for generative AI models

[0146] "How can I use a camera app to naturally correct thinning hair on the crown and at the hairline?"

[0147] "Please explain the process of using image recognition technology to detect faces and correct hair in real time."

[0148] "Tell me more about how convolutional neural networks can be used to detect thinning hair and digitally generate natural-looking hair."

[0149] As described above, the system according to the present invention provides a powerful tool for users to take natural-looking photos in real time without worrying about thinning hair.

[0150] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0151] Step 1: Launch the app and initialize the camera

[0152] The user launches the camera app on their smartphone.

[0153] Input: User action (app launch)

[0154] The device initializes the camera module and displays real-time video on the preview screen.

[0155] What happens: The device's internal image recognition engine is activated, and the camera settings and capture mode are adjusted appropriately.

[0156] Output: Initialized camera module and started image recognition engine

[0157] Step 2: Detecting the face and identifying the top of the head

[0158] The device detects human faces from each frame of video using an image recognition algorithm (e.g., Haar detector or FaceNet).

[0159] Input: Real-time video frame data

[0160] What it does: Runs an algorithm to determine the location of the face.

[0161] Output: Detected face location information

[0162] Step 3: Identify thinning areas

[0163] The device analyzes the detected upper face area and uses a convolutional neural network (CNN) to identify areas of thinning hair based on hair density and texture information.

[0164] Input: Image data of the upper region of the face

[0165] Specific operation: Apply CNN algorithm to analyze and generate a mask of the thinning hair area.

[0166] Output: Region mask of identified thinning hair areas

[0167] Step 4: Perform digital correction

[0168] Based on the mask of the thinning hair area, the device selects the most suitable patch from a library of hair patches.

[0169] Input: Thinning hair area mask

[0170] What it does: Selects a suitable patch from a patch library based on color and texture match.

[0171] Output: Selected hair patch

[0172] Step 5: Patching and Integration

[0173] The device uses Poisson blending techniques to naturally integrate the selected patches into the original image.

[0174] Input: Selected hair patch, original image data

[0175] What it does: It applies patches using Poisson blending techniques to achieve a natural look.

[0176] Output: Corrected image data

[0177] Step 6: Save and view the corrected image

[0178] The device saves the corrected image in local storage.

[0179] Input: Corrected image data

[0180] Specific behavior: Saves to local storage in a standard image format such as JPEG or PNG.

[0181] Output: Image file saved in local storage

[0182] The device displays the saved image to the user, who can check it on a preview screen.

[0183] Specific operation: Loads the saved image file and displays it on the preview screen.

[0184] Users can view the images and share them on social media if desired.

[0185] Input: Saved image file

[0186] Output: Corrected image displayed in preview window

[0187] Through these processing steps, the system allows users to take natural-looking photos that are corrected in real time without worrying about thinning hair.

[0188] (Application example 1)

[0189] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0190] Currently, many users are concerned about their thinning hair, which can affect their ability to try on hats and hair accessories. To enhance the in-store try-on experience, it is important for users to be able to see how they look in real-life. However, current try-on systems lack the ability to naturally correct thinning hair, potentially damaging users' self-image and confidence. Therefore, there is a need for a system that can naturally correct thinning hair and provide a real-time overall view of how the item will look when trying on.

[0191] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0192] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas of thinning hair, a means for digitally generating hair for the identified areas of thinning hair and correcting it to a natural look, a means for saving the corrected image, a means for displaying the identified areas of thinning hair in real time while keeping the corrected natural look, and a means for displaying, saving, or sharing the corrected image in real time so that the user can check how the clothes will look when tried on. This allows the user to try on clothes with their thinning hair corrected and check how they will look natural. This improves the try-on experience in a physical store and increases the user's confidence in their self-image.

[0193] "Face detection means" refers to technology that automatically identifies a person's face from image data acquired using a camera.

[0194] "Methods for identifying thinning hair areas" refer to techniques that use image analysis to identify areas with particularly low hair density, typically using convolutional neural networks (CNNs).

[0195] "Digitally generating and correcting hair to a natural appearance" refers to a technique for digitally generating and correcting natural-looking hair in a specified thinning area. Poisson blending is commonly used.

[0196] "Means for saving corrected images" refers to a function for saving image data that has been digitally corrected in a storage device of the terminal.

[0197] "Means for displaying in real time" refers to the function of instantly displaying corrected images or videos on the user's screen.

[0198] "Means for users to check the fitting condition" refers to technology that allows users to check the condition of the fitting items they are currently wearing on digital video.

[0199] "Means for displaying, saving, or sharing video in real time" refers to the ability to provide the user with corrected video in real time and save the video for later viewing or share it with others.

[0200] The system based on this invention provides an application that allows users to try on clothes in real time in a physical store while correcting thinning areas of the hair.

[0201] System configuration

[0202] Hardware

[0203] This system utilizes the camera module of mobile devices such as smartphones and tablets. The device camera is used to capture real-time images of the user.

[0204] software

[0205] The software components used include:

[0206] 1. Image recognition software: Face detection is performed using the open source OpenCV library.

[0207] 2. Convolutional Neural Network (CNN) model: This model is used to identify areas of thinning hair on the user, using a pre-trained model such as "hair_thinning_model.h5."

[0208] 3. Poisson Blending Technique: A technique for digitally correcting thinning areas of hair, used to produce natural-looking hair.

[0209] System Operation

[0210] Face detection and analysis

[0211] The device activates the camera and captures the user's video in real time, which is then analyzed through the OpenCV library to determine the face position.

[0212] Identifying thinning areas of hair

[0213] To analyze the upper region of the detected face, we use a convolutional neural network (CNN) model, which evaluates hair density on the crown and identifies areas of thinning hair.

[0214] Digital Correction

[0215] Any identified thinning areas are corrected by digitally generating natural hair using Poisson blending technology, resulting in a natural-looking image.

[0216] Real-time display and storage

[0217] The corrected video is displayed on the device screen in real time, allowing users to enjoy trying on clothes while checking the corrected video. Corrected video and images can also be saved to the device's memory and shared on social media if desired.

[0218] Examples of concrete examples and prompts

[0219] Specific examples

[0220] Users can use the system to film themselves while trying on hats in a physical store, and the system will correct thinning areas of hair in real time to provide a natural-looking image. Users can then check the fit and appearance of the item they are trying on while viewing the corrected image.

[0221] Prompt Sentence Examples

[0222] Use technology that naturally corrects thinning hair to provide the perfect virtual try-on app for users trying on hats. This application detects and corrects thinning hair areas when users take a real-time video of themselves using their smartphone camera. Display this corrected real-time video so users can enjoy trying on hats.

[0223] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0224] Step 1:

[0225] The user launches the app on their smartphone

[0226] Input: Tap the application icon on your smartphone

[0227] Operation: The application starts and the initial setup screen is displayed.

[0228] Output: Real-time camera preview of the smartphone is displayed.

[0229] Step 2:

[0230] The device initializes the camera module and captures real-time video.

[0231] Input: Live video stream from the camera module

[0232] Operation: Initialize the camera module and start video input.

[0233] Output: Video frames captured in real time are stored in memory

[0234] Step 3:

[0235] The device detects your face

[0236] Input: Real-time video frame

[0237] Operation: Performs face detection using the Haar Cascade from the OpenCV library

[0238] Output: Face position coordinates (rectangular area) are obtained

[0239] Step 4:

[0240] The device analyzes the upper area of ​​your face to identify areas with thinning hair.

[0241] Input: Face position coordinates and video frame of the upper region

[0242] How it works: Uses a convolutional neural network (CNN) to assess hair density and identify thinning areas.

[0243] Output: Mask information and position data of thinning hair area are obtained

[0244] Step 5:

[0245] The device performs digital processing on the identified thinning areas.

[0246] Input: Position data of thinning hair area and mask information

[0247] How it works: Using Poisson blending technology, patches of hair are digitally generated and applied to thinning areas.

[0248] Output: Naturally corrected video frames

[0249] Step 6:

[0250] The device displays the corrected image in real time.

[0251] Input: Digitally corrected video frame

[0252] Operation: The corrected image is displayed on the screen in real time.

[0253] Output: Users can check the corrected image on the preview screen.

[0254] Step 7:

[0255] The user saves or shares the corrected footage

[0256] Input: Corrected video frame and user action (choice of saving or sharing)

[0257] How it works: Save the corrected footage to your device's storage or share it via social media or messaging apps

[0258] Output: You will get a saved image file or a shared link or message.

[0259] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0260] System Overview

[0261] The system of the present invention uses a device equipped with a camera and provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. It also combines an emotion engine that recognizes the user's emotions to achieve more optimal correction processing. The system includes a face detection means, a means for identifying thinning hair areas, a digital processing means, an emotion recognition means, and a means for saving the corrected image.

[0262] Program processing and explanation

[0263] 1. Launching the app and initializing the camera

[0264] Subject: User, Device

[0265] The user launches the camera app on their smartphone.

[0266] The device initializes the camera module and displays real-time video on the preview screen, while the emotion engine is also activated.

[0267] 2. Face detection and head location

[0268] Subject: Terminal

[0269] The device uses a face detection algorithm (e.g., Haar detector or deep learning-based FaceNet) to detect human faces from each frame of video.

[0270] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[0271] 3. Analysis and identification of thinning hair areas

[0272] Subject: Terminal

[0273] The device analyzes an image region of the top of the head and runs a convolutional neural network (CNN) algorithm to identify areas of thinning hair based on hair density and texture information.

[0274] A region mask of the identified thinning hair area is generated and its location is recorded.

[0275] 4. Emotion Recognition and Analysis

[0276] Subject: Terminal

[0277] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[0278] 5. Performing digital correction

[0279] Subject: Terminal

[0280] The device selects patches of hair from a data library that match the identified thinning areas.

[0281] The selected patch is mapped onto the thinning hair area and then naturally integrated using Poisson blending technology.

[0282] The emotion engine adjusts the correction based on the emotion it recognizes. For example, if the user is smiling, the image brightness and saturation will be improved.

[0283] 6. Creating and saving the corrected image

[0284] Subject: Terminal, User

[0285] The terminal generates an image after the digital correction process is completed and displays it to the user on a preview screen.

[0286] If the user selects save, the device saves the generated corrected image in local storage. The emotion data recognized by the emotion engine is also saved as history.

[0287] Specific examples

[0288] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[0289] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[0290] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[0291] 4. The device identifies the areas of thinning hair and uses an emotion engine to analyze the user's facial expressions. For example, if the user is smiling, that emotion data is recorded.

[0292] 5. The device digitally processes the thinning hair and generates hair. The brightness and saturation of the image are also adjusted based on the user's emotions.

[0293] 6. The device displays the digitally enhanced image on the preview screen, and if the user selects save, the image is saved to local storage. Emotion data is also saved as history.

[0294] 7. The user can review the saved images and share them on social media if desired.

[0295] Through these steps, the system of the present invention can naturally correct thinning hair on the user's head and hairline, and then use the emotion engine to perform optimal image adjustments, allowing users to take photos with confidence and easily share the corrected images.

[0296] The processing flow will be explained below.

[0297] Step 1: Launch the app and initialize the camera

[0298] Subject: User, Device

[0299] The user launches the camera app on their smartphone.

[0300] The device initializes the camera module and displays the real-time camera image on the preview screen, while simultaneously activating the image recognition engine and emotion engine.

[0301] Step 2: Detecting the face and identifying the top of the head

[0302] Subject: Terminal

[0303] The device runs a face detection algorithm (e.g., Haar detector or FaceNet) on each frame of camera footage to detect human faces.

[0304] Based on the position coordinates of the detected face, the top of the head area (upper part of the face) is estimated and its position coordinates are obtained.

[0305] Step 3: Analysis of thinning areas

[0306] Subject: Terminal

[0307] The device uses a convolutional neural network (CNN) to analyze the hair density and texture of the image area of ​​the top of the head and identify areas of thinning hair.

[0308] A region mask of the identified thinning hair area is generated and its location is recorded.

[0309] Step 4: Recognize emotions

[0310] Subject: Terminal

[0311] The device uses an emotion engine to analyze the user's facial expressions, extracting facial features and recognizing emotions such as smiles and sadness in real time.

[0312] Step 5: Prepare for digital correction

[0313] Subject: Terminal

[0314] The device selects a patch of hair from a data library that matches the thinning area, a selection process that includes assessing the suitability based on the size, color, and texture of the thinning area.

[0315] Step 6: Perform digital correction

[0316] Subject: Terminal

[0317] The device maps selected patches of hair to the thinning areas and uses Poisson blending techniques to naturally integrate the selected patches into the original image.

[0318] The emotion engine adjusts the brightness and saturation of the corrected image based on the emotion it recognizes: for example, if the user is smiling, the brightness and saturation will be increased to make the image appear more positive.

[0319] Step 7: Generate and display the corrected image

[0320] Subject: Terminal

[0321] The device generates an image after the digital correction process is complete and displays it on a preview screen, allowing the user to check the results immediately.

[0322] Step 8: Save the corrected image

[0323] Subject: User, Device

[0324] If the user selects the save option, the device saves the generated corrected image to local storage. The user's emotion data is also saved as history.

[0325] Step 9: Share your images

[0326] Subject: User

[0327] Users can view the saved images and share them on social media or other platforms as desired. Images can be easily shared using the in-app share buttons.

[0328] These steps correct thinning hair areas and provide optimal image adjustments based on the user's emotions, allowing users to take photos with confidence and easily share the corrected images.

[0329] Example 2

[0330] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0331] Conventional camera applications have had the problem of difficulty in achieving a natural-looking result when correcting thinning hair. Furthermore, optimal correction processing that reflects the user's emotions is not performed, and corrected images often do not meet the user's expectations. To solve these problems, a system is needed that can naturally correct thinning hair on the top of the head and at the hairline, and also adjust images based on the user's emotions.

[0332] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0333] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face to identify areas with thinning hair, a means for digitally generating hair in the identified areas with thinning hair and correcting it to a natural state, a means for using an emotion engine that analyzes the user's facial expressions from real-time video and recognizes their emotions, a means for fine-tuning the correction based on the emotions recognized by the emotion engine, and a means for saving the corrected image. This makes it possible to naturally correct thinning hair on the top of the user's head and at the hairline, and enables optimal image adjustment based on the user's emotions.

[0334] "Face detection method" refers to algorithms or devices that detect human faces in video, including Haar detectors and deep learning-based detection techniques.

[0335] "Means for identifying thinning hair areas" refers to an algorithm or device that analyzes the detected upper face region and identifies thinning hair areas based on hair density and texture information. This includes a convolutional neural network (CNN).

[0336] "Digitally generated hair enhancements" refers to algorithms and techniques that apply digital image processing to identified thinning hair areas to enhance the appearance of natural-looking hair, including Poisson blending techniques.

[0337] "Means for using an emotion engine" refers to software or hardware that analyzes a user's facial expressions from real-time video and recognizes their emotions.

[0338] "Means for fine-tuning corrections based on emotions" refers to algorithms and technologies that adjust the brightness, saturation, etc. of the corrected image according to the user's emotions recognized by the emotion engine.

[0339] "Means for storing corrected images" refers to software or devices for storing images that have undergone digital correction processing in local storage or cloud storage.

[0340] The system of the present invention provides a camera application that naturally corrects thinning hair on the top of the user's head and at the hairline. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more optimal correction processing can be achieved. The system includes a face detection unit, a means for identifying thinning hair areas, a digital processing unit, an emotion recognition unit, and a means for saving the corrected image.

[0341] Hardware and software used

[0342] The system's main hardware is a device such as the user's smartphone. The device must be equipped with a camera module and a high-performance CPU or GPU. The software used includes a face detection algorithm using a Haar detector and FaceNet, a hair loss detection algorithm using a convolutional neural network (CNN), a digital correction algorithm using Poisson blending technology, and an emotion engine for emotion recognition.

[0343] Processing flow and specific examples

[0344] 1. Launching the app and initializing the camera

[0345] The user launches a dedicated camera app.

[0346] The device will initialize the camera module, display real-time video on the preview screen, and start the emotion engine.

[0347] 2. Face detection and head location

[0348] The device detects faces from each frame of video using a Haar detector or FaceNet.

[0349] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[0350] 3. Analysis and identification of thinning hair areas

[0351] The device analyzes the image area of ​​the top of the head and runs a CNN algorithm to identify areas of thinning hair based on hair density and texture information.

[0352] A region mask of the identified thinning hair area is generated and its location is recorded.

[0353] 4. Emotion Recognition and Analysis

[0354] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[0355] 5. Performing digital correction

[0356] The device selects patches of hair from a data library that match the identified thinning areas.

[0357] The selected patch is mapped onto the thinning hair area and then naturally integrated using Poisson blending technology.

[0358] The emotion engine fine-tunes the correction based on the emotion it recognizes, for example, if the user is smiling, it will improve the brightness and saturation of the image.

[0359] 6. Creating and saving the corrected image

[0360] The terminal generates an image after the digital correction process is completed and displays it to the user on a preview screen.

[0361] If the user selects save, the device saves the generated corrected image in local storage. The emotion data recognized by the emotion engine is also saved as history.

[0362] Specific examples

[0363] 1. The user launches the camera app and takes a picture of themselves.

[0364] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[0365] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[0366] 4. The device identifies areas of thinning hair and uses an emotion engine to analyze the user's facial expressions, for example, brightening the image if the user is smiling.

[0367] 5. The device digitally processes the thinning areas, using selected patches of hair to create a natural-looking correction. The corrected image is then fine-tuned based on the user's emotions.

[0368] 6. The device displays the generated corrected image on the preview screen, and when the user selects save, the image is saved to local storage. At the same time, emotion data is saved.

[0369] 7. Users can view the saved images and share them on social media etc.

[0370] Prompt Sentence Examples

[0371] "Please analyze real-time images captured by a smartphone camera app, identify the state of the user's face and hair, and then digitally correct any thinning hair to make it look natural. Please provide an example of optimal correction that takes the user's emotions into consideration."

[0372] "Please explain an example of a system that automatically corrects thinning hair in images taken by a user using a camera app, and then analyzes real-time facial expressions to adjust the image based on emotion."

[0373] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0374] Program processing flow

[0375] Step 1:

[0376] App launch and camera initialization

[0377] The user simply launches a dedicated camera app.

[0378] The device will initialize the camera module and display real-time video on the preview screen, while also starting the emotion engine.

[0379] Input: Camera app launch command.

[0380] Output: Real-time camera footage and emotion engine initialization complete.

[0381] Specific operation: When a user launches the app, the device's camera module is activated and the image is previewed in real time. The emotion engine is also initialized in the background.

[0382] Step 2:

[0383] Face detection and head location

[0384] The device detects faces in each frame of video using a face detection algorithm (such as a Haar detector or FaceNet).

[0385] The upper area of ​​the detected face, i.e., the top of the head and hairline, is estimated and its position coordinates are obtained.

[0386] Input: Real-time camera footage.

[0387] Output: Detected face position coordinates and head top region.

[0388] Specific operation: A face detection algorithm is run on each video frame captured by the device's camera to determine the position of the face and the coordinates of the top of the head.

[0389] Step 3:

[0390] Analysis and identification of thinning hair areas

[0391] The device extracts the top and hairline regions from the facial detection results and runs a convolutional neural network (CNN) algorithm to analyze these regions.

[0392] The device identifies areas of thinning hair based on hair density and texture information, generates a region mask for the identified areas of thinning hair, and records their location information.

[0393] Input: parietal region coordinates.

[0394] Output: Mask of thinning hair areas and their locations.

[0395] Specific operation: The image of the top of the head is analyzed using a CNN algorithm to identify areas with low hair density, generate a mask, and record it.

[0396] Step 4:

[0397] Emotion Recognition and Analysis

[0398] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[0399] Input: Real-time video of the user.

[0400] Output: User emotion data.

[0401] Specific operation: Facial features are extracted from real-time video and analyzed by an emotion engine to identify the user's emotions.

[0402] Step 5:

[0403] Performing digital correction

[0404] The device selects patches of hair from a data library that match the identified thinning areas.

[0405] Selected patches of hair are mapped onto thinning areas and then naturally integrated using Poisson blending techniques.

[0406] The emotion engine fine-tunes the correction based on the emotions it recognizes, for example, increasing the brightness and saturation of the image if the user is smiling.

[0407] Input: Mask of thinning hair area, hair patch from data library, emotion data.

[0408] Output: The corrected image.

[0409] Specific operation: Select the appropriate hair patch to match the thinning area, use Poisson blending technology to perform natural correction, and further fine-tune the correction based on the user's emotions.

[0410] Step 6:

[0411] Generate and save the corrected image

[0412] The device generates an image with the digital correction process completed and displays it to the user on a preview screen.

[0413] If the user selects save, the device will save the generated corrected image to local storage, along with the emotion data recognized by the emotion engine.

[0414] Input: Corrected image, user save command.

[0415] Output: Saved corrected image and emotion data.

[0416] Specific operation: The corrected image is displayed on the preview screen, and if the user selects save, the image and emotion data are saved to local storage.

[0417] (Application example 2)

[0418] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0419] In today's world, many people suffer from thinning hair, but technology to naturally correct it is still insufficient. Furthermore, there is a lack of a way to generate optimal images based on the user's emotional state, and an effective method to improve the user experience is needed. Improved customer service using such technology is especially desirable in brick-and-mortar stores.

[0420] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas with thinning hair, a means for digitally generating hair for the identified areas with thinning hair and correcting it to a natural state, a means for analyzing the user's emotions and optimizing the image, and a means for saving the corrected image. This makes it possible to naturally correct the user's thinning hair and generate an optimal image according to the user's emotional state.

[0421] "Face detection means" is a technology for recognizing a person's face in an image and identifying its position.

[0422] The "means for analyzing the detected upper region of the face and identifying areas with thinning hair" is a technology that focuses on the upper region of the face and identifies areas with thinning hair by analyzing the density and condition of the hair within that region.

[0423] The "means of generating hair by digital processing in identified thinning areas and correcting it to a natural state" refers to a technology that uses digital technology to generate hair in areas determined to be thinning hair and then blends it naturally with the surrounding hair.

[0424] "Means for analyzing user emotions and optimizing images" refers to technology that analyzes the user's facial expressions and emotional data and adjusts the brightness, saturation, etc. of the image to an optimal state based on that data.

[0425] The "means for saving the corrected image" refers to a technology for saving the image that has been digitally corrected in the memory or storage of the device.

[0426] The system of the present invention includes a face detection means, a means for analyzing the upper region of the detected face to identify areas of thinning hair, a means for digitally generating hair in the identified areas of thinning hair and correcting it to a natural state, a means for analyzing a user's emotions to optimize the image, and a means for saving the corrected image.

[0427] System Configuration

[0428] Hardware

[0429] Built-in camera: A camera for capturing the user's face in real time and acquiring video data.

[0430] Display: A monitor that displays the corrected image and real-time feedback to the user. It is expected to be installed in fitting rooms in physical stores or beauty salons.

[0431] software

[0432] OpenCV: A library for image processing, used for face detection and video analysis.

[0433] Keras: A library for running deep learning models, used in convolutional neural networks (CNNs) and emotion recognition models.

[0434] Haar Cascade: A classifier for face detection

[0435] Process Overview

[0436] 1. Camera initialization and image acquisition

[0437] The server initializes the built-in camera and captures real-time video, which is then analyzed using face detection means.

[0438] 2. Face Detection

[0439] The server uses OpenCV's Haar Cascade to detect faces in the video, and once a face is detected, its location information is obtained.

[0440] 3. Identify thinning areas

[0441] The detected upper face region is analyzed using a convolutional neural network (CNN) using Keras to identify areas with thinning hair. A mask of the thinning hair region is generated and its location is recorded.

[0442] 4. Emotion Analysis

[0443] The server uses Keras emotion models to analyze the user's facial expressions from real-time video, specifically recognizing emotions such as smile or sadness, and adjusts image correction accordingly.

[0444] 5. Digital Processing

[0445] The server selects patches of hair from a data library that match the identified thinning areas, integrates them naturally using Poisson blending techniques, and adjusts the correction based on emotions recognized by the emotion engine.

[0446] 6. Creating and saving the corrected image

[0447] The image after correction processing is displayed on the screen, and if the user selects save, the image is saved to local storage. Emotion analysis data is also saved as history.

[0448] Specific examples

[0449] When a user stands in front of a smart mirror at a beauty salon, the camera detects the user's face. The upper area of ​​the detected face is analyzed, and areas of thinning hair are identified using CNN. Next, the emotion engine analyzes the user's facial expression, adjusting brightness and saturation if the user is smiling. Finally, an image with the thinning hair corrected is displayed on the screen, and if the user is satisfied, they can save the image.

[0450] Prompt Sentence Examples

[0451] It identifies the user's facial area, detects and corrects thinning hair on the top of the head and at the hairline, and recognizes the user's emotions to adjust the brightness and saturation of the image to generate an optimally corrected image.

[0452] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0453] Step 1:

[0454] The server initializes the built-in camera and acquires real-time video. The input is the camera image, and the output is real-time video data from the initialized camera module. Specifically, it performs initial settings so that the camera device operates correctly and starts the video stream.

[0455] Step 2:

[0456] The server uses OpenCV's Haar Cascade to detect faces from real-time video frames. The input is real-time video data, and the output is position information (coordinate data) of detected faces. Specifically, it analyzes each frame and identifies the face area based on facial features.

[0457] Step 3:

[0458] The server analyzes the detected upper face region using a convolutional neural network (CNN) with Keras to identify areas with thinning hair. The input is the face's position information and video data of that region, and the output is a mask of the thinning hair area. Specifically, the server evaluates the hair density in the upper face region and recognizes the pattern of thinning hair.

[0459] Step 4:

[0460] The server uses Keras to analyze the user's facial expressions from real-time video and recognize emotions. The input is video data of the face area, and the output is recognized emotion data. Specifically, it extracts facial features and distinguishes between emotional states such as smiling and sad.

[0461] Step 5:

[0462] The server selects hair patches from a data library that match the identified thinning areas and integrates them naturally using Poisson blending technology. The input is a mask of the thinning area and the selected hair patches, and the output is the corrected image data. Specifically, it selects an appropriate hair texture and blends it naturally into the thinning areas.

[0463] Step 6:

[0464] The server adjusts the brightness and saturation of the corrected image based on the emotion recognized by the emotion engine. The input is emotion data and corrected image data, and the output is optimized corrected image data. Specifically, for example, if a smiling emotion is recognized, the brightness and saturation of the image are increased to create a more positive impression.

[0465] Step 7:

[0466] The server displays the image after correction processing is complete on the display, and if the user selects save, it saves the image to local storage. The input is the optimized corrected image data and the user's save instruction, and the output is the saved corrected image data. Specifically, it accepts operations from the user interface and executes the save process.

[0467] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0468] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0469] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0470] [Second embodiment]

[0471] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0472] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0473] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0474] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0475] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0476] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0477] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0478] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0479] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0480] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0481] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0482] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0483] System Overview

[0484] The system of the present invention uses a device equipped with a camera and provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. The system includes a face detection unit, a hair thinning area identification unit, a digital processing unit, and a corrected image storage unit.

[0485] Program processing and explanation

[0486] 1. Launching the app and initializing the camera

[0487] Subject: User, Device

[0488] The user launches the camera app on their smartphone.

[0489] The device initializes the camera module and displays real-time video on the preview screen, activating the device's internal image recognition engine.

[0490] 2. Face detection and head location

[0491] Subject: Terminal

[0492] The device uses image recognition algorithms, such as the Haar detector or deep learning-based FaceNet, to detect human faces from each frame of video.

[0493] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[0494] 3. Analysis and identification of thinning hair areas

[0495] Subject: Terminal

[0496] The device analyzes an image region of the top of the head and runs a convolutional neural network (CNN) algorithm to identify areas of thinning hair based on hair density and texture information.

[0497] A region mask of the identified thinning hair area is generated and its location is recorded.

[0498] 4. Performing digital correction

[0499] Subject: Terminal

[0500] For each identified thinning area, the device selects an appropriate patch from a library of hair patches and digitally processes it to fit the thinning area.

[0501] The digital processing uses Poisson blending techniques to naturally integrate the patches into the original image.

[0502] 5. Saving and displaying the corrected image

[0503] Subject: Terminal, User

[0504] The device stores the corrected image in local storage and displays it to the user.

[0505] Users can view the saved images and share them on social media if desired.

[0506] Specific examples

[0507] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[0508] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[0509] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[0510] 4. The device identifies areas of thinning hair and generates hair in those areas using digital processing means.

[0511] 5. The device saves the digitally enhanced image and displays it to the user, who can then review it and save or share it as desired.

[0512] In this way, the system of the present invention instantly corrects thinning hair on the top of the head or at the hairline in a photograph, allowing you to take photos with confidence. Our invention provides a powerful tool for taking natural-looking photos without worrying about thinning hair.

[0513] The processing flow will be explained below.

[0514] Step 1: Launch the app

[0515] Subject: User

[0516] The user launches the camera app on their smartphone by tapping the icon on the home screen.

[0517] Step 2: Initialize the camera and acquire video

[0518] Subject: Terminal

[0519] The device initializes the camera module, displays the real-time camera image on the preview screen, and starts capturing the video stream.

[0520] Step 3: Face detection

[0521] Subject: Terminal

[0522] The device uses a face detection algorithm (such as a Haar detector or deep learning-based FaceNet) to detect human faces in each frame of video, then locates the detected face and obtains its coordinates.

[0523] Step 4: Parietal region estimation

[0524] Subject: Terminal

[0525] Based on the detected face position, the device estimates the top of the head area, which is located at the top of the face and may contain thinning hair.

[0526] Step 5: Analysis of thinning areas

[0527] Subject: Terminal

[0528] The device analyzes the image area of ​​the top of the head using a convolutional neural network (CNN) to analyze hair density and texture information, thereby identifying areas with thinning hair and generating a region mask.

[0529] Step 6: Prepare for digital correction

[0530] Subject: Terminal

[0531] The device selects a patch of hair from a data library that matches the identified thinning area and prepares to map the selected patch to the thinning area.

[0532] Step 7: Perform digital correction

[0533] Subject: Terminal

[0534] The device uses Poisson blending technology to seamlessly integrate selected hair patches into thinning areas, smoothly connecting different image regions and creating natural-looking boundaries.

[0535] Step 8: Generate the corrected image

[0536] Subject: Terminal

[0537] The device generates an image after the digital correction process is complete and displays it on the preview screen. The correction results are then displayed on the screen for the user to review.

[0538] Step 9: Save the image

[0539] Subject: User, Device

[0540] If the user selects save, the device saves the generated corrected image to local storage and notifies the user that the save is complete.

[0541] Step 10: Share your images

[0542] Subject: User

[0543] Users can view the saved images and share them on social media or other platforms as needed. Images can be easily shared via the share button.

[0544] Through each processing step, images with naturally corrected thinning hair are generated, which can then be saved and shared, helping users take photos with confidence.

[0545] Example 1

[0546] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0547] Conventional camera apps lack the ability to naturally correct thinning hair, preventing many users who are concerned about their thinning hair from taking natural-looking photos. Furthermore, simple image editing software often struggles to correct thinning hair in real time, often resulting in a poor user experience. There is a need for a system that can solve these issues and allow users to enjoy taking photos with confidence.

[0548] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0549] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas with thinning hair, a means for digitally generating hair in the identified areas with thinning hair and correcting the hair to a natural state, a means for saving the corrected image, a means for displaying the image on a preview screen in real time, a means for estimating the detected upper region of the face and acquiring its position coordinates, a means for identifying areas with thinning hair based on the analysis results and drawing the areas as a mask, a means for selecting an optimal patch from a hair patch library, and a means for applying the selected patch to the areas with thinning hair and digitally processing them. This allows users to take natural-looking photos that have been corrected in real time without worrying about thinning hair.

[0550] A "face detection method" is an algorithm or technology for recognizing a person's face in each frame of video and identifying its location.

[0551] The "means for identifying areas of thinning hair" is a technique for analyzing the detected upper face region and identifying areas of thinning hair based on hair density and texture.

[0552] "Digitally generating hair and natural-looking correction" is a process of applying appropriate hair patches to identified thinning areas and integrating them naturally into the original image using digital image processing techniques.

[0553] The "means for saving corrected images" is a function for saving images that have completed processing in local storage or the cloud.

[0554] "Means for displaying video on a preview screen in real time" is a function that displays video captured by a camera in real time, allowing the user to check it immediately.

[0555] The "means for estimating the upper region of the detected face and acquiring its position coordinates" is a technology for calculating the position coordinates of the top of the head and hairline in particular based on the position information of the face.

[0556] "Means for identifying areas of thinning hair based on the analysis results and drawing those areas as a mask" is a function for identifying areas of thinning hair based on the information obtained by analysis and displaying those areas as a mask superimposed on the image.

[0557] The "means for selecting an optimal patch from a hair patch library" refers to an algorithm or technique for selecting an optimal hair patch from a library based on hair color and texture.

[0558] The "means of applying selected patches to the thinning hair area and digitally processing" refers to the process of applying selected hair patches to the thinning hair area and processing them to create a natural-looking composite.

[0559] This invention provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. The system includes a face detection unit, a hair thinning area identification unit, a digital processing unit, and a corrected image storage unit. These units are realized through a real-time video preview screen display unit, a unit for estimating the upper face area and acquiring its coordinates, a unit for identifying the thinning hair area based on the analysis results and drawing the area as a mask, a unit for selecting an optimal patch from a hair patch library, and a unit for applying the selected patch to the thinning hair area and performing digital processing.

[0560] App launch and camera initialization

[0561] The user launches the camera app on their smartphone.

[0562] The device will initialize the camera module and display real-time video on the preview screen. The internal image recognition engine will be ready.

[0563] Face detection and head location

[0564] The device detects human faces from each frame of video using image recognition algorithms, such as the Haar detector or FaceNet.

[0565] The upper area of ​​the detected face (top of the head and hairline area) is estimated and its position coordinates are obtained.

[0566] Analysis and identification of thinning hair areas

[0567] The device analyzes an image region of the top of the head and uses a convolutional neural network (CNN) to identify areas of thinning hair based on hair density and texture information.

[0568] A region mask of the identified thinning hair area is generated and its location is recorded.

[0569] Performing digital correction

[0570] Based on the mask of the bald area, the device selects the best patch from a hair patch library, based on color and texture match criteria.

[0571] The device uses Poisson blending technology to apply the selected patch to the thinning area, allowing it to blend in naturally.

[0572] Saving and displaying the corrected image

[0573] The device saves the corrected image to local storage in a standard image format such as JPEG or PNG.

[0574] The device displays the saved image to the user, allowing them to check it on a preview screen.

[0575] Users can review the images and use the in-app sharing features to share them on social media or with friends if desired.

[0576] Specific examples

[0577] Example 1: Taking a daily selfie

[0578] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[0579] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[0580] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[0581] 4. The terminal identifies areas of thinning hair and generates hair in those areas using digital processing means.

[0582] 5. The device saves the digitally enhanced image and displays it to the user, who can then review it and save or share it as desired.

[0583] Example 2: When taking photos for social media

[0584] 1. The user launches the app to take a photo for social media.

[0585] 2. The device applies filters to the image for social media, while also detecting and correcting faces and areas of baldness using the process described above.

[0586] 3. The device saves the corrected image to local storage and displays it to the user.

[0587] 4. The user uploads this image to a social networking site.

[0588] Example prompts for generative AI models

[0589] "How can I use a camera app to naturally correct thinning hair on the crown and at the hairline?"

[0590] "Please explain the process of using image recognition technology to detect faces and correct hair in real time."

[0591] "Tell me more about how convolutional neural networks can be used to detect thinning hair and digitally generate natural-looking hair."

[0592] As described above, the system according to the present invention provides a powerful tool for users to take natural-looking photos in real time without worrying about thinning hair.

[0593] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0594] Step 1: Launch the app and initialize the camera

[0595] The user launches the camera app on their smartphone.

[0596] Input: User action (app launch)

[0597] The device initializes the camera module and displays real-time video on the preview screen.

[0598] What happens: The device's internal image recognition engine is activated, and the camera settings and capture mode are adjusted appropriately.

[0599] Output: Initialized camera module and started image recognition engine

[0600] Step 2: Detecting the face and identifying the top of the head

[0601] The device detects human faces from each frame of video using an image recognition algorithm (e.g., Haar detector or FaceNet).

[0602] Input: Real-time video frame data

[0603] What it does: Runs an algorithm to determine the location of the face.

[0604] Output: Detected face location information

[0605] Step 3: Identify thinning areas

[0606] The device analyzes the detected upper face area and uses a convolutional neural network (CNN) to identify areas of thinning hair based on hair density and texture information.

[0607] Input: Image data of the upper region of the face

[0608] Specific operation: Apply CNN algorithm to analyze and generate a mask of the thinning hair area.

[0609] Output: Region mask of identified thinning hair areas

[0610] Step 4: Perform digital correction

[0611] Based on the mask of the thinning hair area, the device selects the most suitable patch from a library of hair patches.

[0612] Input: Thinning hair area mask

[0613] What it does: Selects a suitable patch from a patch library based on color and texture match.

[0614] Output: Selected hair patch

[0615] Step 5: Patching and Integration

[0616] The device uses Poisson blending techniques to naturally integrate the selected patches into the original image.

[0617] Input: Selected hair patch, original image data

[0618] What it does: It applies patches using Poisson blending techniques to achieve a natural look.

[0619] Output: Corrected image data

[0620] Step 6: Save and view the corrected image

[0621] The device saves the corrected image in local storage.

[0622] Input: Corrected image data

[0623] Specific behavior: Saves to local storage in a standard image format such as JPEG or PNG.

[0624] Output: Image file saved in local storage

[0625] The device displays the saved image to the user, who can check it on a preview screen.

[0626] Specific operation: Loads the saved image file and displays it on the preview screen.

[0627] Users can view the images and share them on social media if desired.

[0628] Input: Saved image file

[0629] Output: Corrected image displayed in preview window

[0630] Through these processing steps, the system allows users to take natural-looking photos that are corrected in real time without worrying about thinning hair.

[0631] (Application example 1)

[0632] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0633] Currently, many users are concerned about their thinning hair, which can affect their ability to try on hats and hair accessories. To enhance the in-store try-on experience, it is important for users to be able to see how they look in real-life. However, current try-on systems lack the ability to naturally correct thinning hair, potentially damaging users' self-image and confidence. Therefore, there is a need for a system that can naturally correct thinning hair and provide a real-time overall view of how the item will look when trying on.

[0634] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0635] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas of thinning hair, a means for digitally generating hair for the identified areas of thinning hair and correcting it to a natural look, a means for saving the corrected image, a means for displaying the identified areas of thinning hair in real time while keeping the corrected natural look, and a means for displaying, saving, or sharing the corrected image in real time so that the user can check how the clothes will look when tried on. This allows the user to try on clothes with their thinning hair corrected and check how they will look natural. This improves the try-on experience in a physical store and increases the user's confidence in their self-image.

[0636] "Face detection means" refers to technology that automatically identifies a person's face from image data acquired using a camera.

[0637] "Methods for identifying thinning hair areas" refer to techniques that use image analysis to identify areas with particularly low hair density, typically using convolutional neural networks (CNNs).

[0638] "Digitally generating and correcting hair to a natural appearance" refers to a technique for digitally generating and correcting natural-looking hair in a specified thinning area. Poisson blending is commonly used.

[0639] "Means for saving corrected images" refers to a function for saving image data that has been digitally corrected in a storage device of the terminal.

[0640] "Means for displaying in real time" refers to the function of instantly displaying corrected images or videos on the user's screen.

[0641] "Means for users to check the fitting condition" refers to technology that allows users to check the condition of the fitting items they are currently wearing on digital video.

[0642] "Means for displaying, saving, or sharing video in real time" refers to the ability to provide the user with corrected video in real time and save the video for later viewing or share it with others.

[0643] The system based on this invention provides an application that allows users to try on clothes in real time in a physical store while correcting thinning areas of the hair.

[0644] System configuration

[0645] Hardware

[0646] This system utilizes the camera module of mobile devices such as smartphones and tablets. The device camera is used to capture real-time images of the user.

[0647] software

[0648] The software components used include:

[0649] 1. Image recognition software: Face detection is performed using the open source OpenCV library.

[0650] 2. Convolutional Neural Network (CNN) model: This model is used to identify areas of thinning hair on the user, using a pre-trained model such as "hair_thinning_model.h5."

[0651] 3. Poisson Blending Technique: A technique for digitally correcting thinning areas of hair, used to produce natural-looking hair.

[0652] System Operation

[0653] Face detection and analysis

[0654] The device activates the camera and captures the user's video in real time, which is then analyzed through the OpenCV library to determine the face position.

[0655] Identifying thinning areas of hair

[0656] To analyze the upper region of the detected face, we use a convolutional neural network (CNN) model, which evaluates hair density on the crown and identifies areas of thinning hair.

[0657] Digital Correction

[0658] Any identified thinning areas are corrected by digitally generating natural hair using Poisson blending technology, resulting in a natural-looking image.

[0659] Real-time display and storage

[0660] The corrected video is displayed on the device screen in real time, allowing users to enjoy trying on clothes while checking the corrected video. Corrected video and images can also be saved to the device's memory and shared on social media if desired.

[0661] Examples of concrete examples and prompts

[0662] Specific examples

[0663] Users can use the system to film themselves while trying on hats in a physical store, and the system will correct thinning areas of hair in real time to provide a natural-looking image. Users can then check the fit and appearance of the item they are trying on while viewing the corrected image.

[0664] Prompt Sentence Examples

[0665] Use technology that naturally corrects thinning hair to provide the perfect virtual try-on app for users trying on hats. This application detects and corrects thinning hair areas when users take a real-time video of themselves using their smartphone camera. Display this corrected real-time video so users can enjoy trying on hats.

[0666] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0667] Step 1:

[0668] The user launches the app on their smartphone

[0669] Input: Tap the application icon on your smartphone

[0670] Operation: The application starts and the initial setup screen is displayed.

[0671] Output: Real-time camera preview of the smartphone is displayed.

[0672] Step 2:

[0673] The device initializes the camera module and captures real-time video.

[0674] Input: Live video stream from the camera module

[0675] Operation: Initialize the camera module and start video input.

[0676] Output: Video frames captured in real time are stored in memory

[0677] Step 3:

[0678] The device detects your face

[0679] Input: Real-time video frame

[0680] Operation: Performs face detection using the Haar Cascade from the OpenCV library

[0681] Output: Face position coordinates (rectangular area) are obtained

[0682] Step 4:

[0683] The device analyzes the upper area of ​​your face to identify areas with thinning hair.

[0684] Input: Face position coordinates and video frame of the upper region

[0685] How it works: Uses a convolutional neural network (CNN) to assess hair density and identify thinning areas.

[0686] Output: Mask information and position data of thinning hair area are obtained

[0687] Step 5:

[0688] The device performs digital processing on the identified thinning areas.

[0689] Input: Position data of thinning hair area and mask information

[0690] How it works: Using Poisson blending technology, patches of hair are digitally generated and applied to thinning areas.

[0691] Output: Naturally corrected video frames

[0692] Step 6:

[0693] The device displays the corrected image in real time.

[0694] Input: Digitally corrected video frame

[0695] Operation: The corrected image is displayed on the screen in real time.

[0696] Output: Users can check the corrected image on the preview screen.

[0697] Step 7:

[0698] The user saves or shares the corrected footage

[0699] Input: Corrected video frame and user action (choice of saving or sharing)

[0700] How it works: Save the corrected footage to your device's storage or share it via social media or messaging apps

[0701] Output: You will get a saved image file or a shared link or message.

[0702] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0703] System Overview

[0704] The system of the present invention uses a device equipped with a camera and provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. It also combines an emotion engine that recognizes the user's emotions to achieve more optimal correction processing. The system includes a face detection means, a means for identifying thinning hair areas, a digital processing means, an emotion recognition means, and a means for saving the corrected image.

[0705] Program processing and explanation

[0706] 1. Launching the app and initializing the camera

[0707] Subject: User, Device

[0708] The user launches the camera app on their smartphone.

[0709] The device initializes the camera module and displays real-time video on the preview screen, while the emotion engine is also activated.

[0710] 2. Face detection and head location

[0711] Subject: Terminal

[0712] The device uses a face detection algorithm (e.g., Haar detector or deep learning-based FaceNet) to detect human faces from each frame of video.

[0713] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[0714] 3. Analysis and identification of thinning hair areas

[0715] Subject: Terminal

[0716] The device analyzes an image region of the top of the head and runs a convolutional neural network (CNN) algorithm to identify areas of thinning hair based on hair density and texture information.

[0717] A region mask of the identified thinning hair area is generated and its location is recorded.

[0718] 4. Emotion Recognition and Analysis

[0719] Subject: Terminal

[0720] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[0721] 5. Performing digital correction

[0722] Subject: Terminal

[0723] The device selects patches of hair from a data library that match the identified thinning areas.

[0724] The selected patch is mapped onto the thinning hair area and then naturally integrated using Poisson blending technology.

[0725] The emotion engine adjusts the correction based on the emotion it recognizes. For example, if the user is smiling, the image brightness and saturation will be improved.

[0726] 6. Creating and saving the corrected image

[0727] Subject: Terminal, User

[0728] The terminal generates an image after the digital correction process is completed and displays it to the user on a preview screen.

[0729] If the user selects save, the device saves the generated corrected image in local storage. The emotion data recognized by the emotion engine is also saved as history.

[0730] Specific examples

[0731] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[0732] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[0733] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[0734] 4. The device identifies the areas of thinning hair and uses an emotion engine to analyze the user's facial expressions. For example, if the user is smiling, that emotion data is recorded.

[0735] 5. The device digitally processes the thinning hair and generates hair. The brightness and saturation of the image are also adjusted based on the user's emotions.

[0736] 6. The device displays the digitally enhanced image on the preview screen, and if the user selects save, the image is saved to local storage. Emotion data is also saved as history.

[0737] 7. The user can review the saved images and share them on social media if desired.

[0738] Through these steps, the system of the present invention can naturally correct thinning hair on the user's head and hairline, and then use the emotion engine to perform optimal image adjustments, allowing users to take photos with confidence and easily share the corrected images.

[0739] The processing flow will be explained below.

[0740] Step 1: Launch the app and initialize the camera

[0741] Subject: User, Device

[0742] The user launches the camera app on their smartphone.

[0743] The device initializes the camera module and displays the real-time camera image on the preview screen, while simultaneously activating the image recognition engine and emotion engine.

[0744] Step 2: Detecting the face and identifying the top of the head

[0745] Subject: Terminal

[0746] The device runs a face detection algorithm (e.g., Haar detector or FaceNet) on each frame of camera footage to detect human faces.

[0747] Based on the position coordinates of the detected face, the top of the head area (upper part of the face) is estimated and its position coordinates are obtained.

[0748] Step 3: Analysis of thinning areas

[0749] Subject: Terminal

[0750] The device uses a convolutional neural network (CNN) to analyze the hair density and texture of the image area of ​​the top of the head and identify areas of thinning hair.

[0751] A region mask of the identified thinning hair area is generated and its location is recorded.

[0752] Step 4: Recognize emotions

[0753] Subject: Terminal

[0754] The device uses an emotion engine to analyze the user's facial expressions, extracting facial features and recognizing emotions such as smiles and sadness in real time.

[0755] Step 5: Prepare for digital correction

[0756] Subject: Terminal

[0757] The device selects a patch of hair from a data library that matches the thinning area, a selection process that includes assessing the suitability based on the size, color, and texture of the thinning area.

[0758] Step 6: Perform digital correction

[0759] Subject: Terminal

[0760] The device maps selected patches of hair to the thinning areas and uses Poisson blending techniques to naturally integrate the selected patches into the original image.

[0761] The emotion engine adjusts the brightness and saturation of the corrected image based on the emotion it recognizes: for example, if the user is smiling, the brightness and saturation will be increased to make the image appear more positive.

[0762] Step 7: Generate and display the corrected image

[0763] Subject: Terminal

[0764] The device generates an image after the digital correction process is complete and displays it on a preview screen, allowing the user to check the results immediately.

[0765] Step 8: Save the corrected image

[0766] Subject: User, Device

[0767] If the user selects the save option, the device saves the generated corrected image to local storage. The user's emotion data is also saved as history.

[0768] Step 9: Share your images

[0769] Subject: User

[0770] Users can view the saved images and share them on social media or other platforms as desired. Images can be easily shared using the in-app share buttons.

[0771] These steps correct thinning hair areas and provide optimal image adjustments based on the user's emotions, allowing users to take photos with confidence and easily share the corrected images.

[0772] Example 2

[0773] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0774] Conventional camera applications have had the problem of difficulty in achieving a natural-looking result when correcting thinning hair. Furthermore, optimal correction processing that reflects the user's emotions is not performed, and corrected images often do not meet the user's expectations. To solve these problems, a system is needed that can naturally correct thinning hair on the top of the head and at the hairline, and also adjust images based on the user's emotions.

[0775] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0776] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face to identify areas with thinning hair, a means for digitally generating hair in the identified areas with thinning hair and correcting it to a natural state, a means for using an emotion engine that analyzes the user's facial expressions from real-time video and recognizes their emotions, a means for fine-tuning the correction based on the emotions recognized by the emotion engine, and a means for saving the corrected image. This makes it possible to naturally correct thinning hair on the top of the user's head and at the hairline, and enables optimal image adjustment based on the user's emotions.

[0777] "Face detection method" refers to algorithms or devices that detect human faces in video, including Haar detectors and deep learning-based detection techniques.

[0778] "Means for identifying thinning hair areas" refers to an algorithm or device that analyzes the detected upper face region and identifies thinning hair areas based on hair density and texture information. This includes a convolutional neural network (CNN).

[0779] "Digitally generated hair enhancements" refers to algorithms and techniques that apply digital image processing to identified thinning hair areas to enhance the appearance of natural-looking hair, including Poisson blending techniques.

[0780] "Means for using an emotion engine" refers to software or hardware that analyzes a user's facial expressions from real-time video and recognizes their emotions.

[0781] "Means for fine-tuning corrections based on emotions" refers to algorithms and technologies that adjust the brightness, saturation, etc. of the corrected image according to the user's emotions recognized by the emotion engine.

[0782] "Means for storing corrected images" refers to software or devices for storing images that have undergone digital correction processing in local storage or cloud storage.

[0783] The system of the present invention provides a camera application that naturally corrects thinning hair on the top of the user's head and at the hairline. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more optimal correction processing can be achieved. The system includes a face detection unit, a means for identifying thinning hair areas, a digital processing unit, an emotion recognition unit, and a means for saving the corrected image.

[0784] Hardware and software used

[0785] The system's main hardware is a device such as the user's smartphone. The device must be equipped with a camera module and a high-performance CPU or GPU. The software used includes a face detection algorithm using a Haar detector and FaceNet, a hair loss detection algorithm using a convolutional neural network (CNN), a digital correction algorithm using Poisson blending technology, and an emotion engine for emotion recognition.

[0786] Processing flow and specific examples

[0787] 1. Launching the app and initializing the camera

[0788] The user launches a dedicated camera app.

[0789] The device will initialize the camera module, display real-time video on the preview screen, and start the emotion engine.

[0790] 2. Face detection and head location

[0791] The device detects faces from each frame of video using a Haar detector or FaceNet.

[0792] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[0793] 3. Analysis and identification of thinning hair areas

[0794] The device analyzes the image area of ​​the top of the head and runs a CNN algorithm to identify areas of thinning hair based on hair density and texture information.

[0795] A region mask of the identified thinning hair area is generated and its location is recorded.

[0796] 4. Emotion Recognition and Analysis

[0797] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[0798] 5. Performing digital correction

[0799] The device selects patches of hair from a data library that match the identified thinning areas.

[0800] The selected patch is mapped onto the thinning hair area and then naturally integrated using Poisson blending technology.

[0801] The emotion engine fine-tunes the correction based on the emotion it recognizes, for example, if the user is smiling, it will improve the brightness and saturation of the image.

[0802] 6. Creating and saving the corrected image

[0803] The terminal generates an image after the digital correction process is completed and displays it to the user on a preview screen.

[0804] If the user selects save, the device saves the generated corrected image in local storage. The emotion data recognized by the emotion engine is also saved as history.

[0805] Specific examples

[0806] 1. The user launches the camera app and takes a picture of themselves.

[0807] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[0808] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[0809] 4. The device identifies areas of thinning hair and uses an emotion engine to analyze the user's facial expressions, for example, brightening the image if the user is smiling.

[0810] 5. The device digitally processes the thinning areas, using selected patches of hair to create a natural-looking correction. The corrected image is then fine-tuned based on the user's emotions.

[0811] 6. The device displays the generated corrected image on the preview screen, and when the user selects save, the image is saved to local storage. At the same time, emotion data is saved.

[0812] 7. Users can view the saved images and share them on social media etc.

[0813] Prompt Sentence Examples

[0814] "Please analyze real-time images captured by a smartphone camera app, identify the state of the user's face and hair, and then digitally correct any thinning hair to make it look natural. Please provide an example of optimal correction that takes the user's emotions into consideration."

[0815] "Please explain an example of a system that automatically corrects thinning hair in images taken by a user using a camera app, and then analyzes real-time facial expressions to adjust the image based on emotion."

[0816] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0817] Program processing flow

[0818] Step 1:

[0819] App launch and camera initialization

[0820] The user simply launches a dedicated camera app.

[0821] The device will initialize the camera module and display real-time video on the preview screen, while also starting the emotion engine.

[0822] Input: Camera app launch command.

[0823] Output: Real-time camera footage and emotion engine initialization complete.

[0824] Specific operation: When a user launches the app, the device's camera module is activated and the image is previewed in real time. The emotion engine is also initialized in the background.

[0825] Step 2:

[0826] Face detection and head location

[0827] The device detects faces in each frame of video using a face detection algorithm (such as a Haar detector or FaceNet).

[0828] The upper area of ​​the detected face, i.e., the top of the head and hairline, is estimated and its position coordinates are obtained.

[0829] Input: Real-time camera footage.

[0830] Output: Detected face position coordinates and head top region.

[0831] Specific operation: A face detection algorithm is run on each video frame captured by the device's camera to determine the position of the face and the coordinates of the top of the head.

[0832] Step 3:

[0833] Analysis and identification of thinning hair areas

[0834] The device extracts the top and hairline regions from the facial detection results and runs a convolutional neural network (CNN) algorithm to analyze these regions.

[0835] The device identifies areas of thinning hair based on hair density and texture information, generates a region mask for the identified areas of thinning hair, and records their location information.

[0836] Input: parietal region coordinates.

[0837] Output: Mask of thinning hair areas and their locations.

[0838] Specific operation: The image of the top of the head is analyzed using a CNN algorithm to identify areas with low hair density, generate a mask, and record it.

[0839] Step 4:

[0840] Emotion Recognition and Analysis

[0841] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[0842] Input: Real-time video of the user.

[0843] Output: User emotion data.

[0844] Specific operation: Facial features are extracted from real-time video and analyzed by an emotion engine to identify the user's emotions.

[0845] Step 5:

[0846] Performing digital correction

[0847] The device selects patches of hair from a data library that match the identified thinning areas.

[0848] Selected patches of hair are mapped onto thinning areas and then naturally integrated using Poisson blending techniques.

[0849] The emotion engine fine-tunes the correction based on the emotions it recognizes, for example, increasing the brightness and saturation of the image if the user is smiling.

[0850] Input: Mask of thinning hair area, hair patch from data library, emotion data.

[0851] Output: The corrected image.

[0852] Specific operation: Select the appropriate hair patch to match the thinning area, use Poisson blending technology to perform natural correction, and further fine-tune the correction based on the user's emotions.

[0853] Step 6:

[0854] Generate and save the corrected image

[0855] The device generates an image with the digital correction process completed and displays it to the user on a preview screen.

[0856] If the user selects save, the device will save the generated corrected image to local storage, along with the emotion data recognized by the emotion engine.

[0857] Input: Corrected image, user save command.

[0858] Output: Saved corrected image and emotion data.

[0859] Specific operation: The corrected image is displayed on the preview screen, and if the user selects save, the image and emotion data are saved to local storage.

[0860] (Application example 2)

[0861] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0862] In today's world, many people suffer from thinning hair, but technology to naturally correct it is still insufficient. Furthermore, there is a lack of a way to generate optimal images based on the user's emotional state, and an effective method to improve the user experience is needed. Improved customer service using such technology is especially desirable in brick-and-mortar stores.

[0863] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas with thinning hair, a means for digitally generating hair for the identified areas with thinning hair and correcting it to a natural state, a means for analyzing the user's emotions and optimizing the image, and a means for saving the corrected image. This makes it possible to naturally correct the user's thinning hair and generate an optimal image according to the user's emotional state.

[0864] "Face detection means" is a technology for recognizing a person's face in an image and identifying its position.

[0865] The "means for analyzing the detected upper region of the face and identifying areas with thinning hair" is a technology that focuses on the upper region of the face and identifies areas with thinning hair by analyzing the density and condition of the hair within that region.

[0866] The "means of generating hair by digital processing in identified thinning areas and correcting it to a natural state" refers to a technology that uses digital technology to generate hair in areas determined to be thinning hair and then blends it naturally with the surrounding hair.

[0867] "Means for analyzing user emotions and optimizing images" refers to technology that analyzes the user's facial expressions and emotional data and adjusts the brightness, saturation, etc. of the image to an optimal state based on that data.

[0868] The "means for saving the corrected image" refers to a technology for saving the image that has been digitally corrected in the memory or storage of the device.

[0869] The system of the present invention includes a face detection means, a means for analyzing the upper region of the detected face to identify areas of thinning hair, a means for digitally generating hair in the identified areas of thinning hair and correcting it to a natural state, a means for analyzing a user's emotions to optimize the image, and a means for saving the corrected image.

[0870] System Configuration

[0871] Hardware

[0872] Built-in camera: A camera for capturing the user's face in real time and acquiring video data.

[0873] Display: A monitor that displays the corrected image and real-time feedback to the user. It is expected to be installed in fitting rooms in physical stores or beauty salons.

[0874] software

[0875] OpenCV: A library for image processing, used for face detection and video analysis.

[0876] Keras: A library for running deep learning models, used in convolutional neural networks (CNNs) and emotion recognition models.

[0877] Haar Cascade: A classifier for face detection

[0878] Process Overview

[0879] 1. Camera initialization and image acquisition

[0880] The server initializes the built-in camera and captures real-time video, which is then analyzed using face detection means.

[0881] 2. Face Detection

[0882] The server uses OpenCV's Haar Cascade to detect faces in the video, and once a face is detected, its location information is obtained.

[0883] 3. Identify thinning areas

[0884] The detected upper face region is analyzed using a convolutional neural network (CNN) using Keras to identify areas with thinning hair. A mask of the thinning hair region is generated and its location is recorded.

[0885] 4. Emotion Analysis

[0886] The server uses Keras emotion models to analyze the user's facial expressions from real-time video, specifically recognizing emotions such as smile or sadness, and adjusts image correction accordingly.

[0887] 5. Digital Processing

[0888] The server selects patches of hair from a data library that match the identified thinning areas, integrates them naturally using Poisson blending techniques, and adjusts the correction based on emotions recognized by the emotion engine.

[0889] 6. Creating and saving the corrected image

[0890] The image after correction processing is displayed on the screen, and if the user selects save, the image is saved to local storage. Emotion analysis data is also saved as history.

[0891] Specific examples

[0892] When a user stands in front of a smart mirror at a beauty salon, the camera detects the user's face. The upper area of ​​the detected face is analyzed, and areas of thinning hair are identified using CNN. Next, the emotion engine analyzes the user's facial expression, adjusting brightness and saturation if the user is smiling. Finally, an image with the thinning hair corrected is displayed on the screen, and if the user is satisfied, they can save the image.

[0893] Prompt Sentence Examples

[0894] It identifies the user's facial area, detects and corrects thinning hair on the top of the head and at the hairline, and recognizes the user's emotions to adjust the brightness and saturation of the image to generate an optimally corrected image.

[0895] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0896] Step 1:

[0897] The server initializes the built-in camera and acquires real-time video. The input is the camera image, and the output is real-time video data from the initialized camera module. Specifically, it performs initial settings so that the camera device operates correctly and starts the video stream.

[0898] Step 2:

[0899] The server uses OpenCV's Haar Cascade to detect faces from real-time video frames. The input is real-time video data, and the output is position information (coordinate data) of detected faces. Specifically, it analyzes each frame and identifies the face area based on facial features.

[0900] Step 3:

[0901] The server analyzes the detected upper face region using a convolutional neural network (CNN) with Keras to identify areas with thinning hair. The input is the face's position information and video data of that region, and the output is a mask of the thinning hair area. Specifically, the server evaluates the hair density in the upper face region and recognizes the pattern of thinning hair.

[0902] Step 4:

[0903] The server uses Keras to analyze the user's facial expressions from real-time video and recognize emotions. The input is video data of the face area, and the output is recognized emotion data. Specifically, it extracts facial features and distinguishes between emotional states such as smiling and sad.

[0904] Step 5:

[0905] The server selects hair patches from a data library that match the identified thinning areas and integrates them naturally using Poisson blending technology. The input is a mask of the thinning area and the selected hair patches, and the output is the corrected image data. Specifically, it selects an appropriate hair texture and blends it naturally into the thinning areas.

[0906] Step 6:

[0907] The server adjusts the brightness and saturation of the corrected image based on the emotion recognized by the emotion engine. The input is emotion data and corrected image data, and the output is optimized corrected image data. Specifically, for example, if a smiling emotion is recognized, the brightness and saturation of the image are increased to create a more positive impression.

[0908] Step 7:

[0909] The server displays the image after correction processing is complete on the display, and if the user selects save, it saves the image to local storage. The input is the optimized corrected image data and the user's save instruction, and the output is the saved corrected image data. Specifically, it accepts operations from the user interface and executes the save process.

[0910] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0911] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0912] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0913] [Third embodiment]

[0914] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0915] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0916] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0917] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0918] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0919] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0920] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0921] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0922] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0923] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0924] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0925] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0926] System Overview

[0927] The system of the present invention uses a device equipped with a camera and provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. The system includes a face detection unit, a hair thinning area identification unit, a digital processing unit, and a corrected image storage unit.

[0928] Program processing and explanation

[0929] 1. Launching the app and initializing the camera

[0930] Subject: User, Device

[0931] The user launches the camera app on their smartphone.

[0932] The device initializes the camera module and displays real-time video on the preview screen, activating the device's internal image recognition engine.

[0933] 2. Face detection and head location

[0934] Subject: Terminal

[0935] The device uses image recognition algorithms, such as the Haar detector or deep learning-based FaceNet, to detect human faces from each frame of video.

[0936] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[0937] 3. Analysis and identification of thinning hair areas

[0938] Subject: Terminal

[0939] The device analyzes an image region of the top of the head and runs a convolutional neural network (CNN) algorithm to identify areas of thinning hair based on hair density and texture information.

[0940] A region mask of the identified thinning hair area is generated and its location is recorded.

[0941] 4. Performing digital correction

[0942] Subject: Terminal

[0943] For each identified thinning area, the device selects an appropriate patch from a library of hair patches and digitally processes it to fit the thinning area.

[0944] The digital processing uses Poisson blending techniques to naturally integrate the patches into the original image.

[0945] 5. Saving and displaying the corrected image

[0946] Subject: Terminal, User

[0947] The device stores the corrected image in local storage and displays it to the user.

[0948] Users can view the saved images and share them on social media if desired.

[0949] Specific examples

[0950] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[0951] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[0952] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[0953] 4. The device identifies areas of thinning hair and generates hair in those areas using digital processing means.

[0954] 5. The device saves the digitally enhanced image and displays it to the user, who can then review it and save or share it as desired.

[0955] In this way, the system of the present invention instantly corrects thinning hair on the top of the head or at the hairline in a photograph, allowing you to take photos with confidence. Our invention provides a powerful tool for taking natural-looking photos without worrying about thinning hair.

[0956] The processing flow will be explained below.

[0957] Step 1: Launch the app

[0958] Subject: User

[0959] The user launches the camera app on their smartphone by tapping the icon on the home screen.

[0960] Step 2: Initialize the camera and acquire video

[0961] Subject: Terminal

[0962] The device initializes the camera module, displays the real-time camera image on the preview screen, and starts capturing the video stream.

[0963] Step 3: Face detection

[0964] Subject: Terminal

[0965] The device uses a face detection algorithm (such as a Haar detector or deep learning-based FaceNet) to detect human faces in each frame of video, then identifies the location of the detected face and obtains its coordinates.

[0966] Step 4: Parietal region estimation

[0967] Subject: Terminal

[0968] Based on the detected face position, the device estimates the top of the head area, which is located at the top of the face and may contain thinning hair.

[0969] Step 5: Analysis of thinning areas

[0970] Subject: Terminal

[0971] The device analyzes the image area of ​​the top of the head using a convolutional neural network (CNN) to analyze hair density and texture information, thereby identifying areas with thinning hair and generating a region mask.

[0972] Step 6: Prepare for digital correction

[0973] Subject: Terminal

[0974] The device selects a patch of hair from a data library that matches the identified thinning area and prepares to map the selected patch to the thinning area.

[0975] Step 7: Perform digital correction

[0976] Subject: Terminal

[0977] The device uses Poisson blending technology to seamlessly integrate selected hair patches into thinning areas, smoothly connecting different image regions and creating natural-looking boundaries.

[0978] Step 8: Generate the corrected image

[0979] Subject: Terminal

[0980] The device generates an image after the digital correction process is complete and displays it on the preview screen. The correction results are then displayed on the screen for the user to review.

[0981] Step 9: Save the image

[0982] Subject: User, Device

[0983] If the user selects save, the device saves the generated corrected image to local storage and notifies the user that the save is complete.

[0984] Step 10: Share your images

[0985] Subject: User

[0986] Users can view the saved images and share them on social media or other platforms as needed. Images can be easily shared via the share button.

[0987] Through each processing step, images with naturally corrected thinning hair are generated, which can then be saved and shared, giving users the confidence to take photos.

[0988] Example 1

[0989] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0990] Conventional camera apps lack the ability to naturally correct thinning hair, preventing many users who are concerned about their thinning hair from taking natural-looking photos. Furthermore, simple image editing software often struggles to correct thinning hair in real time, often resulting in a poor user experience. There is a need for a system that can solve these issues and allow users to enjoy taking photos with confidence.

[0991] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0992] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas with thinning hair, a means for digitally generating hair in the identified areas with thinning hair and correcting the hair to a natural state, a means for saving the corrected image, a means for displaying the image on a preview screen in real time, a means for estimating the detected upper region of the face and acquiring its position coordinates, a means for identifying areas with thinning hair based on the analysis results and drawing the areas as a mask, a means for selecting an optimal patch from a hair patch library, and a means for applying the selected patch to the areas with thinning hair and digitally processing them. This allows users to take natural-looking photos that have been corrected in real time without worrying about thinning hair.

[0993] A "face detection method" is an algorithm or technology for recognizing a person's face in each frame of video and identifying its location.

[0994] The "means for identifying areas of thinning hair" is a technique for analyzing the detected upper face region and identifying areas of thinning hair based on hair density and texture.

[0995] "Digitally generating hair and natural-looking correction" is a process of applying appropriate hair patches to identified thinning areas and integrating them naturally into the original image using digital image processing techniques.

[0996] The "means for saving corrected images" is a function for saving images that have completed processing in local storage or the cloud.

[0997] "Means for displaying video on a preview screen in real time" is a function that displays video captured by a camera in real time, allowing the user to check it immediately.

[0998] The "means for estimating the upper region of the detected face and acquiring its position coordinates" is a technology for calculating the position coordinates of the top of the head and hairline in particular based on the position information of the face.

[0999] "Means for identifying areas of thinning hair based on the analysis results and drawing those areas as a mask" is a function for identifying areas of thinning hair based on the information obtained by analysis and displaying those areas as a mask superimposed on the image.

[1000] The "means for selecting an optimal patch from a hair patch library" refers to an algorithm or technique for selecting an optimal hair patch from a library based on hair color and texture.

[1001] The "means of applying selected patches to the thinning hair area and digitally processing" refers to the process of applying selected hair patches to the thinning hair area and processing them to create a natural-looking composite.

[1002] This invention provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. The system includes a face detection unit, a hair thinning area identification unit, a digital processing unit, and a corrected image storage unit. These units are realized through a real-time video display on a preview screen, a unit for estimating the upper face area and acquiring its coordinates, a unit for identifying the thinning hair area based on the analysis results and drawing the area as a mask, a unit for selecting an optimal patch from a hair patch library, and a unit for applying the selected patch to the thinning hair area and performing digital processing.

[1003] App launch and camera initialization

[1004] The user launches the camera app on their smartphone.

[1005] The device will initialize the camera module and display real-time video on the preview screen. The internal image recognition engine will be ready.

[1006] Face detection and head location

[1007] The device detects human faces from each frame of video using image recognition algorithms, such as the Haar detector or FaceNet.

[1008] The upper area of ​​the detected face (top of the head and hairline area) is estimated and its position coordinates are obtained.

[1009] Analysis and identification of thinning hair areas

[1010] The device analyzes an image region of the top of the head and uses a convolutional neural network (CNN) to identify areas of thinning hair based on hair density and texture information.

[1011] A region mask of the identified thinning hair area is generated and its location is recorded.

[1012] Performing digital correction

[1013] Based on the mask of the bald area, the device selects the best patch from a hair patch library, based on color and texture match criteria.

[1014] The device uses Poisson blending technology to apply the selected patch to the thinning area, allowing it to blend in naturally.

[1015] Saving and displaying the corrected image

[1016] The device saves the corrected image to local storage in a standard image format such as JPEG or PNG.

[1017] The device displays the saved image to the user, allowing them to check it on a preview screen.

[1018] Users can review the images and use the in-app sharing features to share them on social media or with friends if desired.

[1019] Specific examples

[1020] Example 1: Taking a daily selfie

[1021] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[1022] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[1023] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[1024] 4. The terminal identifies areas of thinning hair and generates hair in those areas using digital processing means.

[1025] 5. The device saves the digitally enhanced image and displays it to the user, who can then review it and save or share it as desired.

[1026] Example 2: When taking photos for social media

[1027] 1. The user launches the app to take a photo for social media.

[1028] 2. The device applies filters to the image for social media, while also detecting and correcting faces and areas of baldness using the process described above.

[1029] 3. The device saves the corrected image to local storage and displays it to the user.

[1030] 4. The user uploads this image to a social networking site.

[1031] Example prompts for generative AI models

[1032] "How can I use a camera app to naturally correct thinning hair on the crown and at the hairline?"

[1033] "Please explain the process of using image recognition technology to detect faces and correct hair in real time."

[1034] "Tell me more about how convolutional neural networks can be used to detect thinning hair and digitally generate natural-looking hair."

[1035] As described above, the system according to the present invention provides a powerful tool for users to take natural-looking photos in real time without worrying about thinning hair.

[1036] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1037] Step 1: Launch the app and initialize the camera

[1038] The user launches the camera app on their smartphone.

[1039] Input: User action (app launch)

[1040] The device initializes the camera module and displays real-time video on the preview screen.

[1041] What happens: The device's internal image recognition engine is activated, and the camera settings and capture mode are adjusted appropriately.

[1042] Output: Initialized camera module and started image recognition engine

[1043] Step 2: Detecting the face and identifying the top of the head

[1044] The device detects human faces from each frame of video using an image recognition algorithm (e.g., Haar detector or FaceNet).

[1045] Input: Real-time video frame data

[1046] What it does: Runs an algorithm to determine the location of the face.

[1047] Output: Detected face location information

[1048] Step 3: Identify thinning areas

[1049] The device analyzes the detected upper face area and uses a convolutional neural network (CNN) to identify areas of thinning hair based on hair density and texture information.

[1050] Input: Image data of the upper region of the face

[1051] Specific operation: Apply CNN algorithm to analyze and generate a mask of the thinning hair area.

[1052] Output: Region mask of identified thinning hair areas

[1053] Step 4: Perform digital correction

[1054] Based on the mask of the thinning hair area, the device selects the most suitable patch from a library of hair patches.

[1055] Input: Thinning hair area mask

[1056] What it does: Selects a suitable patch from a patch library based on color and texture match.

[1057] Output: Selected hair patch

[1058] Step 5: Patching and Integration

[1059] The device uses Poisson blending techniques to naturally integrate the selected patches into the original image.

[1060] Input: Selected hair patch, original image data

[1061] What it does: It applies patches using Poisson blending techniques to achieve a natural look.

[1062] Output: Corrected image data

[1063] Step 6: Save and view the corrected image

[1064] The device saves the corrected image in local storage.

[1065] Input: Corrected image data

[1066] Specific behavior: Saves to local storage in a standard image format such as JPEG or PNG.

[1067] Output: Image file saved in local storage

[1068] The device displays the saved image to the user, who can check it on a preview screen.

[1069] Specific operation: Loads the saved image file and displays it on the preview screen.

[1070] Users can view the images and share them on social media if desired.

[1071] Input: Saved image file

[1072] Output: Corrected image displayed in preview window

[1073] Through these processing steps, the system allows users to take natural-looking photos that are corrected in real time without worrying about thinning hair.

[1074] (Application example 1)

[1075] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1076] Currently, many users are concerned about their thinning hair, which can affect their ability to try on hats and hair accessories. To enhance the in-store try-on experience, it is important for users to be able to see how they look in real-life. However, current try-on systems lack the ability to naturally correct thinning hair, potentially damaging users' self-image and confidence. Therefore, there is a need for a system that can naturally correct thinning hair and provide a real-time overall view of how the item will look when trying on.

[1077] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1078] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas of thinning hair, a means for digitally generating hair for the identified areas of thinning hair and correcting it to a natural look, a means for saving the corrected image, a means for displaying the identified areas of thinning hair in real time while keeping the corrected natural look, and a means for displaying, saving, or sharing the corrected image in real time so that the user can check how the clothes will look when tried on. This allows the user to try on clothes with their thinning hair corrected and check how they will look natural. This improves the try-on experience in a physical store and increases the user's confidence in their self-image.

[1079] "Face detection means" refers to technology that automatically identifies a person's face from image data acquired using a camera.

[1080] "Methods for identifying thinning hair areas" refer to techniques that use image analysis to identify areas with particularly low hair density, typically using convolutional neural networks (CNNs).

[1081] "Digitally generating and correcting hair to a natural appearance" refers to a technique for digitally generating and correcting natural-looking hair in a specified thinning area. Poisson blending is commonly used.

[1082] "Means for saving corrected images" refers to a function for saving image data that has been digitally corrected in a storage device of the terminal.

[1083] "Means for displaying in real time" refers to the function of instantly displaying corrected images or videos on the user's screen.

[1084] "Means for users to check the fitting condition" refers to technology that allows users to check the condition of the fitting items they are currently wearing on digital video.

[1085] "Means for displaying, saving, or sharing video in real time" refers to the ability to provide the user with corrected video in real time and save the video for later viewing or share it with others.

[1086] The system based on this invention provides an application that allows users to try on clothes in real time in a physical store while correcting thinning areas of the hair.

[1087] System configuration

[1088] Hardware

[1089] This system utilizes the camera module of mobile devices such as smartphones and tablets. The device camera is used to capture real-time images of the user.

[1090] software

[1091] The software components used include:

[1092] 1. Image recognition software: Face detection is performed using the open source OpenCV library.

[1093] 2. Convolutional Neural Network (CNN) model: This model is used to identify areas of thinning hair on the user, using a pre-trained model such as "hair_thinning_model.h5."

[1094] 3. Poisson Blending Technique: A technique for digitally correcting thinning areas of hair, used to produce natural-looking hair.

[1095] System Operation

[1096] Face detection and analysis

[1097] The device activates the camera and captures the user's video in real time, which is then analyzed through the OpenCV library to determine the face position.

[1098] Identifying thinning areas of hair

[1099] To analyze the upper region of the detected face, we use a convolutional neural network (CNN) model, which evaluates hair density on the crown and identifies areas of thinning hair.

[1100] Digital Correction

[1101] Any identified thinning areas are corrected by digitally generating natural hair using Poisson blending technology, resulting in a natural-looking image.

[1102] Real-time display and storage

[1103] The corrected video is displayed on the device screen in real time, allowing users to enjoy trying on clothes while checking the corrected video. Corrected video and images can also be saved to the device's memory and shared on social media if desired.

[1104] Examples of concrete examples and prompts

[1105] Specific examples

[1106] Users can use the system to film themselves while trying on hats in a physical store, and the system will correct thinning areas of hair in real time to provide a natural-looking image. Users can then check the fit and appearance of the item they are trying on while viewing the corrected image.

[1107] Prompt Sentence Examples

[1108] Use technology that naturally corrects thinning hair to provide the perfect virtual try-on app for users trying on hats. This application detects and corrects thinning hair areas when users take a real-time video of themselves using their smartphone camera. Display this corrected real-time video so users can enjoy trying on hats.

[1109] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1110] Step 1:

[1111] The user launches the app on their smartphone

[1112] Input: Tap the application icon on your smartphone

[1113] Operation: The application starts and the initial setup screen is displayed.

[1114] Output: Real-time camera preview of the smartphone is displayed.

[1115] Step 2:

[1116] The device initializes the camera module and captures real-time video.

[1117] Input: Live video stream from the camera module

[1118] Operation: Initialize the camera module and start video input.

[1119] Output: Video frames captured in real time are stored in memory

[1120] Step 3:

[1121] The device detects your face

[1122] Input: Real-time video frame

[1123] Operation: Performs face detection using the Haar Cascade from the OpenCV library

[1124] Output: Face position coordinates (rectangular area) are obtained

[1125] Step 4:

[1126] The device analyzes the upper area of ​​your face to identify areas with thinning hair.

[1127] Input: Face position coordinates and video frame of the upper region

[1128] How it works: Uses a convolutional neural network (CNN) to assess hair density and identify thinning areas.

[1129] Output: Mask information and position data of thinning hair area are obtained

[1130] Step 5:

[1131] The device performs digital processing on the identified thinning areas.

[1132] Input: Position data of thinning hair area and mask information

[1133] How it works: Using Poisson blending technology, patches of hair are digitally generated and applied to thinning areas.

[1134] Output: Naturally corrected video frames

[1135] Step 6:

[1136] The device displays the corrected image in real time.

[1137] Input: Digitally corrected video frame

[1138] Operation: The corrected image is displayed on the screen in real time.

[1139] Output: Users can check the corrected image on the preview screen.

[1140] Step 7:

[1141] The user saves or shares the corrected footage

[1142] Input: Corrected video frame and user action (choice of saving or sharing)

[1143] How it works: Save the corrected footage to your device's storage or share it via social media or messaging apps

[1144] Output: You will get a saved image file or a shared link or message.

[1145] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1146] System Overview

[1147] The system of the present invention uses a device equipped with a camera and provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. It also combines an emotion engine that recognizes the user's emotions to achieve more optimal correction processing. The system includes a face detection means, a means for identifying thinning hair areas, a digital processing means, an emotion recognition means, and a means for saving the corrected image.

[1148] Program processing and explanation

[1149] 1. Launching the app and initializing the camera

[1150] Subject: User, Device

[1151] The user launches the camera app on their smartphone.

[1152] The device initializes the camera module and displays real-time video on the preview screen, while the emotion engine is also activated.

[1153] 2. Face detection and head location

[1154] Subject: Terminal

[1155] The device uses a face detection algorithm (e.g., Haar detector or deep learning-based FaceNet) to detect human faces from each frame of video.

[1156] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[1157] 3. Analysis and identification of thinning hair areas

[1158] Subject: Terminal

[1159] The device analyzes an image region of the top of the head and runs a convolutional neural network (CNN) algorithm to identify areas of thinning hair based on hair density and texture information.

[1160] A region mask of the identified thinning hair area is generated and its location is recorded.

[1161] 4. Emotion Recognition and Analysis

[1162] Subject: Terminal

[1163] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[1164] 5. Performing digital correction

[1165] Subject: Terminal

[1166] The device selects patches of hair from a data library that match the identified thinning areas.

[1167] The selected patch is mapped onto the thinning hair area and then naturally integrated using Poisson blending technology.

[1168] The emotion engine adjusts the correction based on the emotion it recognizes. For example, if the user is smiling, the image brightness and saturation will be improved.

[1169] 6. Creating and saving the corrected image

[1170] Subject: Terminal, User

[1171] The terminal generates an image after the digital correction process is completed and displays it to the user on a preview screen.

[1172] If the user selects save, the device saves the generated corrected image in local storage. The emotion data recognized by the emotion engine is also saved as history.

[1173] Specific examples

[1174] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[1175] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[1176] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[1177] 4. The device identifies the areas of thinning hair and uses an emotion engine to analyze the user's facial expressions. For example, if the user is smiling, that emotion data is recorded.

[1178] 5. The device digitally processes the thinning hair and generates hair. The brightness and saturation of the image are also adjusted based on the user's emotions.

[1179] 6. The device displays the digitally enhanced image on the preview screen, and if the user selects save, the image is saved to local storage. Emotion data is also saved as history.

[1180] 7. The user can review the saved images and share them on social media if desired.

[1181] Through these steps, the system of the present invention can naturally correct thinning hair on the user's head and hairline, and then use the emotion engine to perform optimal image adjustments, allowing users to take photos with confidence and easily share the corrected images.

[1182] The processing flow will be explained below.

[1183] Step 1: Launch the app and initialize the camera

[1184] Subject: User, Device

[1185] The user launches the camera app on their smartphone.

[1186] The device initializes the camera module and displays the real-time camera image on the preview screen, while simultaneously activating the image recognition engine and emotion engine.

[1187] Step 2: Detecting the face and identifying the top of the head

[1188] Subject: Terminal

[1189] The device runs a face detection algorithm (e.g., Haar detector or FaceNet) on each frame of camera footage to detect human faces.

[1190] Based on the position coordinates of the detected face, the top of the head area (upper part of the face) is estimated and its position coordinates are obtained.

[1191] Step 3: Analysis of thinning areas

[1192] Subject: Terminal

[1193] The device uses a convolutional neural network (CNN) to analyze the hair density and texture of the image area of ​​the top of the head and identify areas of thinning hair.

[1194] A region mask of the identified thinning hair area is generated and its location is recorded.

[1195] Step 4: Recognize emotions

[1196] Subject: Terminal

[1197] The device uses an emotion engine to analyze the user's facial expressions, extracting facial features and recognizing emotions such as smiles and sadness in real time.

[1198] Step 5: Prepare for digital correction

[1199] Subject: Terminal

[1200] The device selects a patch of hair from a data library that matches the thinning area, a selection process that includes assessing the suitability based on the size, color, and texture of the thinning area.

[1201] Step 6: Perform digital correction

[1202] Subject: Terminal

[1203] The device maps selected patches of hair to the thinning areas and uses Poisson blending techniques to naturally integrate the selected patches into the original image.

[1204] The emotion engine adjusts the brightness and saturation of the corrected image based on the emotion it recognizes: for example, if the user is smiling, the brightness and saturation will be increased to make the image appear more positive.

[1205] Step 7: Generate and display the corrected image

[1206] Subject: Terminal

[1207] The device generates an image after the digital correction process is complete and displays it on a preview screen, allowing the user to check the results immediately.

[1208] Step 8: Save the corrected image

[1209] Subject: User, Device

[1210] If the user selects the save option, the device saves the generated corrected image to local storage. The user's emotion data is also saved as history.

[1211] Step 9: Share your images

[1212] Subject: User

[1213] Users can view the saved images and share them on social media or other platforms as desired. Images can be easily shared using the in-app share buttons.

[1214] These steps correct thinning hair areas and provide optimal image adjustments based on the user's emotions, allowing users to take photos with confidence and easily share the corrected images.

[1215] Example 2

[1216] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1217] Conventional camera applications have had the problem of difficulty in achieving a natural-looking result when correcting thinning hair. Furthermore, optimal correction processing that reflects the user's emotions is not performed, and corrected images often do not meet the user's expectations. To solve these problems, a system is needed that can naturally correct thinning hair on the top of the head and at the hairline, and also adjust images based on the user's emotions.

[1218] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1219] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face to identify areas with thinning hair, a means for digitally generating hair in the identified areas with thinning hair and correcting it to a natural state, a means for using an emotion engine that analyzes the user's facial expressions from real-time video and recognizes their emotions, a means for fine-tuning the correction based on the emotions recognized by the emotion engine, and a means for saving the corrected image. This makes it possible to naturally correct thinning hair on the top of the user's head and at the hairline, and enables optimal image adjustment based on the user's emotions.

[1220] "Face detection method" refers to algorithms or devices that detect human faces in video, including Haar detectors and deep learning-based detection techniques.

[1221] "Means for identifying thinning hair areas" refers to an algorithm or device that analyzes the detected upper face region and identifies thinning hair areas based on hair density and texture information. This includes a convolutional neural network (CNN).

[1222] "Digitally generated hair enhancements" refers to algorithms and techniques that apply digital image processing to identified thinning hair areas to enhance the appearance of natural-looking hair, including Poisson blending techniques.

[1223] "Means for using an emotion engine" refers to software or hardware that analyzes a user's facial expressions from real-time video and recognizes their emotions.

[1224] "Means for fine-tuning corrections based on emotions" refers to algorithms and technologies that adjust the brightness, saturation, etc. of the corrected image according to the user's emotions recognized by the emotion engine.

[1225] "Means for storing corrected images" refers to software or devices for storing images that have undergone digital correction processing in local storage or cloud storage.

[1226] The system of the present invention provides a camera application that naturally corrects thinning hair on the top of the user's head and at the hairline. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more optimal correction processing can be achieved. The system includes a face detection unit, a means for identifying thinning hair areas, a digital processing unit, an emotion recognition unit, and a means for saving the corrected image.

[1227] Hardware and software used

[1228] The system's main hardware is a device such as the user's smartphone. The device must be equipped with a camera module and a high-performance CPU or GPU. The software used includes a face detection algorithm using a Haar detector and FaceNet, a hair loss detection algorithm using a convolutional neural network (CNN), a digital correction algorithm using Poisson blending technology, and an emotion engine for emotion recognition.

[1229] Processing flow and specific examples

[1230] 1. Launching the app and initializing the camera

[1231] The user launches a dedicated camera app.

[1232] The device will initialize the camera module, display real-time video on the preview screen, and start the emotion engine.

[1233] 2. Face detection and head location

[1234] The device detects faces from each frame of video using a Haar detector or FaceNet.

[1235] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[1236] 3. Analysis and identification of thinning hair areas

[1237] The device analyzes the image area of ​​the top of the head and runs a CNN algorithm to identify areas of thinning hair based on hair density and texture information.

[1238] A region mask of the identified thinning hair area is generated and its location is recorded.

[1239] 4. Emotion Recognition and Analysis

[1240] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[1241] 5. Performing digital correction

[1242] The device selects patches of hair from a data library that match the identified thinning areas.

[1243] The selected patch is mapped onto the thinning hair area and then naturally integrated using Poisson blending technology.

[1244] The emotion engine fine-tunes the correction based on the emotion it recognizes, for example, if the user is smiling, it will improve the brightness and saturation of the image.

[1245] 6. Creating and saving the corrected image

[1246] The terminal generates an image after the digital correction process is completed and displays it to the user on a preview screen.

[1247] If the user selects save, the device saves the generated corrected image in local storage. The emotion data recognized by the emotion engine is also saved as history.

[1248] Specific examples

[1249] 1. The user launches the camera app and takes a picture of themselves.

[1250] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[1251] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[1252] 4. The device identifies areas of thinning hair and uses an emotion engine to analyze the user's facial expressions, for example, brightening the image if the user is smiling.

[1253] 5. The device digitally processes the thinning areas, using selected patches of hair to create a natural-looking correction. The corrected image is then fine-tuned based on the user's emotions.

[1254] 6. The device displays the generated corrected image on the preview screen, and when the user selects save, the image is saved to local storage. At the same time, emotion data is saved.

[1255] 7. Users can view the saved images and share them on social media etc.

[1256] Prompt Sentence Examples

[1257] "Please analyze real-time images captured by a smartphone camera app, identify the state of the user's face and hair, and then digitally correct any thinning hair to make it look natural. Please provide an example of optimal correction that takes the user's emotions into consideration."

[1258] "Please explain an example of a system that automatically corrects thinning hair in images taken by a user using a camera app, and then analyzes real-time facial expressions to adjust the image based on emotion."

[1259] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1260] Program processing flow

[1261] Step 1:

[1262] App launch and camera initialization

[1263] The user simply launches a dedicated camera app.

[1264] The device will initialize the camera module and display real-time video on the preview screen, while also starting the emotion engine.

[1265] Input: Camera app launch command.

[1266] Output: Real-time camera footage and emotion engine initialization complete.

[1267] Specific operation: When a user launches the app, the device's camera module is activated and the image is previewed in real time. The emotion engine is also initialized in the background.

[1268] Step 2:

[1269] Face detection and head location

[1270] The device detects faces in each frame of video using a face detection algorithm (such as a Haar detector or FaceNet).

[1271] The upper area of ​​the detected face, i.e., the top of the head and hairline, is estimated and its position coordinates are obtained.

[1272] Input: Real-time camera footage.

[1273] Output: Detected face position coordinates and head top region.

[1274] Specific operation: A face detection algorithm is run on each video frame captured by the device's camera to determine the position of the face and the coordinates of the top of the head.

[1275] Step 3:

[1276] Analysis and identification of thinning hair areas

[1277] The device extracts the top and hairline regions from the facial detection results and runs a convolutional neural network (CNN) algorithm to analyze these regions.

[1278] The device identifies areas of thinning hair based on hair density and texture information, generates a region mask for the identified areas of thinning hair, and records their location information.

[1279] Input: parietal region coordinates.

[1280] Output: Mask of thinning hair areas and their locations.

[1281] Specific operation: The image of the top of the head is analyzed using a CNN algorithm to identify areas with low hair density, generate a mask, and record it.

[1282] Step 4:

[1283] Emotion Recognition and Analysis

[1284] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[1285] Input: Real-time video of the user.

[1286] Output: User emotion data.

[1287] Specific operation: Facial features are extracted from real-time video and analyzed by an emotion engine to identify the user's emotions.

[1288] Step 5:

[1289] Performing digital correction

[1290] The device selects patches of hair from a data library that match the identified thinning areas.

[1291] Selected patches of hair are mapped onto thinning areas and then naturally integrated using Poisson blending techniques.

[1292] The emotion engine fine-tunes the correction based on the emotions it recognizes, for example, increasing the brightness and saturation of the image if the user is smiling.

[1293] Input: Mask of thinning hair area, hair patch from data library, emotion data.

[1294] Output: The corrected image.

[1295] Specific operation: Select the appropriate hair patch to match the thinning area, use Poisson blending technology to perform natural correction, and further fine-tune the correction based on the user's emotions.

[1296] Step 6:

[1297] Generate and save the corrected image

[1298] The device generates an image with the digital correction process completed and displays it to the user on a preview screen.

[1299] If the user selects save, the device will save the generated corrected image to local storage, along with the emotion data recognized by the emotion engine.

[1300] Input: Corrected image, user save command.

[1301] Output: Saved corrected image and emotion data.

[1302] Specific operation: The corrected image is displayed on the preview screen, and if the user selects save, the image and emotion data are saved to local storage.

[1303] (Application example 2)

[1304] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1305] In today's world, many people suffer from thinning hair, but technology to naturally correct it is still insufficient. Furthermore, there is a lack of a way to generate optimal images based on the user's emotional state, and an effective method to improve the user experience is needed. Improved customer service using such technology is especially desirable in brick-and-mortar stores.

[1306] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas with thinning hair, a means for digitally generating hair for the identified areas with thinning hair and correcting it to a natural state, a means for analyzing the user's emotions and optimizing the image, and a means for saving the corrected image. This makes it possible to naturally correct the user's thinning hair and generate an optimal image according to the user's emotional state.

[1307] "Face detection means" is a technology for recognizing a person's face in an image and identifying its position.

[1308] The "means for analyzing the detected upper region of the face and identifying areas with thinning hair" is a technology that focuses on the upper region of the face and identifies areas with thinning hair by analyzing the density and condition of the hair within that region.

[1309] The "means of generating hair by digital processing in identified thinning areas and correcting it to a natural state" refers to a technology that uses digital technology to generate hair in areas determined to be thinning hair and then blends it naturally with the surrounding hair.

[1310] "Means for analyzing user emotions and optimizing images" refers to technology that analyzes the user's facial expressions and emotional data and adjusts the brightness, saturation, etc. of the image to an optimal state based on that data.

[1311] The "means for saving the corrected image" refers to a technology for saving the image that has been digitally corrected in the memory or storage of the device.

[1312] The system of the present invention includes a face detection means, a means for analyzing the upper region of the detected face to identify areas of thinning hair, a means for digitally generating hair in the identified areas of thinning hair and correcting it to a natural state, a means for analyzing a user's emotions to optimize the image, and a means for saving the corrected image.

[1313] System Configuration

[1314] Hardware

[1315] Built-in camera: A camera for capturing the user's face in real time and acquiring video data.

[1316] Display: A monitor that displays the corrected image and real-time feedback to the user. It is expected to be installed in fitting rooms in physical stores or beauty salons.

[1317] software

[1318] OpenCV: A library for image processing, used for face detection and video analysis.

[1319] Keras: A library for running deep learning models, used in convolutional neural networks (CNNs) and emotion recognition models.

[1320] Haar Cascade: A classifier for face detection

[1321] Process Overview

[1322] 1. Camera initialization and image acquisition

[1323] The server initializes the built-in camera and captures real-time video, which is then analyzed using face detection means.

[1324] 2. Face Detection

[1325] The server uses OpenCV's Haar Cascade to detect faces in the video, and once a face is detected, its location information is obtained.

[1326] 3. Identify thinning areas

[1327] The detected upper face region is analyzed using a convolutional neural network (CNN) using Keras to identify areas with thinning hair. A mask of the thinning hair region is generated and its location is recorded.

[1328] 4. Emotion Analysis

[1329] The server uses Keras emotion models to analyze the user's facial expressions from real-time video, specifically recognizing emotions such as smile or sadness, and adjusts image correction accordingly.

[1330] 5. Digital Processing

[1331] The server selects patches of hair from a data library that match the identified thinning areas, integrates them naturally using Poisson blending techniques, and adjusts the correction based on emotions recognized by the emotion engine.

[1332] 6. Creating and saving the corrected image

[1333] The image after correction processing is displayed on the screen, and if the user selects save, the image is saved to local storage. Emotion analysis data is also saved as history.

[1334] Specific examples

[1335] When a user stands in front of a smart mirror at a beauty salon, the camera detects the user's face. The upper area of ​​the detected face is analyzed, and areas of thinning hair are identified using CNN. Next, the emotion engine analyzes the user's facial expression, adjusting brightness and saturation if the user is smiling. Finally, an image with the thinning hair corrected is displayed on the screen, and if the user is satisfied, they can save the image.

[1336] Prompt Sentence Examples

[1337] It identifies the user's facial area, detects and corrects thinning hair on the top of the head and at the hairline, and recognizes the user's emotions to adjust the brightness and saturation of the image to generate an optimally corrected image.

[1338] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1339] Step 1:

[1340] The server initializes the built-in camera and acquires real-time video. The input is the camera image, and the output is real-time video data from the initialized camera module. Specifically, it performs initial settings so that the camera device operates correctly and starts the video stream.

[1341] Step 2:

[1342] The server uses OpenCV's Haar Cascade to detect faces from real-time video frames. The input is real-time video data, and the output is position information (coordinate data) of detected faces. Specifically, it analyzes each frame and identifies the face area based on facial features.

[1343] Step 3:

[1344] The server analyzes the detected upper face region using a convolutional neural network (CNN) with Keras to identify areas with thinning hair. The input is the face's position information and video data of that region, and the output is a mask of the thinning hair area. Specifically, the server evaluates the hair density in the upper face region and recognizes the pattern of thinning hair.

[1345] Step 4:

[1346] The server uses Keras to analyze the user's facial expressions from real-time video and recognize emotions. The input is video data of the face area, and the output is recognized emotion data. Specifically, it extracts facial features and distinguishes between emotional states such as smiling and sad.

[1347] Step 5:

[1348] The server selects hair patches from a data library that match the identified thinning areas and integrates them naturally using Poisson blending technology. The input is a mask of the thinning area and the selected hair patches, and the output is the corrected image data. Specifically, it selects an appropriate hair texture and blends it naturally into the thinning areas.

[1349] Step 6:

[1350] The server adjusts the brightness and saturation of the corrected image based on the emotion recognized by the emotion engine. The input is emotion data and corrected image data, and the output is optimized corrected image data. Specifically, for example, if a smiling emotion is recognized, the brightness and saturation of the image are increased to create a more positive impression.

[1351] Step 7:

[1352] The server displays the image after correction processing is complete on the display, and if the user selects save, it saves the image to local storage. The input is the optimized corrected image data and the user's save instruction, and the output is the saved corrected image data. Specifically, it accepts operations from the user interface and executes the save process.

[1353] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1354] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1355] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1356] [Fourth embodiment]

[1357] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1358] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1359] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1360] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1361] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1362] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1363] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1364] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1365] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1366] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1367] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1368] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1369] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1370] System Overview

[1371] The system of the present invention uses a device equipped with a camera and provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. The system includes a face detection unit, a hair thinning area identification unit, a digital processing unit, and a corrected image storage unit.

[1372] Program processing and explanation

[1373] 1. Launching the app and initializing the camera

[1374] Subject: User, Device

[1375] The user launches the camera app on their smartphone.

[1376] The device initializes the camera module and displays real-time video on the preview screen, activating the device's internal image recognition engine.

[1377] 2. Face detection and head location

[1378] Subject: Terminal

[1379] The device uses image recognition algorithms, such as the Haar detector or deep learning-based FaceNet, to detect human faces from each frame of video.

[1380] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[1381] 3. Analysis and identification of thinning hair areas

[1382] Subject: Terminal

[1383] The device analyzes an image region of the top of the head and runs a convolutional neural network (CNN) algorithm to identify areas of thinning hair based on hair density and texture information.

[1384] A region mask of the identified thinning hair area is generated and its location is recorded.

[1385] 4. Performing digital correction

[1386] Subject: Terminal

[1387] For each identified thinning area, the device selects an appropriate patch from a library of hair patches and digitally processes it to fit the thinning area.

[1388] The digital processing uses Poisson blending techniques to naturally integrate the patches into the original image.

[1389] 5. Saving and displaying the corrected image

[1390] Subject: Terminal, User

[1391] The device stores the corrected image in local storage and displays it to the user.

[1392] Users can view the saved images and share them on social media if desired.

[1393] Specific examples

[1394] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[1395] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[1396] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[1397] 4. The device identifies areas of thinning hair and generates hair in those areas using digital processing means.

[1398] 5. The device saves the digitally enhanced image and displays it to the user, who can then review it and save or share it as desired.

[1399] In this way, the system of the present invention instantly corrects thinning hair on the top of the head or at the hairline in a photograph, allowing you to take photos with confidence. Our invention provides a powerful tool for taking natural-looking photos without worrying about thinning hair.

[1400] The processing flow will be explained below.

[1401] Step 1: Launch the app

[1402] Subject: User

[1403] The user launches the camera app on their smartphone by tapping the icon on the home screen.

[1404] Step 2: Initialize the camera and acquire video

[1405] Subject: Terminal

[1406] The device initializes the camera module, displays the real-time camera image on the preview screen, and starts capturing the video stream.

[1407] Step 3: Face detection

[1408] Subject: Terminal

[1409] The device uses a face detection algorithm (such as a Haar detector or deep learning-based FaceNet) to detect human faces in each frame of video, then identifies the location of the detected face and obtains its coordinates.

[1410] Step 4: Parietal region estimation

[1411] Subject: Terminal

[1412] Based on the detected face position, the device estimates the top of the head area, which is located at the top of the face and may contain thinning hair.

[1413] Step 5: Analysis of thinning areas

[1414] Subject: Terminal

[1415] The device analyzes the image area of ​​the top of the head using a convolutional neural network (CNN) to analyze hair density and texture information, thereby identifying areas with thinning hair and generating a region mask.

[1416] Step 6: Prepare for digital correction

[1417] Subject: Terminal

[1418] The device selects a patch of hair from a data library that matches the identified thinning area and prepares to map the selected patch to the thinning area.

[1419] Step 7: Perform digital correction

[1420] Subject: Terminal

[1421] The device uses Poisson blending technology to seamlessly integrate selected hair patches into thinning areas, smoothly connecting different image regions and creating natural-looking boundaries.

[1422] Step 8: Generate the corrected image

[1423] Subject: Terminal

[1424] The device generates an image after the digital correction process is complete and displays it on the preview screen. The correction results are then displayed on the screen for the user to review.

[1425] Step 9: Save the image

[1426] Subject: User, Device

[1427] If the user selects save, the device saves the generated corrected image to local storage and notifies the user that the save is complete.

[1428] Step 10: Share your images

[1429] Subject: User

[1430] Users can view the saved images and share them on social media or other platforms as needed. Images can be easily shared via the share button.

[1431] Through each processing step, images with naturally corrected thinning hair are generated, which can then be saved and shared, giving users the confidence to take photos.

[1432] Example 1

[1433] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1434] Conventional camera apps lack the ability to naturally correct thinning hair, preventing many users who are concerned about their thinning hair from taking natural-looking photos. Furthermore, simple image editing software often struggles to correct thinning hair in real time, often resulting in a poor user experience. There is a need for a system that can solve these issues and allow users to enjoy taking photos with confidence.

[1435] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1436] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas where hair is thinning, a means for digitally generating hair in the identified areas where hair is thinning and correcting the image to a natural state, a means for saving the corrected image, a means for displaying the image on a preview screen in real time, a means for estimating the detected upper region of the face and acquiring its position coordinates, a means for identifying areas where hair is thinning based on the analysis results and drawing the areas as a mask, a means for selecting an optimal patch from a hair patch library, and a means for applying the selected patch to the areas where hair is thinning and digitally processing the image. This allows the user to take natural-looking photos that have been corrected in real time without worrying about thinning hair.

[1437] A "face detection method" is an algorithm or technology for recognizing a person's face in each frame of video and identifying its location.

[1438] The "means for identifying areas of thinning hair" is a technique for analyzing the detected upper face region and identifying areas of thinning hair based on hair density and texture.

[1439] "Digitally generating hair and natural-looking correction" is a process of applying appropriate hair patches to identified thinning areas and integrating them naturally into the original image using digital image processing techniques.

[1440] The "means for saving corrected images" is a function for saving images that have completed processing in local storage or the cloud.

[1441] "Means for displaying video on a preview screen in real time" is a function that displays video captured by a camera in real time, allowing the user to check it immediately.

[1442] The "means for estimating the upper region of the detected face and acquiring its position coordinates" is a technology for calculating the position coordinates of the top of the head and hairline in particular based on the position information of the face.

[1443] "Means for identifying areas of thinning hair based on the analysis results and drawing those areas as a mask" is a function for identifying areas of thinning hair based on the information obtained by analysis and displaying those areas as a mask superimposed on the image.

[1444] The "means for selecting an optimal patch from a hair patch library" refers to an algorithm or technique for selecting an optimal hair patch from a library based on hair color and texture.

[1445] The "means of applying selected patches to the thinning hair area and digitally processing" refers to the process of applying selected hair patches to the thinning hair area and processing them to create a natural-looking composite.

[1446] This invention provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. The system includes a face detection unit, a hair thinning area identification unit, a digital processing unit, and a corrected image storage unit. These units are realized through a real-time video display on a preview screen, a unit for estimating the upper face area and acquiring its coordinates, a unit for identifying the thinning hair area based on the analysis results and drawing the area as a mask, a unit for selecting an optimal patch from a hair patch library, and a unit for applying the selected patch to the thinning hair area and performing digital processing.

[1447] App launch and camera initialization

[1448] The user launches the camera app on their smartphone.

[1449] The device will initialize the camera module and display real-time video on the preview screen. The internal image recognition engine will be ready.

[1450] Face detection and head location

[1451] The device detects human faces from each frame of video using image recognition algorithms, such as the Haar detector or FaceNet.

[1452] The upper area of ​​the detected face (top of the head and hairline area) is estimated and its position coordinates are obtained.

[1453] Analysis and identification of thinning hair areas

[1454] The device analyzes an image region of the top of the head and uses a convolutional neural network (CNN) to identify areas of thinning hair based on hair density and texture information.

[1455] A region mask of the identified thinning hair area is generated and its location is recorded.

[1456] Performing digital correction

[1457] Based on the mask of the bald area, the device selects the best patch from a hair patch library, based on color and texture match criteria.

[1458] The device uses Poisson blending technology to apply the selected patch to the thinning area, allowing it to blend in naturally.

[1459] Saving and displaying the corrected image

[1460] The device saves the corrected image to local storage in a standard image format such as JPEG or PNG.

[1461] The device displays the saved image to the user, allowing them to check it on a preview screen.

[1462] Users can review the images and use the in-app sharing features to share them on social media or with friends if desired.

[1463] Specific examples

[1464] Example 1: Taking a daily selfie

[1465] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[1466] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[1467] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[1468] 4. The terminal identifies areas of thinning hair and generates hair in those areas using digital processing means.

[1469] 5. The device saves the digitally enhanced image and displays it to the user, who can then review it and save or share it as desired.

[1470] Example 2: When taking photos for social media

[1471] 1. The user launches the app to take a photo for social media.

[1472] 2. The device applies filters to the image for social media, while also detecting and correcting faces and areas of baldness using the process described above.

[1473] 3. The device saves the corrected image to local storage and displays it to the user.

[1474] 4. The user uploads this image to a social networking site.

[1475] Example prompts for generative AI models

[1476] "How can I use a camera app to naturally correct thinning hair on the crown and at the hairline?"

[1477] "Please explain the process of using image recognition technology to detect faces and correct hair in real time."

[1478] "Tell me more about how convolutional neural networks can be used to detect thinning hair and digitally generate natural-looking hair."

[1479] As described above, the system according to the present invention provides a powerful tool for users to take natural-looking photos in real time without worrying about thinning hair.

[1480] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1481] Step 1: Launch the app and initialize the camera

[1482] The user launches the camera app on their smartphone.

[1483] Input: User action (app launch)

[1484] The device initializes the camera module and displays real-time video on the preview screen.

[1485] What happens: The device's internal image recognition engine is activated, and the camera settings and capture mode are adjusted appropriately.

[1486] Output: Initialized camera module and started image recognition engine

[1487] Step 2: Detecting the face and identifying the top of the head

[1488] The device detects human faces from each frame of video using an image recognition algorithm (e.g., Haar detector or FaceNet).

[1489] Input: Real-time video frame data

[1490] What it does: Runs an algorithm to determine the location of the face.

[1491] Output: Detected face location information

[1492] Step 3: Identify thinning areas

[1493] The device analyzes the detected upper face area and uses a convolutional neural network (CNN) to identify areas of thinning hair based on hair density and texture information.

[1494] Input: Image data of the upper region of the face

[1495] Specific operation: Apply CNN algorithm to analyze and generate a mask of the thinning hair area.

[1496] Output: Region mask of identified thinning hair areas

[1497] Step 4: Perform digital correction

[1498] Based on the mask of the thinning hair area, the device selects the most suitable patch from a library of hair patches.

[1499] Input: Thinning hair area mask

[1500] What it does: Selects a suitable patch from a patch library based on color and texture match.

[1501] Output: Selected hair patch

[1502] Step 5: Patching and Integration

[1503] The device uses Poisson blending techniques to naturally integrate the selected patches into the original image.

[1504] Input: Selected hair patch, original image data

[1505] What it does: It applies patches using Poisson blending techniques to achieve a natural look.

[1506] Output: Corrected image data

[1507] Step 6: Save and view the corrected image

[1508] The device saves the corrected image in local storage.

[1509] Input: Corrected image data

[1510] Specific behavior: Saves to local storage in a standard image format such as JPEG or PNG.

[1511] Output: Image file saved in local storage

[1512] The device displays the saved image to the user, who can check it on a preview screen.

[1513] Specific operation: Loads the saved image file and displays it on the preview screen.

[1514] Users can view the images and share them on social media if desired.

[1515] Input: Saved image file

[1516] Output: Corrected image displayed in preview window

[1517] Through these processing steps, the system allows users to take natural-looking photos that are corrected in real time without worrying about thinning hair.

[1518] (Application example 1)

[1519] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1520] Currently, many users are concerned about their thinning hair, which can affect their ability to try on hats and hair accessories. To enhance the in-store try-on experience, it is important for users to be able to see how they look in real-life. However, current try-on systems lack the ability to naturally correct thinning hair, potentially damaging users' self-image and confidence. Therefore, there is a need for a system that can naturally correct thinning hair and provide a real-time overall view of how the item will look when trying on.

[1521] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1522] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas of thinning hair, a means for digitally generating hair for the identified areas of thinning hair and correcting it to a natural look, a means for saving the corrected image, a means for displaying the identified areas of thinning hair in real time while keeping the corrected natural look, and a means for displaying, saving, or sharing the corrected image in real time so that the user can check how the clothes will look when tried on. This allows the user to try on clothes with their thinning hair corrected and check how they will look natural. This improves the try-on experience in a physical store and increases the user's confidence in their self-image.

[1523] "Face detection means" refers to technology that automatically identifies a person's face from image data acquired using a camera.

[1524] "Methods for identifying thinning hair areas" refer to techniques that use image analysis to identify areas with particularly low hair density, typically using convolutional neural networks (CNNs).

[1525] "Digitally generating and correcting hair to a natural appearance" refers to a technique for digitally generating and correcting natural-looking hair in a specified thinning area. Poisson blending is commonly used.

[1526] "Means for saving corrected images" refers to a function for saving image data that has been digitally corrected in a storage device of the terminal.

[1527] "Means for displaying in real time" refers to the function of instantly displaying corrected images or videos on the user's screen.

[1528] "Means for users to check the fitting condition" refers to technology that allows users to check the condition of the fitting items they are currently wearing on digital video.

[1529] "Means for displaying, saving, or sharing video in real time" refers to the ability to provide the user with corrected video in real time and save the video for later viewing or share it with others.

[1530] The system based on this invention provides an application that allows users to try on clothes in real time in a physical store while correcting thinning areas of the hair.

[1531] System configuration

[1532] Hardware

[1533] This system utilizes the camera module of mobile devices such as smartphones and tablets. The device camera is used to capture real-time images of the user.

[1534] software

[1535] The software components used include:

[1536] 1. Image recognition software: Face detection is performed using the open source OpenCV library.

[1537] 2. Convolutional Neural Network (CNN) model: This model is used to identify areas of thinning hair on the user, using a pre-trained model such as "hair_thinning_model.h5."

[1538] 3. Poisson Blending Technique: A technique for digitally correcting thinning areas of hair, used to produce natural-looking hair.

[1539] System Operation

[1540] Face detection and analysis

[1541] The device activates the camera and captures the user's video in real time, which is then analyzed through the OpenCV library to determine the face position.

[1542] Identifying thinning areas of hair

[1543] To analyze the upper region of the detected face, we use a convolutional neural network (CNN) model, which evaluates hair density on the crown of the head and identifies areas of thinning hair.

[1544] Digital Correction

[1545] Any identified thinning areas are corrected by digitally generating natural hair using Poisson blending technology, resulting in a natural-looking image.

[1546] Real-time display and storage

[1547] The corrected video is displayed on the device screen in real time, allowing users to enjoy trying on clothes while checking the corrected video. Corrected video and images can also be saved to the device's memory and shared on social media if desired.

[1548] Examples of specific examples and prompts

[1549] Specific examples

[1550] Users can use the system to film themselves while trying on hats in a physical store, and the system will correct thinning areas of hair in real time to provide a natural-looking image. Users can then check the fit and appearance of the item they are trying on while viewing the corrected image.

[1551] Prompt Sentence Examples

[1552] Use technology that naturally corrects thinning hair to provide the perfect virtual try-on app for users trying on hats. This application detects and corrects thinning hair areas when users take a real-time video of themselves using their smartphone camera. Display this corrected real-time video so users can enjoy trying on hats.

[1553] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1554] Step 1:

[1555] The user launches the app on their smartphone

[1556] Input: Tap the application icon on your smartphone

[1557] Operation: The application starts and the initial setup screen is displayed.

[1558] Output: Real-time camera preview of the smartphone is displayed.

[1559] Step 2:

[1560] The device initializes the camera module and captures real-time video.

[1561] Input: Live video stream from the camera module

[1562] Operation: Initialize the camera module and start video input.

[1563] Output: Video frames captured in real time are stored in memory

[1564] Step 3:

[1565] The device performs face detection

[1566] Input: Real-time video frame

[1567] Operation: Performs face detection using the Haar Cascade from the OpenCV library

[1568] Output: Face position coordinates (rectangular area) are obtained

[1569] Step 4:

[1570] The device analyzes the upper area of ​​your face to identify areas with thinning hair.

[1571] Input: Face position coordinates and video frame of the upper region

[1572] How it works: Uses a convolutional neural network (CNN) to assess hair density and identify thinning areas.

[1573] Output: Mask information and position data of thinning hair area are obtained

[1574] Step 5:

[1575] The device performs digital processing on the identified thinning areas.

[1576] Input: Position data of thinning hair area and mask information

[1577] How it works: Using Poisson blending technology, patches of hair are digitally generated and applied to thinning areas.

[1578] Output: Naturally corrected video frames

[1579] Step 6:

[1580] The device displays the corrected image in real time.

[1581] Input: Digitally corrected video frame

[1582] Operation: The corrected image is displayed on the screen in real time.

[1583] Output: Users can check the corrected image on the preview screen.

[1584] Step 7:

[1585] The user saves or shares the corrected footage

[1586] Input: Corrected video frame and user action (choice of saving or sharing)

[1587] How it works: Save the corrected footage to your device's storage or share it via social media or messaging apps

[1588] Output: You will get a saved image file or a shared link or message.

[1589] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1590] System Overview

[1591] The system of the present invention uses a device equipped with a camera and provides a camera app that uses image recognition technology to naturally correct thinning hair on the user's head and hairline. It also combines an emotion engine that recognizes the user's emotions to achieve more optimal correction processing. The system includes a face detection means, a means for identifying thinning hair areas, a digital processing means, an emotion recognition means, and a means for saving the corrected image.

[1592] Program processing and explanation

[1593] 1. Launching the app and initializing the camera

[1594] Subject: User, Device

[1595] The user launches the camera app on their smartphone.

[1596] The device initializes the camera module and displays real-time video on the preview screen, while the emotion engine is also activated.

[1597] 2. Face detection and head location

[1598] Subject: Terminal

[1599] The device uses a face detection algorithm (e.g., Haar detector or deep learning-based FaceNet) to detect human faces from each frame of video.

[1600] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[1601] 3. Analysis and identification of thinning hair areas

[1602] Subject: Terminal

[1603] The device analyzes an image region of the top of the head and runs a convolutional neural network (CNN) algorithm to identify areas of thinning hair based on hair density and texture information.

[1604] A region mask of the identified thinning hair area is generated and its location is recorded.

[1605] 4. Emotion Recognition and Analysis

[1606] Subject: Terminal

[1607] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[1608] 5. Performing digital correction

[1609] Subject: Terminal

[1610] The device selects patches of hair from a data library that match the identified thinning areas.

[1611] The selected patch is mapped onto the thinning hair area and then naturally integrated using Poisson blending technology.

[1612] The emotion engine adjusts the correction based on the emotion it recognizes. For example, if the user is smiling, the image brightness and saturation will be improved.

[1613] 6. Creating and saving the corrected image

[1614] Subject: Terminal, User

[1615] The terminal generates an image after the digital correction process is completed and displays it to the user on a preview screen.

[1616] If the user selects save, the device saves the generated corrected image in local storage. The emotion data recognized by the emotion engine is also saved as history.

[1617] Specific examples

[1618] 1. The user launches the app and takes a photo of themselves with their smartphone camera.

[1619] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[1620] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[1621] 4. The device identifies the areas of thinning hair and uses an emotion engine to analyze the user's facial expressions. For example, if the user is smiling, that emotion data is recorded.

[1622] 5. The device digitally processes the thinning hair and generates hair. The brightness and saturation of the image are also adjusted based on the user's emotions.

[1623] 6. The device displays the digitally enhanced image on the preview screen, and if the user selects save, the image is saved to local storage. Emotion data is also saved as history.

[1624] 7. The user can review the saved images and share them on social media if desired.

[1625] Through these steps, the system of the present invention can naturally correct thinning hair on the user's head and hairline, and then use the emotion engine to perform optimal image adjustments, allowing users to take photos with confidence and easily share the corrected images.

[1626] The processing flow will be explained below.

[1627] Step 1: Launch the app and initialize the camera

[1628] Subject: User, Device

[1629] The user launches the camera app on their smartphone.

[1630] The device initializes the camera module and displays the real-time camera image on the preview screen, while simultaneously activating the image recognition engine and emotion engine.

[1631] Step 2: Detecting the face and identifying the top of the head

[1632] Subject: Terminal

[1633] The device runs a face detection algorithm (e.g., Haar detector or FaceNet) on each frame of camera footage to detect human faces.

[1634] Based on the position coordinates of the detected face, the top of the head area (upper part of the face) is estimated and its position coordinates are obtained.

[1635] Step 3: Analysis of thinning areas

[1636] Subject: Terminal

[1637] The device uses a convolutional neural network (CNN) to analyze the hair density and texture of the image area of ​​the top of the head and identify areas of thinning hair.

[1638] A region mask of the identified thinning hair area is generated and its location is recorded.

[1639] Step 4: Recognize emotions

[1640] Subject: Terminal

[1641] The device uses an emotion engine to analyze the user's facial expressions, extracting facial features and recognizing emotions such as smiles and sadness in real time.

[1642] Step 5: Prepare for digital correction

[1643] Subject: Terminal

[1644] The device selects a patch of hair from a data library that matches the thinning area, a selection process that includes assessing the suitability based on the size, color, and texture of the thinning area.

[1645] Step 6: Perform digital correction

[1646] Subject: Terminal

[1647] The device maps selected patches of hair to the thinning areas and uses Poisson blending techniques to naturally integrate the selected patches into the original image.

[1648] The emotion engine adjusts the brightness and saturation of the corrected image based on the emotion it recognizes: for example, if the user is smiling, the brightness and saturation will be increased to make the image appear more positive.

[1649] Step 7: Generate and display the corrected image

[1650] Subject: Terminal

[1651] The device generates an image after the digital correction process is complete and displays it on a preview screen, allowing the user to check the results immediately.

[1652] Step 8: Save the corrected image

[1653] Subject: User, Device

[1654] If the user selects the save option, the device saves the generated corrected image to local storage. The user's emotion data is also saved as history.

[1655] Step 9: Share your images

[1656] Subject: User

[1657] Users can view the saved images and share them on social media or other platforms as desired. Images can be easily shared using the in-app share buttons.

[1658] These steps correct thinning hair areas and provide optimal image adjustments based on the user's emotions, allowing users to take photos with confidence and easily share the corrected images.

[1659] Example 2

[1660] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1661] Conventional camera applications have had the problem of difficulty in achieving a natural-looking result when correcting thinning hair. Furthermore, optimal correction processing that reflects the user's emotions is not performed, and corrected images often do not meet the user's expectations. To solve these problems, a system is needed that can naturally correct thinning hair on the top of the head and at the hairline, and also adjust images based on the user's emotions.

[1662] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1663] In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face to identify areas with thinning hair, a means for generating hair in the identified areas with thinning hair by digital processing and correcting it to a natural state, a means for using an emotion engine that analyzes the user's facial expressions from real-time video and recognizes their emotions, a means for fine-tuning the correction based on the emotions recognized by the emotion engine, and a means for saving the corrected image. This makes it possible to naturally correct thinning hair on the top of the user's head and at the hairline, and enables optimal image adjustment based on the user's emotions.

[1664] "Face detection method" refers to algorithms or devices that detect human faces in video, including Haar detectors and deep learning-based detection techniques.

[1665] "Means for identifying thinning hair areas" refers to an algorithm or device that analyzes the detected upper face region and identifies thinning hair areas based on hair density and texture information, such as a convolutional neural network (CNN).

[1666] "Digitally generated hair enhancements" refers to algorithms and techniques that apply digital image processing to identified thinning hair areas to enhance the appearance of natural-looking hair, including Poisson blending techniques.

[1667] "Means for using an emotion engine" refers to software or hardware that analyzes a user's facial expressions from real-time video and recognizes their emotions.

[1668] "Means for fine-tuning corrections based on emotions" refers to algorithms and technologies that adjust the brightness, saturation, etc. of the corrected image according to the user's emotions recognized by the emotion engine.

[1669] "Means for storing corrected images" refers to software or devices for storing images that have undergone digital correction processing in local storage or cloud storage.

[1670] The system of the present invention provides a camera application that naturally corrects thinning hair on the top of the user's head and at the hairline. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, more optimal correction processing can be achieved. The system includes a face detection unit, a means for identifying thinning hair areas, a digital processing unit, an emotion recognition unit, and a means for saving the corrected image.

[1671] Hardware and software used

[1672] The system's main hardware is a device such as the user's smartphone. The device must be equipped with a camera module and a high-performance CPU or GPU. The software used includes a face detection algorithm using a Haar detector and FaceNet, a hair loss detection algorithm using a convolutional neural network (CNN), a digital correction algorithm using Poisson blending technology, and an emotion engine for emotion recognition.

[1673] Processing flow and specific examples

[1674] 1. Launching the app and initializing the camera

[1675] The user launches a dedicated camera app.

[1676] The device will initialize the camera module, display real-time video on the preview screen, and start the emotion engine.

[1677] 2. Face detection and head location

[1678] The device detects faces from each frame of video using a Haar detector or FaceNet.

[1679] The upper region of the detected face, i.e., the top of the head and the hairline, is estimated and its position coordinates are obtained.

[1680] 3. Analysis and identification of thinning hair areas

[1681] The device analyzes the image area of ​​the top of the head and runs a CNN algorithm to identify areas of thinning hair based on hair density and texture information.

[1682] A region mask of the identified thinning hair area is generated and its location is recorded.

[1683] 4. Emotion Recognition and Analysis

[1684] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[1685] 5. Performing digital correction

[1686] The device selects patches of hair from a data library that match the identified thinning areas.

[1687] The selected patch is mapped onto the thinning hair area and then naturally integrated using Poisson blending technology.

[1688] The emotion engine fine-tunes the correction based on the emotion it recognizes, for example, if the user is smiling, it will improve the brightness and saturation of the image.

[1689] 6. Creating and saving the corrected image

[1690] The terminal generates an image after the digital correction process is completed and displays it to the user on a preview screen.

[1691] If the user selects save, the device saves the generated corrected image in local storage. The emotion data recognized by the emotion engine is also saved as history.

[1692] Specific examples

[1693] 1. The user launches the camera app and takes a picture of themselves.

[1694] 2. The device analyzes the video in real time and detects the user's face using face detection means.

[1695] 3. The terminal identifies the top region of the detected face and executes a means for identifying thinning hair areas to analyze the thinning hair areas.

[1696] 4. The device identifies areas of thinning hair and uses an emotion engine to analyze the user's facial expressions, for example, brightening the image if the user is smiling.

[1697] 5. The device digitally processes the thinning areas, using selected patches of hair to create a natural-looking correction. The corrected image is then fine-tuned based on the user's emotions.

[1698] 6. The device displays the generated corrected image on the preview screen, and when the user selects save, the image is saved to local storage. At the same time, emotion data is saved.

[1699] 7. Users can view the saved images and share them on social media etc.

[1700] Prompt Sentence Examples

[1701] "Please analyze real-time images captured by a smartphone camera app, identify the state of the user's face and hair, and then digitally correct any thinning hair to make it look natural. Please provide an example of optimal correction that takes the user's emotions into consideration."

[1702] "Please explain an example of a system that automatically corrects thinning hair in images taken by a user using a camera app, and also analyzes real-time facial expressions to adjust the image based on emotion."

[1703] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1704] Program processing flow

[1705] Step 1:

[1706] App launch and camera initialization

[1707] The user simply launches a dedicated camera app.

[1708] The device will initialize the camera module and display real-time video on the preview screen, while also starting the emotion engine.

[1709] Input: Camera app launch command.

[1710] Output: Real-time camera footage and emotion engine initialization complete.

[1711] Specific operation: When a user launches the app, the device's camera module is activated and the image is previewed in real time. The emotion engine is also initialized in the background.

[1712] Step 2:

[1713] Face detection and head location

[1714] The device detects faces in each frame of video using a face detection algorithm (such as a Haar detector or FaceNet).

[1715] The upper area of ​​the detected face, i.e., the top of the head and hairline, is estimated and its position coordinates are obtained.

[1716] Input: Real-time camera footage.

[1717] Output: Detected face position coordinates and head top region.

[1718] Specific operation: A face detection algorithm is run on each video frame captured by the device's camera to determine the position of the face and the coordinates of the top of the head.

[1719] Step 3:

[1720] Analysis and identification of thinning hair areas

[1721] The device extracts the top and hairline regions from the facial detection results and runs a convolutional neural network (CNN) algorithm to analyze these regions.

[1722] The device identifies areas of thinning hair based on hair density and texture information, generates a region mask for the identified areas of thinning hair, and records their location information.

[1723] Input: parietal region coordinates.

[1724] Output: Mask of thinning hair areas and their locations.

[1725] Specific operation: The image of the top of the head is analyzed using a CNN algorithm to identify areas with low hair density, generate a mask, and record it.

[1726] Step 4:

[1727] Emotion Recognition and Analysis

[1728] The device uses an emotion engine to analyze the user's facial expressions from real-time video footage, extracting facial features and recognizing emotions such as smiles and sadness.

[1729] Input: Real-time video of the user.

[1730] Output: User emotion data.

[1731] Specific operation: Facial features are extracted from real-time video and analyzed by an emotion engine to identify the user's emotions.

[1732] Step 5:

[1733] Performing digital correction

[1734] The device selects patches of hair from a data library that match the identified thinning areas.

[1735] Selected patches of hair are mapped onto thinning areas and then naturally integrated using Poisson blending techniques.

[1736] The emotion engine fine-tunes the correction based on the emotions it recognizes, for example, increasing the brightness and saturation of the image if the user is smiling.

[1737] Input: Mask of thinning hair area, hair patch from data library, emotion data.

[1738] Output: The corrected image.

[1739] Specific operation: Select the appropriate hair patch to match the thinning area, use Poisson blending technology to perform natural correction, and further fine-tune the correction based on the user's emotions.

[1740] Step 6:

[1741] Generate and save the corrected image

[1742] The device generates an image with the digital correction process completed and displays it to the user on a preview screen.

[1743] If the user selects save, the device will save the generated corrected image to local storage, along with the emotion data recognized by the emotion engine.

[1744] Input: Corrected image, user save command.

[1745] Output: Saved corrected image and emotion data.

[1746] Specific operation: The corrected image is displayed on the preview screen, and if the user selects save, the image and emotion data are saved to local storage.

[1747] (Application example 2)

[1748] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1749] In today's world, many people suffer from thinning hair, but technology to naturally correct it is still insufficient. Furthermore, there is a lack of a way to generate optimal images based on the user's emotional state, and an effective method to improve the user experience is needed. Improved customer service using such technology is especially desirable in brick-and-mortar stores.

[1750] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a face detection means, a means for analyzing the upper region of the detected face and identifying areas with thinning hair, a means for digitally generating hair for the identified areas with thinning hair and correcting it to a natural state, a means for analyzing the user's emotions and optimizing the image, and a means for saving the corrected image. This makes it possible to naturally correct the user's thinning hair and generate an optimal image according to the user's emotional state.

[1751] "Face detection means" is a technology for recognizing a person's face in an image and identifying its position.

[1752] The "means for analyzing the detected upper region of the face and identifying areas with thinning hair" is a technology that focuses on the upper region of the face and identifies areas with thinning hair by analyzing the density and condition of the hair within that region.

[1753] The "means of generating hair by digital processing in identified thinning areas and correcting it to a natural state" refers to a technology that uses digital technology to generate hair in areas determined to be thinning hair and then blends it naturally with the surrounding hair.

[1754] "Means for analyzing user emotions and optimizing images" refers to technology that analyzes the user's facial expressions and emotional data and adjusts the brightness, saturation, etc. of the image to an optimal state based on that data.

[1755] The "means for saving the corrected image" refers to a technology for saving the image that has been digitally corrected in the memory or storage of the device.

[1756] The system of the present invention includes a face detection means, a means for analyzing the upper region of the detected face to identify areas of thinning hair, a means for digitally generating hair in the identified areas of thinning hair and correcting it to a natural state, a means for analyzing a user's emotions to optimize the image, and a means for saving the corrected image.

[1757] System Configuration

[1758] Hardware

[1759] Built-in camera: A camera for capturing the user's face in real time and acquiring video data.

[1760] Display: A monitor that displays the corrected image and real-time feedback to the user. It is expected to be installed in fitting rooms in physical stores or beauty salons.

[1761] software

[1762] OpenCV: A library for image processing, used for face detection and video analysis.

[1763] Keras: A library for running deep learning models, used in convolutional neural networks (CNNs) and emotion recognition models.

[1764] Haar Cascade: A classifier for face detection

[1765] Process Overview

[1766] 1. Camera initialization and image acquisition

[1767] The server initializes the built-in camera and captures real-time video, which is then analyzed using face detection means.

[1768] 2. Face Detection

[1769] The server uses OpenCV's Haar Cascade to detect faces in the video, and once a face is detected, its location information is obtained.

[1770] 3. Identify thinning areas

[1771] The detected upper face region is analyzed using a convolutional neural network (CNN) using Keras to identify areas with thinning hair. A mask of the thinning hair region is generated and its location is recorded.

[1772] 4. Emotion Analysis

[1773] The server uses Keras emotion models to analyze the user's facial expressions from real-time video, specifically recognizing emotions such as smile or sadness, and adjusts image correction accordingly.

[1774] 5. Digital Processing

[1775] The server selects patches of hair from a data library that match the identified thinning areas, integrates them naturally using Poisson blending techniques, and adjusts the correction based on emotions recognized by the emotion engine.

[1776] 6. Creating and saving the corrected image

[1777] The image after correction processing is displayed on the screen, and if the user selects save, the image is saved to local storage. Emotion analysis data is also saved as history.

[1778] Specific examples

[1779] When a user stands in front of a smart mirror at a beauty salon, the camera detects the user's face. The upper area of ​​the detected face is analyzed, and areas of thinning hair are identified using CNN. Next, the emotion engine analyzes the user's facial expression, adjusting brightness and saturation if the user is smiling. Finally, an image with the thinning hair corrected is displayed on the screen, and if the user is satisfied, they can save the image.

[1780] Prompt Sentence Examples

[1781] It identifies the user's facial area, detects and corrects thinning hair on the top of the head and at the hairline, and recognizes the user's emotions to adjust the brightness and saturation of the image to generate an optimally corrected image.

[1782] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1783] Step 1:

[1784] The server initializes the built-in camera and acquires real-time video. The input is the camera image, and the output is real-time video data from the initialized camera module. Specifically, it performs initial settings so that the camera device operates correctly and starts the video stream.

[1785] Step 2:

[1786] The server uses OpenCV's Haar Cascade to detect faces from real-time video frames. The input is real-time video data, and the output is position information (coordinate data) of detected faces. Specifically, it analyzes each frame and identifies the face area based on facial features.

[1787] Step 3:

[1788] The server analyzes the detected upper face region using a convolutional neural network (CNN) with Keras to identify areas with thinning hair. The input is the face's position information and video data of that region, and the output is a mask of the thinning hair area. Specifically, the server evaluates the hair density in the upper face region and recognizes the pattern of thinning hair.

[1789] Step 4:

[1790] The server uses Keras to analyze the user's facial expressions from real-time video and recognize emotions. The input is video data of the face area, and the output is recognized emotion data. Specifically, it extracts facial features and distinguishes between emotional states such as smiling and sad.

[1791] Step 5:

[1792] The server selects hair patches from a data library that match the identified thinning areas and integrates them naturally using Poisson blending technology. The input is a mask of the thinning area and the selected hair patches, and the output is the corrected image data. Specifically, it selects an appropriate hair texture and blends it naturally into the thinning areas.

[1793] Step 6:

[1794] The server adjusts the brightness and saturation of the corrected image based on the emotion recognized by the emotion engine. The input is emotion data and corrected image data, and the output is optimized corrected image data. Specifically, for example, if a smiling emotion is recognized, the brightness and saturation of the image are increased to create a more positive impression.

[1795] Step 7:

[1796] The server displays the image after correction processing is complete on the display, and if the user selects save, it saves the image to local storage. The input is the optimized corrected image data and the user's save instruction, and the output is the saved corrected image data. Specifically, it accepts operations from the user interface and executes the save process.

[1797] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1798] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1799] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1800] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1801] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1802] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1803] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1804] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1805] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1806] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1807] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1808] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1809] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1810] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1811] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1812] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1813] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1814] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1815] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1816] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1817] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1818] The following is further disclosed regarding the above embodiment.

[1819] (Claim 1)

[1820] a face detection means;

[1821] means for analyzing the detected upper face region to identify areas of thinning hair;

[1822] a means for generating hair by digital processing in the identified thin hair portion and correcting it to a natural state;

[1823] means for storing the corrected image;

[1824] A system including:

[1825] (Claim 2)

[1826] 10. The system of claim 1, wherein the upper face region is identified using a convolutional neural network.

[1827] (Claim 3)

[1828] 10. The system of claim 1, wherein the digital processing for generating the hair uses Poisson blending techniques.

[1829] "Example 1"

[1830] (Claim 1)

[1831] a face detection means;

[1832] means for analyzing the detected upper face region to identify areas of thinning hair;

[1833] A means for generating hair by digital processing in the identified thin hair area and correcting it to a natural state;

[1834] means for storing the corrected image;

[1835] A means for displaying the video on a preview screen in real time;

[1836] means for estimating the upper region of the detected face and obtaining its position coordinates;

[1837] A method for identifying thinning hair areas based on the analysis results and drawing those areas as a mask;

[1838] A means of selecting the best patch from a library of hair patches;

[1839] A means for applying the selected patch to the thinning hair area and digitally processing it;

[1840] A system including:

[1841] (Claim 2)

[1842] 10. The system of claim 1, wherein the upper face region is identified using a convolutional neural network.

[1843] (Claim 3)

[1844] 10. The system of claim 1, wherein the digital processing for generating the hair uses Poisson blending techniques.

[1845] "Application Example 1"

[1846] (Claim 1)

[1847] a face detection means;

[1848] means for analyzing the detected upper face region to identify areas of thinning hair;

[1849] a means for generating hair by digital processing in the identified thin hair portion and correcting it to a natural state;

[1850] means for storing the corrected image;

[1851] A means for displaying the identified thinning hair areas in real time while correcting them to a natural state;

[1852] A means for displaying the corrected image in real time so that the user can check the fitting condition, and for saving or sharing the image;

[1853] A system including:

[1854] (Claim 2)

[1855] 10. The system of claim 1, wherein the upper face region is identified using a convolutional neural network.

[1856] (Claim 3)

[1857] 10. The system of claim 1, wherein the digital processing for generating the hair uses Poisson blending techniques.

[1858] "Example 2: Combining Emotion Engines"

[1859] (Claim 1)

[1860] a face detection means;

[1861] means for analyzing the detected upper face region to identify areas of thinning hair;

[1862] a means for generating hair by digital processing in the identified thin hair portion and correcting it to a natural state;

[1863] A means for analyzing a user's facial expressions from real-time video and using an emotion engine to recognize emotions;

[1864] means for fine-tuning the correction based on the emotions recognized by the emotion engine;

[1865] means for storing the corrected image;

[1866] A system including:

[1867] (Claim 2)

[1868] 10. The system of claim 1, wherein the upper face region is identified using a convolutional neural network.

[1869] (Claim 3)

[1870] 10. The system of claim 1, wherein the digital processing for generating the hair uses Poisson blending techniques.

[1871] "Application example 2 when combining emotion engines"

[1872] (Claim 1)

[1873] a face detection means;

[1874] means for analyzing the detected upper face region to identify areas of thinning hair;

[1875] a means for generating hair by digital processing in the identified thin hair portion and correcting it to a natural state;

[1876] A means for analyzing user emotions to optimize images;

[1877] means for storing the corrected image;

[1878] A system including:

[1879] (Claim 2)

[1880] 10. The system of claim 1, wherein the upper face region is identified using a convolutional neural network.

[1881] (Claim 3)

[1882] 10. The system of claim 1, wherein the digital processing for generating the hair uses Poisson blending techniques. [Explanation of symbols]

[1883] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a face detection means; means for analyzing the detected upper face region to identify areas of thinning hair; a means for generating hair by digital processing in the identified thin hair portion and correcting it to a natural state; means for storing the corrected image; A system including:

2. 10. The system of claim 1, wherein the upper face region identification uses a convolutional neural network.

3. 10. The system of claim 1, wherein the digital process for generating the hair uses Poisson blending techniques.

Images