Hair Image Analysis Method

Abstract

To provide a hair image analysis technique for acquiring an index value of a hair attribute or hair state for each sub-section of the hair.SOLUTION: In a hair image analysis method, one or more processors which can use a learned model that has been machine-learned on the basis of a plurality of pieces of teacher data including a plurality of combinations of correct answer information indicating an attribute or state common to the hair photographed in a teacher hair image and a plurality of teacher hair patch image groups in a prescribed image size extracted from the teacher hair image acquire an evaluation hair image, acquire a hair patch image group in a prescribed image size from the acquired evaluation hair image, perform normalization of each pixel value for each acquired hair patch image, and acquire an index value of the hair attribute or hair state related to each hair patch image by inputting each normalized hair patch image to the learned model.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a hair image analysis technique for analyzing a human hair image using a learned model obtained by machine learning.

Background Art

[0002] Patent Document 1 below discloses a dedicated device used by contacting the scalp or hair. In this device, measurement of the moisture content and imaging of an image of the hair area are performed, and these are sent to a computer for evaluation, and finally, a method of proposing an appropriate product to a consumer is disclosed.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above method, there are still points for improvement, such as the need for a dedicated device equipped with a moisture content measuring instrument or the like. The present invention provides a hair image analysis technique for obtaining index values of hair attributes or hair states for each small section of hair.

Means for Solving the Problems

[0005] According to the present invention, correct information indicating an attribute or state common to the hair shown in the teacher hair image and a plurality of predetermined image sizes extracted from the teacher hair image Comprising a hair patch image for teachersOne or more processors that can utilize a trained model machine-learned based on a plurality of teacher data including a plurality of combinations with teacher hair patch image groups acquire an evaluation hair image in which the hair of the person to be evaluated is captured, a step of obtaining a hair patch image group of a predetermined image size from the acquired evaluation hair image, a step of normalizing pixel values for each of the acquired hair patch images, and by inputting each normalized hair patch image into the trained model, respectively obtaining index values of hair attributes or hair states for each hair patch image, a hair image analysis method can be provided that executes the steps.

[0006] Also, according to the present invention, in response to the input of a hair patch image of a predetermined image size, an index value of the hair attribute or hair state of the hair patch image is calculated For causing a processor to execute a process A model learning method, One or more processors a step of obtaining a plurality of combinations of a teacher hair image and correct information indicating an attribute or state common to the hair shown in the teacher hair image, a step of obtaining a teacher hair patch image group of a predetermined image size that includes a hair region at a predetermined ratio or more from the acquired teacher hair image, a step of normalizing pixel values for each of the acquired teacher hair patch images, a step of generating a teacher data group in which the correct information corresponding to the original teacher hair image from which each teacher hair patch image was obtained is associated with each of the normalized teacher hair patch image groups, and learning the model using the teacher data group Step and, Execute the model learning method can be provided. Also, a hair image analysis apparatus including at least the above-described one or more processors and a memory, the hair image analysis apparatus capable of executing the above-described hair image analysis method, A model learning device comprising at least the above-described one or more processors and a memory a model learning apparatus capable of executing the above-described model learning method, etc. can also be provided.

Effects of the Invention

[0007] According to each of the above aspects, it is possible to provide a hair image analysis technique for obtaining index values of hair attributes or hair states for each small section of hair.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Mode for Carrying Out the Invention

[0009] Hereinafter, examples of preferred embodiments of the present invention (hereinafter referred to as this embodiment) will be described. The embodiments described below are illustrative, and the present invention is not limited to the configurations of the following embodiments.

[0010] The hair image analysis method (hereinafter referred to as this analysis method) and the model learning method (hereinafter referred to as this learning method) according to this embodiment are executed by one or more processors included in one or more information processing apparatuses. FIG. 1 conceptually shows a hardware configuration example of an information processing apparatus 10 capable of executing the present analysis method and the present learning method. The information processing apparatus 10 is a so-called computer and includes a CPU 11, a memory 12, an input / output interface (I / F) 13, a communication unit 14, and the like. The information processing apparatus 10 may be a desktop PC (Personal Computer), a portable PC, a mobile terminal such as a smartphone or a tablet, or a dedicated computer.

[0011] The CPU 11 is a so-called processor and may include, in addition to a general CPU (Central Processing Unit), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a graphics processing unit (GPU), and the like. The memory 12 is a RAM (Random Access Memory), a ROM (Read Only Memory), and an auxiliary storage device (such as a hard disk). The input / output I / F 13 can be connected to user interface devices such as a display device 15 and an input device 16. The display device 15 is a device that displays a screen corresponding to drawing data processed by the CPU 11 or the like, such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube) display. The input device 16 is a device that receives user operations such as a keyboard and a mouse. The display device 15 and the input device 16 may be integrated and realized as a touch panel. The communication unit 14 communicates with other computers via a communication network and exchanges signals with other devices such as a printer. A portable recording medium or the like may also be connected to the communication unit 14.

[0012] The hardware configuration of the information processing apparatus 10 is not limited to the example in FIG. 1. The information processing apparatus 10 may include other hardware elements not shown in the drawings. Also, the number of each hardware element is not limited to the example in FIG. 1. For example, the information processing apparatus 10 may have a plurality of CPUs 11. Further, the information processing apparatus 10 may be realized by a plurality of computers each consisting of a plurality of enclosures.

[0013] The information processing apparatus 10 can execute the present analysis method and the present learning method by executing a computer program stored in the memory 12 by the CPU 11. This computer program is installed via the input / output I / F 13 or the communication unit 14 from a portable recording medium such as a CD (Compact Disc), a memory card, or the like, or another computer on a network, and stored in the memory 12. In this specification, for the sake of convenience of explanation, an example in which the present analysis method and the present learning method are executed by the common information processing apparatus 10 is given. However, the present analysis method and the present learning method may be executed by separate information processing apparatuses each having the same or different hardware configurations. Also, the present analysis method and the present learning method may be implemented by separate computer programs.

[0014] The information processing apparatus 10 (CPU 11) can utilize a learned model that has been machine-learned based on teacher data, and can further execute the present learning method to learn the model. The "learned model" here is a model obtained by machine learning using teacher data, that is, supervised learning, and can be expressed as an AI (Artificial Intelligence) model, a machine learning (ML) model, or the like. Also, the "learned model" may be simply abbreviated as "model". The "model" in this specification includes classification models that predict the class to which an object belongs, such as binary (two-class) classification and multi-class classification, and regression models that predict numerical values. The "model" also includes models obtained by analysis methods also called statistical analyses such as simple regression analysis, multiple regression analysis, and principal component analysis. In this specification, machine learning and statistical analysis are not distinguished from each other. The model used in this embodiment only needs to be able to output an index value of the hair attribute or hair state of the hair patch image in response to the input of the hair patch image, and its data structure, learning algorithm, etc. are not limited. For example, when a binary classification model is used, index values indicating the possibility that the hair patch image belongs to two classes indicating different hair attributes or hair states may be output respectively, or an index value (identification value) that can identify one class with a high possibility that the hair patch image belongs to among the two classes may be output. Further, when a regression model is used, evaluation scores of one or more hair attributes or hair states regarding the hair patch image may be output as the index value. Note that the hair patch image input to the model and the hair attributes and hair states that can be obtained from the output of the model will be described later.

[0015] The model may be, for example, a regression equation obtained by regression analysis, or may be composed of a convolutional neural network including a deep neural network, and may be realized by a combination of a computer program and parameters, a combination of a plurality of functions and parameters, etc. Further, when the model is constructed by a neural network and the input layer, intermediate layer, and output layer are regarded as units of one neural network, it may refer to one neural network or a combination of a plurality of neural networks. Further, the model may be composed of a combination of a plurality of multiple regression equations or may be composed of one multiple regression equation. Further, the model may be composed of a combination of a neural network and other models such as a regression equation or a discriminant equation, and the value obtained as the output of the neural network may be input to the other model and the obtained value may be output. In the present embodiment, an example in which the model is composed of a deep neural network is given, and in the following description, it may be referred to as an AI model.

[0016] This AI model may be stored in the memory 12 in the information processing device 10, or may be stored in the memory of another computer that can be accessed by the information processing device 10 through communication. In this way, the information processing device 10 can be described as a hair image analysis device that can use the AI model and can execute this analysis method, and can also be described as a model learning device that can learn the AI model. In the following description, the CPU 11 will be described as the execution entity of this analysis method and this learning method.

[0017] [Hair Image Analysis Method (This Analysis Method)] Here, this analysis method will be described with reference to FIGS. 2 and 3. FIG. 2 is a conceptual diagram of the processing flow of the hair image analysis method according to the present embodiment, and FIG. 3 is a flowchart of the hair image analysis method according to the present embodiment. This analysis method is executed by the information processing device 10 (CPU 11) as illustrated in FIG. 1. As described above, the CPU 11 can utilize a learned AI model that outputs an index value of the hair attribute or hair state of a hair patch image having a predetermined image size in response to the input of the hair patch image. This AI model is machine-learned based on a plurality of teacher data including a plurality of combinations of correct information indicating an attribute or state common to the hair shown in the teacher hair image and a group of teacher hair patch images of a predetermined image size extracted from the teacher hair image. Note that this AI model may be learned by this learning method, which will be described later.

[0018] The teacher hair image is an image that is used as teacher data and in which a person's hair is shown. For example, an image obtained by imaging a person's head from behind is used as the teacher hair image. The teacher hair image only needs to show the hair that is the object of discrimination of the hair attribute or hair state, and may also show a human body part other than the hair that is the object of discrimination, such as the shoulders or back, or an accessory such as an earring. The hair attribute is, for example, an attribute of the person having the hair, such as age, generation, gender, etc. The hair state is, for example, a state indicating dyed hair or natural hair, a state of hair gloss (moisture), a state of hair damage, a state of sideburns, a state of progression to white hair, etc. The group of teacher hair patch images is an image set including a plurality of hair patch images extracted from one teacher hair image. The hair patch image is a small block image that contains a hair region (an image region showing hair) at a predetermined ratio or more. The predetermined ratio is at least 90% or more. The hair patch image has a predetermined shape and is preferably rectangular for ease of processing, but the shape is not limited, such as circular, elliptical, etc. The image size of the hair patch image is predetermined. For example, it is determined to be a size of 224 pixels × 224 pixels. However, the size of the hair patch image is not limited to such an example and may be appropriately determined according to the specifications of the AI model.

[0019] In this analysis method, first, the CPU 11 acquires an evaluation hair image (S31). The hair image for evaluation may be an image in which the hair of the person to be evaluated is captured, and other parts may also be captured. The image size of the hair image for evaluation is not restricted at all as long as a plurality of hair patch images can be extracted from the hair image for evaluation. The hair image for evaluation obtained in step (S31) is obtained as image data in, for example, JPEG (Joint Photographic Experts Group) format, BMP (Bitmap image) format, TIFF (Tagged Image File Format) format, GIF (Graphic Interchange Format) format, etc. However, the data format of the hair image for evaluation is not restricted. Also, the obtained hair image for evaluation is preferably a color image captured by a camera that captures visible light in order to evaluate the appearance state or attributes of the hair. The CPU 11 may obtain the image from the camera that captured the hair image for evaluation, or may obtain the image from another computer or a portable recording medium.

[0020] Next, the CPU 11 obtains a group of hair patch images from the hair image for evaluation obtained in step (S31) (S32). The number of hair patch images included in the group of hair patch images is preferably large in order to correspond to the number of index values to be obtained, which will be described later, but is not particularly restricted. Also, the predetermined image size of each hair patch image obtained in step (S32) is preferably the same as the image size of each hair patch image forming the above-described group of teacher hair patch images, but does not necessarily have to match the image size.

[0021] The hair area of the subject shown in each individual hair patch image included in the group of hair patch images obtained in step (S32) and each individual teacher hair patch image included in the group of teacher hair patch images of the teacher data is, for example, 1 cm 2 or more and 100 cm 2 or less of the subject's hair area. Therefore, it is preferable that the evaluation hair images and teacher hair images, which are the sources of the individual hair patch images and individual teacher hair patch images, are captured under imaging conditions such as resolution such that the hair patch images and teacher hair patch images having a predetermined image size indicate a hair region within the above-described range. Further, by photographing the subject's hair with a color patch of a predetermined size (for example, 1 cm square) attached thereto, the relationship between the pixel size on the evaluation hair image and teacher hair image and the actual size of the subject's hair region can be grasped.

[0022] In step (S32), the hair patch image group can be obtained by various methods. For example, there is a method of dividing the evaluation hair image into rectangular sub-region images of a predetermined image size and cutting out all the rectangular sub-region images that contain a hair region (image region indicating hair) of a predetermined ratio or more as hair patch images. In this method, the sub-region images may be cut out with an image size larger than the final image size, and then the hair patch images of the final image size may be further cut out from the sub-region images. Further, there is also a method of obtaining a hair patch image group by moving a cutting window of a predetermined image size on the evaluation hair image and cutting out the sub-region image within the window at the timing when the hair region is included in the window at a predetermined ratio or more. In this method, the same hair region may be partially included in an overlapping manner among the hair patch images. In this case, if the determination of the hair region is performed when the cutting window is moved by a predetermined amount or more from the previously cut-out position, it is possible to avoid the same hair region occupying most of the area among the hair patch images. Further, the hair patch image group may be obtained by other methods, and the method for obtaining the hair patch image group is not limited.

[0023] Next, the CPU 11 normalizes the pixel values for each hair patch image acquired in step (S32) (S35). For example, the CPU 11 calculates the average value and variance value of the pixel values (RGB values) in one hair patch image, and linearly operates on each pixel value of the hair patch image using the average value and variance value, thereby normalizing the pixel values of the hair patch image. At this time, the average value and variance value of the pixel values calculated for all the hair patch images acquired in step (S32) may be used. Also, as for the normalization method, not only a general normalization method that sets the average value to zero and the variance value to 1, but also a normalization method based on normalization parameters determined for each AI model may be used. Various known methods can be used as the normalization method in step (S32).

[0024] Subsequently, the CPU 11 inputs each normalized hair patch image into the AI model (S37), and respectively obtains an index value of the hair attribute or hair state regarding each hair patch image (S38). The hair attribute or hair state indicated by the obtained index value is determined by the AI model to be used. For example, when an AI model (binary classification model) that is learned with correct information indicating either dyed hair or natural hair as the common hair state of the hair shown in the teacher hair image is used, in (S38), for each hair patch image, either one or both of the index values of the dyed hair likeness or the natural hair likeness, or the identification value indicating either dyed hair or natural hair are respectively obtained as the index value of the hair state. Also, when an AI model (regression model) that is learned with correct information indicating the actual age of the subject as the common attribute of the hair shown in the teacher hair image is used, for each hair patch image, the index value indicating the apparent age of the hair is respectively obtained as the index value of the hair attribute. When an AI model that is learned with correct information indicating the actual age range (decade) of the subject as the common attribute of the hair shown in the teacher hair image is used, for each hair patch image, the index value indicating the likelihood for each apparent age range of the hair or the identification value capable of identifying any one apparent age range are respectively obtained as the index value of the hair attribute. In addition, when an AI model that has been learned with an evaluation value indicating the degree of gloss, degree of damage, degree of split ends, or degree of gray hair of the hair as the correct answer information for the common state of the hair shown in the teacher's hair image is used, for each hair patch image, an index value indicating the degree of gloss, degree of damage, degree of split ends, or degree of gray hair of the hair is respectively obtained as an index value of the hair state. However, the index value output by the AI model is not limited to such examples, and any value that can be an index of hair attributes or hair state may be used.

[0025] In FIG. 3, a plurality of steps (processes) are shown in order, but the execution order of each step of this analysis method is not limited to the example of FIG. 3. For example, instead of executing steps (S37) and (S38) after normalization is completed for all hair patch images in the hair patch image group, for each individual hair patch image, normalization (S35), input to the AI model (S37), and acquisition of the index value (S38) may be executed in one flow. Also, instead of executing steps (S35) and subsequent steps after all hair patch images in the hair patch image group are acquired in step (S32), steps (S35) and subsequent steps may be executed each time one hair patch image is acquired in step (S32).

[0026] In this analysis method, as described above, for a learned model that has been machine-learned using a teacher data group consisting of correct answer information indicating common attributes or states of the hair shown in the teacher's hair image and a group of teacher hair patch images extracted from the teacher's hair image, by inputting a group of hair patch images of a predetermined image size obtained from the evaluation hair image, index values of hair attributes or hair states are respectively obtained for each hair patch image. By using an AI model learned using correct answer information indicating common attributes or states of the entire hair in this way and obtaining index values of hair attributes or hair states for each hair patch image, it is possible to suppress the influence of variations in the shooting environment of the evaluation hair image such as lighting and differences in the appearance of local parts of the hair (reflections, etc.), and analyze the visual hair state or hair attributes obtained from the entire subject hair shown in the evaluation hair image.

[0027] The CPU 11 can process the index values of each hair patch image obtained in step (S38) as follows. For example, the CPU 11 can generate index value distribution information indicating the distribution of the index values of each hair patch image according to the position of the hair region shown in each hair patch image. This index value distribution information may be, for example, a graph or a map in which the index values are arranged according to the position of the hair region shown in each hair patch image. According to this index value distribution information, the distribution of the index values of the hair state or hair attributes in the entire hair of the subject can be grasped.

[0028] FIG. 4 is a diagram showing an example of a color map image as the index value distribution information of a group of hair patch images. As shown in FIG. 4, the CPU 11 can generate, as the index value distribution information, a color map image in which the index values of each hair patch image are mapped to the evaluation hair image obtained in step (S31) according to the position of the hair region shown in each hair patch image. Although FIG. 4 has to be a binary image due to drawing constraints and is difficult to understand, it shows that the closer to black, the higher the index value of the scalp-likeness, and the closer to white, the lower the index value of the scalp-likeness. Although it is shown in black and white in FIG. 4, in the color map image, the position of each hair patch image on the evaluation hair image is set to the color corresponding to its index value. According to this color map image, the distribution of the index values can be grasped visually, and since the distribution is shown on the evaluation hair image, the indices of the hair state or hair attributes for each part of the hair can be grasped intuitively.

[0029] In addition, the CPU 11 can also calculate a representative evaluation value of the hair attributes or hair state of the person to be evaluated based on the index values of the acquired hair patch images. One representative evaluation value may be calculated for the evaluation hair image obtained in step (S31), or the hair region shown in the evaluation hair image may be divided into several parts (for example, the top of the head, the back of the head, the hair tip, etc.) larger than each hair patch image, and the representative evaluation value of each part may be calculated using the index values of the hair patch images belonging to each part. The representative evaluation value is, for example, a statistical value such as the average, standard deviation, or variance of the index values of each hair patch image. By obtaining the representative evaluation value from the index values of each hair patch image in this way, it is possible to easily grasp the overall evaluation of the hair condition or hair attributes of the person being evaluated shown in the evaluation hair image, either for the entire hair or for each part.

[0030] Furthermore, it is also possible to make it possible to specify the hair area for calculating the above-mentioned representative evaluation value. In this case, the CPU 11 further executes a process of obtaining area designation information for designating a desired image area in the evaluation hair image obtained in step (S31), and based on the index values obtained for the hair patch image group corresponding to the image area indicated by the area designation information obtained in that process, it may calculate the representative evaluation value of the hair attribute or hair condition of that image area. The CPU 11 can obtain the area designation information by displaying the evaluation hair image on the display device 15 and specifying the image area designated using the input device 16 with respect to the displayed evaluation hair image. The CPU 11 can also obtain such area designation information from another computer through communication. In this way, it is possible to obtain the representative evaluation value of the hair condition or hair attribute for the hair area that is of interest to the person being evaluated or other users.

[0031] [Learning method of the discrimination model (this learning method)] Next, the learning method of the AI model used in the above-described analysis method will be described with reference to FIG. 5. FIG. 5 is a flowchart of the learning method of the AI model according to the present embodiment. This learning method is executed by the information processing device 10 (CPU 11) as illustrated in FIG. 1. In the present embodiment, as described above, an AI model configured by a deep neural network is used, which calculates the index value of the hair attribute or hair condition of a hair patch image in response to the input of a hair patch image of a predetermined image size. When this learning method is executed, this AI model may be stored in the memory 12 in the information processing device 10, or may be stored in the memory of another computer that the information processing device 10 can access through communication.

[0032] First, the CPU 11 acquires a plurality of combinations of a teacher hair image and correct information indicating common attributes or states of the hair depicted in the teacher hair image (S51). This teacher hair image is, for example, a teacher hair patch image having a predetermined image size, and as described above, 1 cm of the subject 2 or more and 100 cm 2 or less of the hair area is imaged under imaging conditions such as resolution. Also, by imaging the hair of the subject of the teacher hair image with a color patch of a predetermined size (for example, 1 cm square) attached, the relationship between the pixel size on the teacher hair image and the actual size of the hair area of the subject can be grasped. The "teacher hair image" here, the "attributes" and "states" common to the hair depicted in the teacher hair image are as described above. For example, the correct information indicating the common state of the hair depicted in the teacher hair image indicates either dyed hair or natural hair. As another example of the correct information on the hair state, an evaluation value indicating the degree of gloss, the degree of damage, the degree of sideburns, or the degree of gray hair of the hair is used as the correct information. Also, as an example of the correct information on the hair attribute, the actual age or actual age range of the subject is indicated. Note that "common to the hair" means that for one teacher hair image, one correct information indicating the attributes or states of the hair depicted in the teacher hair image is attached, and the one correct information is commonly associated with a group of teacher hair patch images obtained from the teacher hair image as described later.

[0033] Subsequently, the CPU 11 executes steps (S53) to (S57) for each combination of the teacher hair image and the correct information acquired in step (S51).

[0034] The CPU 11 acquires a group of teacher hair patch images of a predetermined image size that includes a hair area of a predetermined ratio or more from the teacher hair image related to the target combination (S53). The number of teacher hair patch images acquired from one teacher hair image is preferably as large as possible from the viewpoint of increasing the amount of teacher data, but is not particularly limited. In addition, the predetermined image size of each hair patch image forming the hair patch image group is determined according to the specifications of the AI model to be learned. Here, the hair region of the subject depicted in each individual teacher hair patch image included in the teacher hair patch image group acquired in step (S53) is, for example, 1 cm of the subject 2 or more and 100 cm 2 or less of the hair region.

[0035] The method of obtaining the teacher hair patch image group from the teacher hair image in step (S53) may be the same as or different from that in step (S32) of this analysis method. Also in the method of obtaining the teacher hair patch image group, various methods can be adopted in the same manner as the method of obtaining the hair patch image group described in step (S32). Since the examples of each acquisition method are as described in step (S32), the description is omitted here. The method of obtaining the teacher hair patch image group is not limited.

[0036] Subsequently, the CPU 11 normalizes the pixel values for each teacher hair patch image acquired in step (S53) (S55). For example, the CPU 11 calculates the average value and variance value of the pixel values (RGB values) in one teacher hair patch image, and linearly operates each pixel value of the teacher hair patch image using the average value and variance value, thereby normalizing the pixel values of the teacher hair patch image. The normalization method in step (S55) is preferably the same as the normalization method for the hair patch image performed in step (S35) of this analysis method. Various known methods can also be used in the normalization method in step (S55).

[0037] When the normalization (S55) is completed for all the teacher hair images included in the teacher hair patch image group acquired in step (S53), the CPU 11 generates teacher data in which the correct answer information corresponding to the teacher hair image related to the target combination is commonly associated with the teacher hair patch image normalized in step (S55) (S57).

[0038] In this way, for all combinations of the teacher hair images obtained in step (S51) and the correct answer information, steps (S53) to (S57) are executed, so that the correct answer information corresponding to the original teacher hair image from which each teacher hair patch image was obtained is associated with each of the normalized teacher hair patch image groups, generating a teacher data group. The CPU 11 causes the AI model to be learned using the generated teacher data group (S59). In the present embodiment, the learning of the AI model is performed by deep learning. However, the specific learning algorithm for the AI model is not limited in any way.

[0039] In FIG. 5, a plurality of steps (processes) are shown in order, but the execution order of each step of this learning method is not limited to only the example of FIG. 5. For example, instead of executing step (S57) after normalization is completed for all teacher hair patch images in the teacher hair patch image group, normalization (S55) and association with correct answer information (S57) may be executed in one flow for each individual teacher hair patch image. Also, instead of executing steps (S55) and subsequent steps after all teacher hair patch images in the teacher hair patch image group are obtained in step (S53), steps (S55) and subsequent steps may be executed each time one teacher hair patch image is obtained in step (S53).

[0040] In this learning method, as described above, the AI model is learned using teacher data in which correct answer information indicating the attributes or states of the hair shown in the teacher hair image is commonly associated with the teacher hair patch image group obtained from one teacher hair image. As a result, although a large amount of teacher data is required to improve the estimation accuracy of the AI model, since a large number of teacher hair patch image groups can be obtained from a small number of teacher hair images, an AI model with high efficiency and high accuracy can be constructed. In addition, by setting the correct information associated with each hair patch image to the correct information indicating the common attributes or states of the entire hair shown in the original teacher hair image, the influence of variations in the shooting environment of the teacher hair image such as lighting and differences in the appearance of local parts of the hair (reflections, etc.) can be suppressed, and a highly accurate AI model can be constructed.

[0041] Some or all of the above-described embodiments and modifications may be specified as follows. However, the above-described embodiments and modifications are not limited to the following description.

[0042] <1> One or more processors capable of using a trained model that has been machine-learned based on a plurality of teacher data including a plurality of combinations of correct information indicating the common attributes or states of the hair shown in the teacher hair image and a group of teacher hair patch images of a predetermined image size extracted from the teacher hair image, obtaining an evaluation hair image showing the hair of the person to be evaluated; obtaining a group of hair patch images of a predetermined image size from the obtained evaluation hair image; performing pixel value normalization on each of the obtained hair patch images; obtaining an index value of the hair attribute or hair state for each hair patch image by inputting each normalized hair patch image into the trained model; A hair image analysis method for executing.

[0043] <2> The one or more processors generating index value distribution information indicating the distribution of the obtained index values according to the positions of the hair regions shown in each hair patch image; The hair image analysis method according to <1>, further executing. <3> Generating a color map image in which the obtained index values are mapped to the obtained evaluation hair image according to the positions of the hair regions of each hair patch image as the index value distribution information, The hair image analysis method according to <2>. <4> The one or more processors A step of calculating a representative evaluation value of the hair attributes or hair condition of the subject based on the index values for each of the acquired hair patch images. The hair image analysis method according to any one of <1> to <3>, further executing this. <5> The one or more processors A step of obtaining region designation information for designating a desired image region in the acquired evaluation hair image; A step of calculating a representative evaluation value of the hair attributes or hair condition of the image region based on the index values obtained for the hair patch image group corresponding to the image region indicated by the acquired region designation information; The hair image analysis method according to any one of <1> to <4>, further executing this. <6> The correct information indicating the common state of the hair shown in the teacher hair image included in the teacher data indicates either dyed hair or natural hair, For each of the hair patch images, an index value of dyed hair-likeness or natural hair-likeness is obtained as an index value of the hair condition, respectively. The hair image analysis method according to any one of <1> to <5>. <7> The correct information indicating the common attribute of the hair shown in the teacher hair image included in the teacher data indicates the actual age or actual age range of the subject of the teacher hair image, For each of the hair patch images, an index value for each apparent age range of the hair or an index value indicating the apparent age of the hair is obtained as an index value of the hair attribute, respectively. The hair image analysis method according to any one of <1> to <5>. <8> The correct information indicating the common state of the hair shown in the teacher hair image included in the teacher data indicates an evaluation value indicating the degree of gloss, degree of damage, degree of split ends, or degree of gray hair of the hair, For each of the hair patch images, an index value indicating the degree of gloss, degree of damage, degree of split ends, or degree of gray hair of the hair is obtained as an index value of the hair condition, respectively. The hair image analysis method according to any one of <1> to <5>. <9> The learned model is composed of a convolutional neural network including a deep neural network. The hair image analysis method according to any one of <1> to <8>.

[0044] <10> A method for training a model that calculates an index value of a hair attribute or a hair state of a hair patch image in response to an input of a hair patch image of a predetermined image size, a step of obtaining a plurality of combinations of a teacher hair image and correct answer information indicating an attribute or a state common to the hair shown in the teacher hair image; a step of obtaining a group of teacher hair patch images of a predetermined image size that includes a hair region at a predetermined ratio or more from the obtained teacher hair images; a step of normalizing pixel values for each of the obtained teacher hair patch images; a step of generating a group of teacher data in which the correct answer information corresponding to the original teacher hair image from which each teacher hair patch image was obtained is associated with each of the normalized teacher hair patch image groups; a step of training the model using the group of teacher data; The method for training the model including the above. <11> A hair image analysis apparatus including at least the one or more processors and a memory, A hair image analysis apparatus capable of executing the hair image analysis method according to any one of <1> to <9>. <12> A model training apparatus capable of executing the method for training the model according to <10>.

[0045] A plurality of examples are given below to explain the above content in more detail. However, the description of each of the following examples does not limit the above content in any way. In each of the following examples, an AI model constructed by transfer learning (fine-tuning including weight fine-tuning) based on a convolutional neural network model consisting of 16 layers called VGG16 was used. Since VGG16 is a model that inputs a rectangular image of 224 × 224 pixels and classifies it into about 1000 categories, an AI model was constructed by transfer learning to fit each example. However, as described above, the method for constructing the AI model is not limited to such an example.

Example

[0046] In Example 1, as described above, a binary classification model for classifying dyed hair or natural hair constructed by fine-tuning VGG16 was used as an AI model, and the accuracy of the estimated value of the hair condition (dyed hair likelihood or natural hair likelihood) obtained by this analysis method, that is, the classification accuracy of the hair condition of the AI model using the hair patch image, was verified.

[0047] In Example 1, an AI model (hereinafter referred to as this model) machine-learned by this learning method using a hair patch image and an AI model (hereinafter referred to as a comparison model) machine-learned using an entire-hair image were used to compare the classification accuracy when using a hair patch image and the classification accuracy when using an entire-hair image. This model was learned using a group of teacher hair patch images of 224×224 pixels obtained from a teacher hair image of 2464×1632 pixels in which the entire back of the head was captured, and the comparison model was learned using the deformed entire-hair image obtained by deforming the teacher hair image into a rectangular image of 224×224 pixels. In Example 1, the hair area of the subject shown in each hair patch image was about 15 cm 2 of hair area.

[0048] In learning and verifying this model and the comparison model, first, 230 posterior scalp hair images were obtained by imaging the heads of 230 women from 10 to 70 years old from the back. The group of posterior scalp hair images was divided into a group with hair that had been colored within one year as the subject (hereinafter referred to as the dyed hair group) and a group with hair that had not been colored for more than one year (including the case of never having been colored) as the subject (hereinafter referred to as the natural hair group). Correct information indicating dyed hair was associated with the former dyed hair group, and correct information indicating natural hair was associated with the latter natural hair group. The group of 230 posterior scalp hair images thus obtained was divided into three groups of 6:2:2. A group of 138 posterior scalp hair images was used as teacher hair images, a group of 46 posterior scalp hair images was used for verification, and the remaining group of 46 posterior scalp hair images was used as evaluation hair images.

[0049] Figure 6 is a graph showing the learning results of the present model and the comparative model. For the present model and the comparative model, the error of the index value output for the teacher hair patch image group or the deformed whole hair image group obtained from the teacher hair image group of 138 persons is calculated using an error evaluation function called entropy loss, and learning is performed so that the error is reduced. Further, for the present model and the comparative model, learning is performed for 50 epochs using teacher data based on the occipital hair images for verification of 46 persons, and the model with the maximum classification accuracy (number of epochs) for the verification data is adopted. As shown in Figure 6, since the present model using the hair patch image showed the maximum accuracy (79.7%) at 15 epochs, the model learned at 15 epochs was adopted as the present model. Similarly, since the comparative model using the whole hair image showed the maximum accuracy of 80.4% at 5 epochs, the model learned at 5 epochs was adopted as the comparative model.

[0050] The classification accuracies of the present model and the comparative model thus adopted were evaluated using the remaining 46 evaluation hair images. As a result, in the present model, the classification accuracy for each hair patch image was 81.1%, and the classification accuracy of the representative index value (average) of the whole hair obtained from the index values for each hair patch image was 89.1%. On the other hand, in the comparative model, the classification accuracy of the whole hair was 84.8%. Thereby, it was demonstrated that the classification accuracy of the hair state of the AI model using the hair patch image is higher than that in the case of using the whole hair image by comprehensively evaluating the index values for each hair patch image for the whole hair. That is, according to Example 1, the high accuracy of the estimated value of the hair state (dyed hair appearance or natural hair appearance) obtained by the present analysis method was demonstrated.

[0051] Furthermore, a color map image was generated by mapping the index value of the dyed hair appearance or the natural hair appearance obtained by applying the evaluation hair patch image to the present model according to the position of the hair region of the evaluation hair patch image in the evaluation hair image. FIG. 7 is a diagram showing a nape hair image belonging to the natural hair group and a color map image obtained by mapping the index values of the hair patch images thereto, and FIG. 8 is a diagram showing a nape hair image belonging to the dyed hair group and a color map image obtained by mapping the index values of the hair patch images thereto. In FIGS. 7 and 8, the upper figure is the nape hair image, and the lower figure is the color map image.

[0052] Although FIGS. 7 and 8 have to be binary images due to drawing constraints and are difficult to understand, it shows that the closer to black, the higher the index value of the natural hair-likeness (the lower the index value of the dyed hair-likeness), and the closer to white, the lower the index value of the natural hair-likeness (the higher the index value of the dyed hair-likeness). Although FIGS. 7 and 8 are shown in black and white, in the color map image, the position of each hair patch image on the nape hair image may be set to the color corresponding to its index value. Although FIGS. 7 and 8 are difficult to understand because they are in black and white, according to Example 1, it can be seen that even if the hair shown in the nape hair image is black with a tinge of brown, it is possible to appropriately distinguish between dyed hair and natural hair. It can also be seen that the distribution state of natural hair and dyed hair according to the passage of time after coloring can be visualized. Thus, it has been demonstrated that according to the color map image, it is possible to intuitively grasp the index of the hair state for each hair part.

Example

[0053] In Example 2, a regression model for estimating six evaluation scores constructed by fine-tuning VGG16 was used as an AI model, and the accuracy of the estimated values of the six hair attributes and hair states obtained by this analysis method, that is, the estimation accuracy of the hair attributes and hair states of the AI model using the hair patch images was verified. The six evaluation items are the age of the subject, the condition of the hair gloss, the condition of the hair damage, the condition of the sideburns (including split ends) of the hair, the condition of the hair curl, and the condition of the gray hair of the hair. The evaluation scores for each item other than age were obtained by averaging the scores given in nine levels at 0.5 intervals from 1 to 5 by two judges for 230 nape hair images used in Example 1, and were used as the correct information for each nape hair image.

[0054] As an AI model for estimating the six-item evaluation scores as described above, in Example 2 as well as in Example 1, an AI model (this model) machine-learned by this learning method using hair patch images and an AI model (comparison model) machine-learned using whole-hair images were used, and the estimation accuracy when using hair patch images was compared with the estimation accuracy when using whole-hair images. The size of the images input to this model and the comparison model and the method for generating the rectangular images input to the comparison model are the same as those in Example 1. Based on the group of 230 occipital hair images and their correct answer information group obtained in this way, in the same manner as in Example 1, a group of 138 occipital hair images and its correct answer information group were used as teacher data, a group of 46 occipital hair images and its correct answer information group were used as verification data, and the remaining group of 46 occipital hair images and its correct answer information group were used as evaluation data.

[0055] This model and the comparison model were calculated using an error evaluation function that takes the average between evaluation items of the root mean square error (RMSE) of the error of the index value (estimated evaluation score) of each item output for the group of teacher hair patch images or the deformed whole-hair images obtained from the group of 138 teacher hair images (occipital hair image group), and were learned so that the error would be small. Furthermore, for this model and the comparison model, learning was performed for 200 epochs using teacher data based on 46 verification occipital hair images, and the model with the number of learning times (number of epochs) with the highest estimation accuracy for the verification data was adopted. The model using hair patch images showed the maximum accuracy (correlation coefficient: 0.388) at 25 epochs, so the model learned at 25 epochs was adopted as this model. Similarly, the comparison model using whole-hair images showed the maximum accuracy (correlation coefficient: 0.536) at 40 epochs, so the model learned at 40 epochs was adopted as the comparison model. The estimation accuracy of this model and the comparison model adopted in this way was evaluated using the remaining 46 evaluation hair images.

[0056] Figure 9 is a graph showing the estimation accuracy for each evaluation item in this model and the comparison model. Note that Figure 9 shows the estimation accuracy for five items excluding hair conditions such as age, gloss, damage, split ends (including frayed ends), and gray hair. According to Figure 9, it can be seen that the estimation accuracy of this model exceeds a correlation coefficient of 0.5 in all five items, indicating high accuracy. Furthermore, it can be seen that the estimation accuracy of this model is significantly higher than that of the comparison model using the entire hair image with respect to the evaluation item of hair damage. This demonstrates that the estimation accuracy of the hair condition and hair attributes of the AI model using hair patch images is higher than that when using the entire hair image. That is, according to Example 2, the high accuracy of the estimated values of the hair condition (gloss, damage, split ends, hair style, gray hair) and hair attributes (age) obtained by this analysis method was demonstrated.

[0057] Furthermore, a color map image was generated by mapping the damage index value (estimated evaluation score) obtained by applying the evaluation hair patch image to this model to the position of the hair region of the evaluation hair patch image in the evaluation hair image. Figure 10 is a diagram showing a color map image in which the damage index value (estimated evaluation score) of the hair patch image is mapped onto the evaluation hair image, and a quarter of the color map image is shown. Although Figure 10 has to be a binary image due to drawing constraints and is difficult to understand, it shows that the closer it is to black, the smaller the degree of damage (the larger the pain index value (estimated evaluation score)), and the closer it is to white, the larger the degree of damage (the smaller the pain index value (estimated evaluation score)). Although it is shown in black and white in Figure 10, in the color map image, the position of each hair patch image on the nape hair image may be the color corresponding to its index value.

[0058] Although it is difficult to understand due to black and white in FIG. 10, according to Example 2, it can be seen that the distribution state of the degree of pain, such as being in a state where the overall damage is large as shown in FIG. 10(b), or in a state where the pain at the hair tips is large as shown in FIGS. 10(a) and 10(d), or in a state where the overall damage is small as shown in FIG. 10(c), can be visualized. It has been demonstrated that, as described above, according to the color map image, it is possible to intuitively grasp the indices of the hair state and hair attributes for each part of the hair.

Explanation of Signs

[0059] 10 Information processing apparatus 11 CPU 12 Memory 13 Input / output I / F 14 Communication unit 15 Display device 16 Input device

Claims

1. One or more processors capable of using a trained model that has been machine-learned based on a plurality of teacher data including a plurality of combinations of correct answer information indicating common attributes or states of hair shown in teacher hair images and a group of teacher hair patch images of a predetermined image size extracted from the teacher hair images, obtain an evaluation hair image showing the hair of the person to be evaluated; obtain a group of hair patch images of a predetermined image size from the obtained evaluation hair image; normalize the pixel values for each of the obtained hair patch images; obtain index values for the hair attributes or hair states regarding each hair patch image by inputting each of the normalized hair patch images into the trained model; A hair image analysis method for executing.

2. The one or more processors, generate index value distribution information indicating a distribution of the obtained index values according to the positions of the hair regions shown in each hair patch image. The hair image analysis method according to claim 1, further executing.

3. Generate a color map image in which the obtained index values are mapped to the obtained evaluation hair image according to the positions of the hair regions of each hair patch image as the index value distribution information. The hair image analysis method according to claim 2.

4. The one or more processors, calculate a representative evaluation value of the hair attributes or hair states of the person to be evaluated based on the index values for each of the obtained hair patch images. The hair image analysis method according to any one of claims 1 to 3, further executing.

5. The one or more processors, obtain region designation information for designating a desired image region in the obtained evaluation hair image; calculate a representative evaluation value of the hair attributes or hair states of the image region based on the index values obtained for the group of hair patch images corresponding to the image region indicated by the obtained region designation information. The hair image analysis method according to any one of claims 1 to 4, further executing.

6. The correct answer information indicating the common state of the hair shown in the teacher hair image included in the teacher data indicates either dyed hair or natural hair, For each of the hair patch images, an index value of dyed hair-likeness or natural hair-likeness is obtained as an index value of the hair state. The hair image analysis method according to any one of claims 1 to 5.

7. The correct information indicating the common attributes of the hair shown in the teacher hair image included in the teacher data indicates the actual age or actual age range of the subject of the teacher hair image, Regarding each hair patch image, an index value for each apparent age range of the hair or an index value indicating the apparent age of the hair is respectively obtained as an index value of the hair attribute, The hair image analysis method according to any one of claims 1 to 5.

8. The correct information indicating the common state of the hair shown in the teacher hair image included in the teacher data indicates an evaluation value indicating the degree of gloss, degree of damage, degree of sideburns, or degree of gray hair of the hair, Regarding each hair patch image, an index value indicating the degree of gloss, degree of damage, degree of sideburns, or degree of gray hair of the hair is respectively obtained as an index value of the hair state, The hair image analysis method according to any one of claims 1 to 5.

9. The learned model is composed of a convolutional neural network including a deep neural network, The hair image analysis method according to any one of claims 1 to 8.

10. A method for learning a model for causing a processor to execute a process of calculating an index value of a hair attribute or a hair state of a hair patch image in response to an input of a hair patch image of a predetermined image size, One or more processors, A step of obtaining a plurality of combinations of a teacher hair image and correct information indicating a common attribute or state of the hair shown in the teacher hair image, A step of obtaining a group of teacher hair patch images of a predetermined image size including a hair region at a predetermined ratio or more from the obtained teacher hair image, A step of normalizing pixel values for each of the obtained teacher hair patch images, A step of generating a group of teacher data in which the correct information corresponding to the original teacher hair image from which each teacher hair patch image is obtained is associated with each of the normalized teacher hair patch image groups, A step of learning the model using the group of teacher data, The method for learning the model that executes the above.

11. A hair image analysis apparatus including at least the one or more processors and a memory, A hair image analysis apparatus capable of executing the hair image analysis method according to any one of claims 1 to 9.

12. A model learning apparatus including at least the one or more processors and a memory, A model learning apparatus capable of executing the method for learning the model according to claim 10.

Images