Information processing method for estimating the number of overlapping cuticles, information processing device, and program

Abstract

To provide an information processing method, information processing device, and program that can easily estimate the number of overlapping cuticles. [Solution] An information processing method that estimates the number of overlapping cuticles of a hair based on hair image data obtained by imaging the surface of a hair, and outputs the estimated number of overlapping cuticles.

Description

Technical Field

[0001] The present invention relates to an information processing method for estimating the number of cuticle overlays, an information processing apparatus, and a program.

Background Art

[0002] In beauty salons and the like, the condition of hair is subjectively evaluated based on treatment history, the sensory evaluation of beauticians, and the self-report of the individual, and treatments are performed according to the condition. For example, in perm treatments and coloring treatments, in order to achieve the desired perm strength or hair color tone, a beautician judges the condition of the hair from the hardness and damage degree of the hair, and selects each agent. Therefore, if the selection of the agent is insufficient, the penetration of the agent may be poor, or the agent may penetrate excessively, resulting in an undesired finish. Similarly, for hair care products, beauticians often select and use hair care products after judging the hair condition. However, if the selection of the hair care product is insufficient, the texture such as the feel may not be improved, resulting in an undesired finish.

[0003] In recent years, methods for evaluating the condition of hair using AI have been proposed. For example, according to Patent Document 1, it is possible to judge the condition of hair such as hair age, gloss, straightness, and gender using a program learned from an image of the hair surface.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In hair perms and hair coloring, the number of overlapping cuticles is a useful factor in selecting the appropriate chemicals, as the chemicals penetrate by widening the gaps in the cuticles that cover the surface of the hair. Furthermore, since hair care products improve texture, shine, and other qualities by reducing the unevenness between the cuticles on the hair surface, the number of overlapping cuticles is a useful factor in selecting appropriate hair care products. On the other hand, the cuticle is gradually damaged by everyday external stimuli such as ultraviolet rays, so the number of overlapping cuticle layers varies greatly from person to person. Furthermore, determining the number of overlapping cuticle layers requires observing the cross-section of the hair using specific equipment such as an electron microscope, which has not been easily done in places like hair salons.

[0006] The present invention has been made in view of the above problems, and its purpose is to provide an information processing method, an information processing device, and a program that can easily estimate the number of overlapping cuticles. [Means for solving the problem]

[0007] [1] An information processing method that estimates the number of overlapping cuticles of a hair based on hair image data obtained by imaging the surface of a hair, and outputs the estimated number of overlapping cuticles. [2] The information processing method according to [1], wherein the acquired hair image data is input to a learning model that has been trained to output the number of overlapping cuticles of the hair when the hair image data is input, and the number of overlapping cuticles of the hair is estimated. [3] The learning model estimates the number of overlapping cuticles of the hair based on feature quantities identified based on regions demarcated by the contours of each cuticle of the hair recognized from the hair image data, wherein the feature quantities are at least five selected from the group consisting of the diagonal width of the region, the short side of the circumscribed rectangle, the mode reference angle, the area, the inclination angle, the number of cuticles, the equivalent diameter of the circle, the aspect ratio, the average pixel value, the circularity, the absolute maximum length, the long side of the circumscribed rectangle, and the hair width, as described in [2]. [4] The information processing method according to [2], wherein the feature quantities are at least four selected from the group consisting of the diagonal width of the region, the shorter side of the circumscribed rectangle, the mode reference angle, the area, the inclination angle, the number of cuticles, the equivalent diameter of the circle, the aspect ratio, and the average pixel value. [5] The information processing method according to [2], wherein the feature quantities are at least two selected from the group consisting of the diagonal width of the region, the shorter side of the circumscribed rectangle, the mode reference angle, the area, and the slope angle. [6] The information processing method according to [2], wherein the feature quantity is at least one selected from the group consisting of the diagonal width of the region, the shorter side of the circumscribed rectangle, the mode reference angle, and the equivalent diameter of the circle. [7] An information processing device that estimates the number of overlapping cuticles of a hair based on hair image data obtained by imaging the surface of a hair, and outputs the estimated number of overlapping cuticles, An information processing device that estimates the number of overlapping cuticles in a hair by inputting image data into a trained model that has been trained to output the number of overlapping cuticles in a hair when hair image data is input. [8] A program that estimates the number of overlapping cuticles of a hair based on hair image data obtained by imaging the surface of a hair, and outputs the estimated number of overlapping cuticles, A program having a learning model that uses hair image data and the number of overlapping cuticles of the hair as training data to output the number of overlapping cuticles of the hair. [Effects of the Invention]

[0008] According to the information processing method, information processing device, and program of the present invention, it is possible to easily estimate the number of overlapping cuticles from an image of the hair surface. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a schematic diagram showing an example configuration of the hardware (A) and functional blocks (B) of the estimation device. [Figure 2]Figure 2(A) is an image of the captured hair surface, and Figure 2(B) is an image showing the region of cuticle fragments recognized by the region recognition unit (region recognition model). [Figure 3] Figure 3 is a schematic diagram showing an example configuration of the hardware (A) and functional blocks (B) of a training data generation device. [Figure 4] Figure 4 is a flowchart showing the processing procedure of the training data generation method of the present invention. [Figure 5] Figure 5 is a schematic diagram showing an example configuration of the hardware (A) and functional blocks (B) of a machine learning device. [Figure 6] Figure 6 is a flowchart showing the processing procedure of the machine learning method of the present invention. [Figure 7] Figure 7 is a flowchart showing the processing procedure for the cuticle overlap count estimation method of the present invention. [Figure 8] Figure 8 is a schematic diagram showing an example of the configuration of one embodiment of the present invention. [Figure 9] Figure 9 is a schematic diagram showing an example of the configuration of another embodiment of the present invention. [Modes for carrying out the invention]

[0010] Overview of the estimation device Figure 1 shows one embodiment of the estimation device 1 according to the present invention. The estimation device 1 is an information processing device that estimates the number of overlapping cuticles based on input hair image data. The hardware of the estimation device 1 can be made up of a general-purpose computer, and preferably includes a storage unit 10, a control unit 11, a display unit 12, an input unit 13, and a communication unit 14, as shown in Figure 1(A), with each unit being connected to communicate via communication lines 15 such as bus cables and data cables.

[0011] The storage unit 10 is a non-volatile auxiliary storage means such as a hard disk or a flash memory. A program for estimating the number of overlapping cuticles is stored in the storage unit 10. By inputting a hair surface image into the learned machine learning model (for example, the region recognition model and the estimation model described later), an estimated value of the number of overlapping cuticles is derived. In addition to the learned machine learning model, the storage unit 10 stores various programs for controlling the estimation device 1 and the like. Also, for example, image data of the hair surface of the subject acquired from the imaging device, processed data of the image data performed as necessary, and data related to the image data acquired as necessary (for example, data related to the hair provider such as age, gender, hair length, treatment history, etc.) may be structured and stored like a database, or may be stored independently. These programs and data may be acquired from an external server, storage device, etc. connected via a communication line such as a public communication network or a data cable. The processed data of the image data is obtained by performing various processes (hereinafter, processing) exemplified by noise removal (various filter processes such as an averaging filter and a Gaussian filter, suppression of specific frequency components, etc.), resizing or cropping of the image, color tone correction, contrast adjustment, affine transformation, sharpening, binarization (edge detection, etc.), grayscale conversion, etc. on the captured hair image.

[0012] The control unit 11 is a means for controlling the entire estimation device 1, interpreting programs, and issuing instructions to other devices. The control unit 11 is composed of an arithmetic device such as a CPU or a GPU, and a main storage device such as a ROM and a RAM. The control unit 11 reads and executes the programs stored in the storage unit 10. Also, the control unit 11 reads the image data of the hair surface acquired by an external device, for example, an imaging device, estimates the number of overlapping cuticles of the hair by a program for estimating the number of overlapping cuticles, and displays the result on the display unit 12. The region recognition model and the estimation model described later are used as the program for estimating the number of overlapping cuticles.

[0013] The display unit 12 is information display means such as a liquid crystal display, and displays information for the user on an input image, an estimation result, a processing status, etc. as necessary. Further, the display unit 12 may be an information display device such as an external user terminal or display connected via the communication unit 14.

[0014] The input unit 13 inputs information regarding the user's attributes, the state of the hair, etc. and control commands for the control unit 11 as necessary to the estimation device 1. The input unit 13 is input means such as a keyboard, a touch pen, a mouse, a touch panel. Further, the input means may be a user terminal connected via the communication unit 14.

[0015] The communication unit 14 performs information communication with an external device via a communication network (for example, the communication network 4 in FIG. 9) or via a communication line (for example, the information transmission path 7 in FIG. 8). The external device includes at least one of a user terminal, a server, and an imaging device. Further, the external device may be the above information display device.

[0016] Estimation principle of the number of overlapping cuticles Hereinafter, the estimation principle of the number of overlapping cuticles according to an embodiment of the present invention will be described. The estimation device 1 estimates the number of overlapping cuticles based on this principle.

[0017] As a result of investigations by the present inventors, it has been found that the number of overlapping cuticles can be estimated from the characteristics of the cuticle covering the hair surface. When observed with a microscope, it can be seen that the hair surface is formed of a plurality of layers of overlapping keratinized plate-like cells (cuticles). Since the cuticle has a structure in which a plurality of layers overlap, a step is generated by the overlap between a certain cuticle and another adjacent cuticle, and this step serves as a boundary line to form the contour of each cuticle (hereinafter sometimes referred to as a scaly region), which can be distinguished from other cuticles. As a result of investigating the relationship between the number of overlapping cuticles examined from a hair cross-section and the scaly region on the hair surface, it has been found that the number of overlapping cuticles and the scaly region are correlated. For example, hair with a large number of overlapping cuticles tends to have a narrower flake-like region, while hair with a small number of overlapping cuticles tends to have a wider flake-like region. As described below, the inventors found that some of the features extracted from the flaky regions have a high correlation with the number of overlapping cuticles. In other words, the number of overlapping cuticles estimated based on specific features has a high correlation with the number of overlapping cuticles counted from the hair cross-section. The number of overlapping cuticles is a value obtained by weighting and averaging in all directions from the center of the hair cross-section. The number of overlapping cuticles estimated by this invention is also a weighted average value.

[0018] Figure 2(A) is an image of the captured hair surface, and Figure 2(B) shows the region of cuticle fragments (scaly region: colored area) recognized by the region recognition model. Table 1 below shows the results of investigating the correlation between features ("explanatory variables") and the number of overlapping cuticles based on a large amount of sample data.

[0019] [Table 1]

[0020] In the table, the p-value represents the probability that the correlation coefficient of data obtained by randomly swapping sample data exceeds the original correlation coefficient. If the p-value is less than 0.05 (p<0.05), it was considered a significant variable (feature). Each feature considered to be a significant variable was grouped based on its correlation with the absolute value of its correlation coefficient. Strong: p < 0.05, and the absolute value is 0.580 or greater. Middle: p < 0.05, and the absolute value is 0.400 or greater and less than 0.580. Weak: p < 0.05 and absolute value less than 0.400 -: Other than the above

[0021] Then, an estimation model was created based on the number of features used within the correlation group (in Table 2, "Number of Variables Used"), and the number of overlapping cuticles was estimated. This estimation result was compared with the estimation result (baseline) based on a model that outputs the mean value of the test data, and the following evaluation was performed. Note that all models used random forest. ○: All combinations of features within the correlation group exceeded the baseline. △: Of all the feature combinations, more than half exceeded the baseline. ×: Less than half of all feature combinations exceeded the baseline. To be considered "above the baseline," the RMSE (Root Mean Squared Error) of the model combining the features within the correlation group is smaller than the RMSE (=1.2766) of the model outputting the mean of the test data. In this invention, combinations exceeding the baseline were evaluated as significant predictive models. The results are shown in Table 2.

[0022] [Table 2]

[0023] For example, in Table 1, there are 7 features ("explanatory variables") whose "correlation group" is "strong". "Variables used" is the number of features used in the combination of these features. Since there are 7 features belonging to the "strong" group, if there are 2 variables used, there are 21 possible combinations. Using these 21 prediction models, we estimated the number of overlapping cuticles for each, and all 21 combinations exceeded the baseline prediction performance (21 / 21), so we gave them a ○ rating. Based on the above results, the present invention determined the preferred feature quantities and the number of combinations thereof, as described below.

[0024] (Configuration of the machine learning system) The learning model used to estimate the number of overlapping cuticles in estimation device 1 is described below. First, we will explain the training data based on Figure 3 (Hardware configuration diagram (A) and example configuration of functional blocks (B) of the training data generation device 26) and Figure 4 (Flowchart diagram). The training data generation device 26 generates the training data necessary for machine learning of the machine learning model. The hardware of the training data generation device 26 may be a general-purpose computer. The hardware configuration diagram (A) of the training data generation device 26 shown in Figure 3 is the same as the hardware configuration diagram (A) of the estimation device 1 shown in Figure 1, so its explanation is omitted. The storage unit 10 of the training data generation device 26 stores the training data generation program, training data, cuticle overlap count data, a training result storage area, and various programs for controlling the training data generation device 26. The training data generation program, training data, cuticle overlap count data, and each control program can be installed in the storage unit 10 via a storage medium such as a USB memory or a network. Figure 3(B) is a schematic diagram showing an example of the functional blocks of the training data generation device 26. The training data generation device 26 has, as functional blocks of the training data generation program, an image acquisition unit 31, a region identification unit 32, a labeling unit 33, an image storage unit 34, an image acquisition unit 35 for acquiring labeled image data, a feature quantity calculation unit 36, a labeling unit 37, and a data storage unit 38. Each of these units is realized by the control unit 11 shown in Figure 1, in which an arithmetic unit such as a CPU executes a program stored in the storage unit 10 using the main memory such as RAM as a working area. Some of the data may be input by a human.

[0025] In this invention, it is preferable to generate training data by performing a two-stage process. In the first stage, the cuticle region is identified from the hair surface image. In the second stage, feature quantities are extracted from the identified cuticle region, and these feature quantities are associated with the number of overlapping cuticles in the hair.

[0026] The memory unit 10 stores training data necessary for creating training data for the region recognition model. The training data is a large dataset of images (hereinafter referred to as image data) of the hair surface captured by a microscope. The image data is an image of the hair surface captured by a microscope under magnification so that the cuticle of the hair surface can be identified. The position of the hair to be imaged is not particularly limited and may be at the tip, root, or any intermediate position. The image data may be preprocessed as needed, such as by noise reduction. As the first stage of processing, a large number of datasets (hereinafter referred to as training data for the region recognition model) are created in which image data and the regions of each cuticle are associated as ground truth data.

[0027] The steps shown in Figure 4 are as follows: Step 1 (Image Data Acquisition Step): As the first step, the image acquisition unit 31 acquires image data from the storage unit 10.

[0028] Step 2 (Region Identification Step): Next, the region identification unit 32 identifies the region of each cuticle from the image data. Identifying the cuticle region involves defining the contour of each cuticle by enclosing it with a boundary line based on the step difference caused by the overlapping of each cuticle. The method for defining the contour of each cuticle is not particularly limited; the contour of the region may be detected based on edges detected using techniques such as edge detection, or image data may be displayed on the display unit 12, and a person may input the boundary line using an input means such as a stylus pen from the input unit 13 while viewing the image data. A combination of these methods may also be used; for example, a person may correct the detected contour. Furthermore, the region identification unit 32 identifies the coordinates of each cuticle region demarcated by the contour.

[0029] Step 3 (Labeling Step): Next, the labeling unit 33 labels the coordinate information of the region identified by the region identification unit 32 to each cuticle in the image data. Labeling may be done automatically using segmentation tools or annotation tools, or it may be done manually by a human via the display unit 12 and input unit 13. Alternatively, a combination of these methods may be used; for example, initial labeling may be done by an automated tool, and then a human may review and correct it.

[0030] Step 4 (Data Storage Step): Next, the image storage unit 34 stores the image data, which has the coordinate information of each labeled cuticle, in the learning result storage area of ​​the storage unit 10 as training data for the region recognition model.

[0031] Next, based on this saved training data for the region recognition model, a large number of datasets (training data for the estimation model) are created in which the feature quantities calculated from each cuticle region and the number of overlapping cuticles are associated as ground truth data. Furthermore, the training data for the region recognition model is not limited to the saved training data for the region recognition model described above; for example, data obtained through augmentation (processing the original data to generate pseudo-new data) may also be used.

[0032] Step 5 (Image Data Acquisition Step): The image acquisition unit 35 acquires the labeled image data (training data for the region recognition model) from the storage unit 10.

[0033] Step 6 (Feature Calculation Step): The feature calculation unit 36 ​​calculates the feature quantities of the cuticle regions demarcated by the boundary lines. Preferably, the feature quantities include at least the above-mentioned feature quantities. Feature quantities are quantitative values ​​calculated based on the regions demarcated by the contours of each cuticle extracted from image data, and can be calculated using various well-known programs such as OpenCV.

[0034] Step 7 (Labeling Step): The labeling unit 37 associates the data of each feature with the cuticle overlap count data and labels them. The cuticle overlap count data is read from the storage unit 10 into the work area as appropriate, and the feature data is labeled. The cuticle layer count data is calculated in advance based on the number of cuticles measured by cutting the imaged hair and observing the cross-section under a microscope. The layer count is obtained by weight-averaging in all directions from the center of the hair cross-section.

[0035] Step 8 (Data Storage Step): Next, the data storage unit 38 stores the labeled dataset as training data for the estimation model in the learning result storage area of ​​the memory unit 10.

[0036] Next, machine learning using the above-mentioned training data will be explained based on Figure 5 (Hardware configuration diagram (A) and example configuration of functional blocks (B) of the machine learning device 27) and Figure 6 (flowchart). The hardware of the machine learning device 27 may be a general-purpose computer. The hardware configuration diagram (A) of the machine learning device 27 shown in Figure 5 is the same as the hardware configuration diagram (A) of the estimation device 1 shown in Figure 1, so its explanation is omitted. The machine learning device 27 has the function of constructing a machine learning model using each training data and training the constructed machine learning model. The memory unit 10 of the machine learning device 27 contains machine learning programs, the generated training data (training data for the region recognition model, training data for the estimation model), a learning result storage area, and various programs for operating the machine learning device. Each program and training data can be installed in the memory unit 10 via a storage medium such as a USB memory stick or a network.

[0037] In this embodiment, the machine learning device 27 may be located on the cloud, or it may be installed in the same location as the training data generation device 26. Alternatively, the training data generation device 26 and the machine learning device 27 may be configured as an integrated unit. As shown in Figure 5(B), the machine learning device 27 includes a learning unit 41 and a learning result output unit 42 as functional blocks.

[0038] The steps shown in Figure 6 are as follows: Step 9 (Machine Learning Step): When the control unit 11 detects that the user has performed a predetermined operation via the input unit 13, it starts the machine learning program. The learning unit 41 uses the training data for the region recognition model to construct a region recognition model using the machine learning program, and uses the training data for the estimation model to construct an estimation model using the machine learning program.

[0039] The machine learning programs (machine learning programs for region recognition models, machine learning programs for estimation models) are not particularly limited and any method can be used, such as neural networks, linear discriminant analysis, support vector machines, K-nearest neighbors, random forests, and deep learning. The machine learning programs for region recognition models and estimation models can employ the same or different methods. Training of a machine learning program is implemented in software by the processor of the machine learning device 27, which reads the training program into main memory and executes it.

[0040] In the learning unit 41, a machine learning program for region recognition is trained using training data for region recognition to detect the contours of each cuticle from image data. Training should ideally be repeated until the cuticle contours can be automatically detected from new image data and the error of the detected contours (boundaries) meets a predetermined standard. Tuning may be performed as needed. The parameters at the end of the learning process are stored in the learning result storage area of ​​the memory unit 10.

[0041] Furthermore, the learning unit 41 trains a machine learning program for estimation to estimate the number of overlapping cuticles from features using training data for the estimation model. Training should ideally be repeated until the error between the estimated number of overlapping cuticles from new image data and the measured value meets a predetermined standard. Tuning may be performed as needed. The parameters at the end of the learning process are stored in the learning result storage area of ​​the memory unit 10.

[0042] Furthermore, the estimation device 1 may update the training data for the region recognition model and the training data for the estimation model using, for example, the results of new analyses performed by the region recognition model and estimation model generated through the above learning process. This increases the amount of training data, and it is expected that a highly accurate trained model can be created.

[0043] The information processing method of the present invention (hereinafter referred to as the cuticle layer count estimation method) will be described below with reference to Figures 1, 7, 8, and 9. Figure 1(A) is a hardware configuration diagram of estimation device 1, and Figure 1(B) is a functional block diagram of estimation device 1. Figure 7 is a flowchart showing the processing procedure of the cuticle overlap count estimation method, and these processes are performed by estimation device 1. Figures 8 and 9 are examples of the configuration of various devices attached to estimation device 1. The region recognition model and estimation model generated through the above learning process are stored in the memory unit 10 of the estimation device 1 and applied to the control unit 11 for estimating the number of overlapping cuticles. By inputting image data of the subject's hair surface into each model, the computer is made to estimate the number of overlapping cuticles and outputs an estimated value.

[0044] The estimation device 1 is connected to the external imaging device 5 via data communication, as illustrated in Figures 8 and 9. When the estimation device 1 is started and each part, device, and program is ready to operate, it enters the start state shown in Figure 7.

[0045] Step 100 (Image Acquisition Step): The control unit 11 receives a command from the user and instructs the image acquisition unit 20 to read an image of the hair surface (hereinafter referred to as the target image D1). The image is an enlarged image of the hair surface collected from the subject, which is used to identify the cuticle. The position of the hair to be imaged is not particularly limited. The hair may be imaged after excision or without excision. Furthermore, the images may be pre-processed as needed, such as by noise reduction, as described above. The image acquisition unit 20 may acquire the target image D1 from the storage unit 10. Alternatively, as shown in Figure 8, the imaging device 5 connected to the estimation device 1 by an information transmission path 7 (e.g., wired communication or wireless communication) may capture images of the hair and acquire the target image D1. Furthermore, as shown in Figure 9, the target image D1 may be acquired via a user terminal 2 connected via a communication network 4 such as the Internet, or from an imaging device 5 connected to the communication network 4 without going through the user terminal 2. The target image D1 may also be acquired from an external storage device such as a server 3 via the communication network 4. Alternatively, the target image D1 may be acquired from a removable storage device such as a USB or flash memory that can be connected to the estimation device 1 as needed. The image acquisition unit 20 reads the target image D1 and saves the image data to a storage device such as RAM.

[0046] Step 101 (Region Recognition Step): In the control unit 11, the region recognition unit 21 recognizes a scale-like region surrounded by a boundary line using a region recognition model for the target image D1 acquired by the target image acquisition unit 20 in the image acquisition step.

[0047] Step 102 (Feature Extraction Step): In the control unit 11, the feature calculation unit 22 calculates the feature quantities of the region recognized by the region recognition unit 21. The feature quantities are region information of the cuticle, such as the size and shape of the recognized region, and are calculated by image analysis. Various known programs can be used for image analysis depending on the feature quantities. The specific features are the diagonal width of the region, the shorter side of the circumscribing rectangle of the region, the mode reference angle of the region, the area of ​​the region, the slope angle of the region, the number of cuticles in the image, the equivalent diameter of the region, the aspect ratio of the region, the average pixel value of the region, the circularity of the region, the absolute maximum length of the region, the longer side of the circumscribing rectangle of the region, the hair width, and their mean and standard deviation. A predicted value for the number of overlapping cuticles is calculated based on some or all of these features. Of the above features, at least five are preferably selected from the group consisting of mean diagonal width, mean short side of the circumscribing rectangle, std diagonal width, std mode reference angle, mean area, std slope angle, mean cuticle count, mean equivalent circle diameter, mean aspect ratio, std equivalent circle diameter, std average pixel value, mean circularity, mean absolute maximum length, std long side of the circumscribing rectangle, mean hair width, std circularity, and std absolute maximum length. As will be described later, these combinations are preferable because they can calculate estimates that exceed the baseline. More preferably, the features are at least four selected from the group consisting of mean diagonal width, mean short side of the circumscribing rectangle, std diagonal width, std mode reference angle, mean area, std slope angle, mean number of cuticles, mean equivalent circle diameter, aspect ratio, and std average pixel value. These features correlate more strongly with the number of overlapping cuticles than other features, and their combinations are preferred because they can produce estimates that exceed the baseline. More preferably, these features are at least two selected from the group consisting of mean diagonal width, mean short side of the circumscribed rectangle, std diagonal width, std mode reference angle, mean area, and std slope angle. These features are preferred because they correlate more strongly with the number of overlapping cuticles than other features, and their combinations can produce estimates that exceed the baseline. More preferably, it is at least one selected from the group consisting of mean diagonal width, mean short side of the circumscribed rectangle, std mode reference angle, and std equivalent circle diameter. These are preferred because they can each produce estimates that exceed the baseline even when used alone.

[0048] Step 103 (Cuticle Overlap Estimation Step): In the control unit 11, the cuticle overlap estimation unit (hereinafter, estimation unit) 23 estimates the cuticle overlap using an estimation model based on the features calculated by the feature calculation unit 22. The estimation model has learned the correlation between features and the cuticle overlap, as shown in Tables 1 and 2, through machine learning, and outputs an estimated result for the cuticle overlap based on the input features. The estimated number of overlapping cuticles is a value obtained by weighting and averaging the number of layers in all directions from the center of the hair cross-section. In the control unit 11, the estimation result is output by the output unit 24. The output destination may be the display unit 12, or an external display via the communication unit 14.

[0049] Table 3 shows the number of overlapping cuticles estimated based on the cuticle overlap estimation method described above, using the collected hair samples 1-4, as well as the number of overlapping cuticles measured by microscopic observation of each hair sample. The circles in the table indicate the features used. As shown in Table 3, the cuticle layer count estimation method of the present invention can output an estimated number of hair cuticle layers that approximates the measured number.

[0050] [Table 3]

[0051] Another embodiment of the present invention will be described with reference to Figure 9. Figure 9 is a schematic diagram showing an example configuration in which the estimation device 1 of the present invention is connected to a user terminal 2 via a communication network 4. In the illustrated example, the estimation device 1 and the user terminal 2 are connected via a communication network 4, and the imaging device 5 is connected to the user terminal 2 via an information transmission path 7. An external storage device 3 is connected to the communication network 4 as needed.

[0052] The estimation device 1 has the configuration described above, and its operation is as described above. In the above description, the estimation results were displayed on the display of the estimation device 1, but if a user terminal 2 is connected, the estimation results may be displayed on the user terminal 2.

[0053] User terminal 2 is a terminal used by a user of the estimation device 1 of the present invention. The user terminal 2 is connected to the estimation device 1 via the communication network 4, and also to the external storage device 3 as needed. The user terminal 2 has the function of transmitting image data captured by the imaging device 5 to the estimation device 1 or the external storage device 3 via the communication network 4, and receiving processed data from the estimation device 1 or the external storage device 3. The user terminal 2 is also preferably equipped with a display means such as a display, which displays the received data. For example, the display of the user terminal 2 displays the number of overlapping cuticles. The display of the user terminal 2 may also display arbitrary data such as hair images captured by the imaging device 5 or user information acquired as needed. User terminal 2 may also have at least some of the functions of estimation device 1 as additional functions. User terminal 2 can be an information processing terminal such as a personal computer (PC), or a mobile information terminal such as a smartphone or tablet, and these can be either general-purpose terminals or dedicated terminals. The operation of estimation device 1 is as described above.

[0054] External storage device 3 is a storage medium that stores various types of data, such as images. In Figure 9, external storage device 3 stores various types of data transmitted from estimation device 1 and user terminal 2, and transmits the stored data to estimation device 1 and user terminal 2 as needed. The external storage device 3 may have functions other than storage (hereinafter sometimes referred to as "other functions"). These other functions may include, for example, at least some of the functions of the estimation device 1. Furthermore, the external storage device 3 may perform at least some of the processing on behalf of the estimation device 1 based on its other functions.

[0055] Communication network 4 is a communication transmission path that performs the function of transmitting information, and examples include various IP networks such as the Internet, intranets, and computer networks, as well as telephone line communication networks and telecommunication networks.

[0056] The imaging device 5 is a means of capturing images of hair obtained by capturing images of the hair surface. The imaging device 5 transmits the image data of the hair obtained by capturing images of the hair surface to the user terminal 2 via an information transmission path 7, which can be a wired connection such as a USB cable or a wireless connection such as Bluetooth®, infrared communication, or Wi-Fi. The information transmission means 7 may be a communication network 4. Although not shown in the diagram, the imaging device 5 may also be connected to the estimation device 1 in a way that allows data transmission without going through the user terminal 2. Alternatively, the image data captured by the imaging device 5 may be saved to an auxiliary storage device such as a memory card without going through the information transmission path 7, and the image data may be input to the user terminal 2 using the auxiliary storage device. Examples of imaging devices 5 include digital cameras, video cameras, microscopes, electron microscopes, and laser microscopes. The captured hair can be either still images or moving images; in the case of moving images, still images can be arbitrarily extracted. To analyze the hair surface more accurately, it is preferable that the imaging device 5 is equipped with a lens capable of imaging the hair surface at high magnification, and a microscope is preferred. The high magnification is preferably 300x, more preferably 500x, and even more preferably 700x. The imaging device 5 may be equipped with lighting fixtures as needed, and the hair should be illuminated as necessary when imaging hair.

[0057] Hair 6 is any hair from the subject. Furthermore, the imaging location for the hair can be any part of the hair shaft.

[0058] As shown in Figure 9, by connecting the estimation device 1 to the communication network 4, the number of overlapping hair cuticles can be estimated even from a remote location. [Industrial applicability]

[0059] According to the present invention, the number of overlapping cuticles can be estimated from the surface of the hair, allowing for the adjustment of perming agents and other products to a more appropriate formulation in beauty salons, making it easier to achieve the desired perm results and hair color development. Furthermore, it enables the appropriate selection of hair care products such as shampoos and hair treatments according to hair type. [Explanation of Symbols]

[0060] 1 Estimation device 2 User terminals 3 External storage device 4. Communication Network 5. Imaging device 6 Hair 7. Information transmission channels 10 Storage section 11 Control Unit 12 Display section 13 Input section 14 Communications Department 15 Communication lines 20 Image acquisition unit 21 Area recognition part 22 Feature Calculation Unit 23 Estimation part 24 Output section 26 Training Data Generation Device 27 Machine Learning Equipment 31 Image acquisition unit 32 Area identification part 33 Labeling section 34 Image storage section 35 Image acquisition unit 36 Feature Calculation Unit 37 Labeling section 38 Data Storage Section 41 Learning Department 42 Learning Result Output Unit D1 Target image

Claims

1. Based on hair image data obtained by imaging the hair surface, the number of overlapping cuticles in the hair is estimated. An information processing method that outputs the estimated number of overlapping cuticles.

2. The acquired hair image data is input to a learning model that has been trained to output the number of overlapping cuticles of the hair when the aforementioned hair image data is input, and the number of overlapping cuticles of the hair is estimated. The information processing method according to claim 1.

3. The learning model estimates the number of overlapping cuticles in the hair based on features identified based on regions demarcated by the contours of each cuticle of the hair recognized from the hair image data. The information processing method according to claim 2, wherein the feature quantities are at least five selected from the group consisting of the diagonal width of the region, the shorter side of the circumscribed rectangle, the mode reference angle, the area, the inclination angle, the number of cuticles, the equivalent diameter of the circle, the aspect ratio, the average pixel value, the circularity, the absolute maximum length, the longer side of the circumscribed rectangle, and the hair width.

4. The information processing method according to claim 2, wherein the feature quantities are at least four selected from the group consisting of the diagonal width of the region, the shorter side of the circumscribed rectangle, the mode reference angle, the area, the inclination angle, the number of cuticles, the equivalent diameter of the circle, the aspect ratio, and the average pixel value.

5. The information processing method according to claim 2, wherein the feature quantities are at least two selected from the group consisting of the diagonal width of the region, the shorter side of the circumscribed rectangle, the mode reference angle, the area, and the slope angle.

6. The information processing method according to claim 2, wherein the feature quantity is at least one selected from the group consisting of the diagonal width of the region, the shorter side of the circumscribed rectangle, the mode reference angle, and the equivalent diameter of a circle.

7. An information processing device that estimates the number of overlapping cuticles in a hair based on hair image data obtained by imaging the surface of the hair, and outputs the estimated number of overlapping cuticles, An information processing device that estimates the number of overlapping cuticles in a hair by inputting image data into a trained model that has been trained to output the number of overlapping cuticles in a hair when image data of a hair is input.

8. A program that estimates the number of overlapping cuticles in a hair based on image data of the hair surface obtained by imaging the hair surface, and outputs the estimated number of overlapping cuticles, A program having a learning model that uses hair image data and the number of overlapping cuticles of the hair as training data to output the number of overlapping cuticles of the hair.

Images