Hair property estimation device, hair property estimation method, hair property estimation program, hair property estimation system, mobile terminal, program for mobile terminal, and storage medium

Abstract

The present invention is a hair property estimation device comprising: a subject image acquisition unit that acquires a subject image including a subject from the top of their head to at least their jaw tip; a contour line extraction unit that extracts contour lines of the hair of the subject in the acquired subject image; and a hair property estimation unit that estimates properties of the hair of the subject on the basis of the extracted contour lines.

Description

Hair property estimation device, hair property estimation method, hair property estimation program, hair property estimation system, mobile terminal, program for mobile terminal, and storage medium

[0001] The present invention relates to a hair property estimating device, a hair property estimating method, a hair property estimating program, a hair property estimating system, a mobile terminal, a program for a mobile terminal, and a storage medium.

[0002] In the above technical fields, Patent Document 1 discloses a method of capturing an image of a person passing in front of an image display device using an image sensor, extracting the person from the captured image, determining the person's characteristics (customer characteristics), and displaying an advertisement corresponding to the determined characteristics on the image display device (paragraph

[0023] , etc.). Patent Document 2 discloses a method of evaluating a hairstyle by grasping the facial contours and image created by a hairstyle defined by an inner line, which is the boundary between the face and the hair, and an outer line, which is the outer line of the hairstyle, and analyzing whether the hairstyle suits the person based on the shape, balance, and image (paragraphs

[0014] to

[0029] , claim 1, etc.). Patent Document 3 discloses a method of extracting features such as hairstyle, hair length, and hair color from a captured image of a person to be identified, and determining whether the person to be identified is a registered person based on the similarity between the extracted features and registered features (paragraphs

[0011] to

[0012] , claim 1, etc.).

[0003] JP2002-073321A JP11-56469A JP11-328401A

[0004] The present invention relates to a hair condition estimating device. In one embodiment, the hair condition estimating device preferably includes: a subject image acquiring unit that acquires a subject image including a subject's head; a contour line extracting unit that extracts a contour line of the subject's hair in the acquired subject image; and a hair condition estimating unit that estimates the hair condition of the subject based on the extracted contour line.

[0005] The present invention also relates to a hair property estimating method. In one embodiment, the hair property estimating method preferably includes: a subject image acquiring step of acquiring a subject image including a portion from a top of the head to at least a tip of the chin of the subject; a contour line extracting step of extracting a contour line of the subject's hair in the acquired subject image; and a hair property estimating step of estimating a property of the subject's hair based on the extracted contour line.

[0006] The present invention further relates to a hair property estimation program. In one embodiment, the hair property estimation program preferably causes a computer to execute the following steps: a subject image acquisition step of acquiring a subject image including a subject's head, a contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image, and a hair property estimation step of estimating the property of the subject's hair based on the extracted contour line.

[0007] Furthermore, the present invention relates to a storage medium. In one embodiment, the storage medium preferably stores a hair property estimation program that causes a computer to execute the following steps: a subject image acquisition step of acquiring a subject image including the subject's head, a contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image, and a hair property estimation step of estimating the property of the subject's hair based on the extracted contour line.

[0008] The present invention further relates to a hair property estimating system. In one embodiment, the hair property estimating system preferably includes a hair property estimating device and a mobile terminal. In one embodiment, the hair property estimating system preferably includes: a subject image acquiring unit that acquires a subject image including the subject's head; a contour line extracting unit that extracts a contour line of the subject's hair in the acquired subject image; and a hair property estimating unit that estimates the property of the subject's hair based on the extracted contour line.

[0009] Furthermore, the present invention relates to a mobile terminal. In one embodiment, the mobile terminal preferably includes: a camera; a subject image acquisition unit that acquires an image of a subject captured by the camera; a contour line extraction unit that extracts a contour line of the subject's hair from the acquired image of the subject; a display control unit that displays the contour line extracted by the contour line extraction unit so as to be superimposed on the image of the subject; a display unit that displays a suitability check image; and a transmission unit that transmits the contour line of the hair extracted by the contour line extraction unit to a hair condition estimating device.

[0010] The present invention further relates to a program for a mobile terminal. In one embodiment, the program for a mobile terminal is preferably a program for a mobile terminal used in a hair property estimating system. In one embodiment, the program for a mobile terminal preferably causes the mobile terminal to execute: a contour line extraction step of extracting a hair contour line from a subject image; and a transmission control step of transmitting the hair contour line extracted by the contour line extraction unit to a hair property estimating device.

[0011] FIG. 1 is a diagram for explaining an overview of the operation of the hair property estimating device according to the first embodiment of the present invention; FIG. 2 is a block diagram for explaining the configuration of the hair property estimating device according to the first embodiment of the present invention; FIG. 3 is a diagram for explaining an example of contour line extraction by the contour line extraction unit of the hair property estimating device according to the first embodiment of the present invention; FIG. 4 is a diagram for explaining an example of a contour line extracted by the contour line extraction unit of the hair property estimating device according to the first embodiment of the present invention; FIG. 5 is a diagram for explaining extraction of contour feature amounts by the hair property estimating unit of the hair property estimating device according to the first embodiment of the present invention; FIG. 6 is a diagram for explaining extraction of an angle as a contour feature amount by the hair property estimating unit of the hair property estimating device according to the first embodiment of the present invention; FIG. 7 is a diagram for explaining an example of a method for generating a trained hair property estimation model used by the hair property estimating unit of the hair property estimating device according to the first embodiment of the present invention; FIG. 8 is a diagram for explaining a correlation between a contour feature amount and a hair property calculated by the hair property estimating unit of the hair property estimating device according to the first embodiment of the present invention; FIG. 9 is a diagram for explaining a correlation between a contour angle and a hair property calculated by the hair property estimating unit of the hair property estimating device according to the first embodiment of the present invention. FIG. 1 is a diagram for explaining the correlation between the length of a contour line and hair properties by the hair property estimation unit of the hair property estimation device according to the first embodiment of the present invention. FIG. 2 is a diagram for explaining estimation results by the hair property estimation unit of the hair property estimation device according to the first embodiment of the present invention. FIG. 3 is a diagram for explaining estimation results by the hair property estimation unit of the hair property estimation device according to the first embodiment of the present invention, when only the angle of the contour line is used as a feature quantity, and when only the length of the contour line is used as a feature quantity. FIG. 4 is a diagram for explaining estimation results by the hair property estimation unit of the hair property estimation device according to the first embodiment of the present invention, when only the feature quantities of each of the regions S1 to S3 are used. FIG. 5 is a diagram for explaining estimation results by the hair property estimation unit of the hair property estimation device according to the first embodiment of the present invention, when only the feature quantities of each of the regions S4 to S6 are used. FIG. 6 is a diagram for explaining estimation results by the hair property estimation unit of the hair property estimation device according to the first embodiment of the present invention, when feature quantities of multiple regions are used. FIG. 7 is a diagram collectively showing estimation results by the hair property estimation unit of the hair property estimation device according to the first embodiment of the present invention.FIG. 1 is a diagram for explaining an example of normalization by a normalization unit of the hair property estimating device according to the first embodiment of the present invention. FIG. 2 is a diagram for explaining an example of a hair property type table included in the hair property estimating device according to the first embodiment of the present invention. FIG. 3 is a diagram for explaining the hardware configuration of the hair property estimating device according to the first embodiment of the present invention. FIG. 4 is a flowchart for explaining a processing procedure of the hair property estimating device according to the first embodiment of the present invention. FIG. 5 is a flowchart for explaining a modified example of the processing procedure of the hair property estimating device according to the first embodiment of the present invention. FIG. 6 is a block diagram for explaining the configuration of a hair property estimating device according to a second embodiment of the present invention. FIG. 7 is a diagram for explaining an example of an action table included in the hair property estimating device according to the second embodiment of the present invention. FIG. 8 is a diagram for explaining the hardware configuration of the hair property estimating device according to the second embodiment of the present invention. FIG. 9 is a flowchart for explaining the processing procedure of the hair property estimating device according to the second embodiment of the present invention. FIG. 10 is a block diagram for explaining the configuration of a hair property estimating device according to a third embodiment of the present invention. FIG. 11 is a diagram for explaining an example of a hair care plan table included in the hair property estimating device according to the third embodiment of the present invention. FIG. 12 is a diagram for explaining the hardware configuration of the hair property estimating device according to the third embodiment of the present invention. FIG. 13 is a flowchart for explaining the processing procedure of the hair property estimating device according to the third embodiment of the present invention. FIG. 1 is a diagram for explaining an overview of a hair property estimating system according to a fourth embodiment of the present invention. FIG. 2 is a block diagram for explaining the configuration of a hair property estimating system according to a fourth embodiment of the present invention. FIG. 3 is a block diagram for explaining the configuration of a hair property estimating system according to a fifth embodiment of the present invention. FIG. 4 is a flowchart for explaining the processing procedure of a mobile terminal included in a hair property estimating system according to a fifth embodiment of the present invention. FIG. 5 is a flowchart for explaining the processing procedure of a hair property estimating server included in a hair property estimating system according to a fifth embodiment of the present invention. FIG. 6 is a flowchart for explaining a modified example of the processing procedure of a mobile terminal included in a hair property estimating system according to a fifth embodiment of the present invention. Detailed Description of the Invention

[0012] The technologies described in the above Patent Documents 1 to 3 display advertisements corresponding to a person's features, evaluate hairstyles from facial contours, and determine whether a person to be identified is registered or not from features such as hairstyle, hair length, and hair color, but are unable to estimate hair properties.

[0013] Hereinafter, embodiments of the present invention will be described in detail by way of example with reference to the drawings. However, the configurations, numerical values, processing flows, functional elements, etc. described in the following embodiments are merely examples, and are open to modification and alteration, and are not intended to limit the technical scope of the present invention to the following description.

[0014] [First Embodiment] A hair property estimation device according to a first embodiment of the present invention will be described with reference to Figs. 1 to 5. Fig. 1 is a diagram for explaining an overview of the operation of a hair property estimation device 100 according to this embodiment. The hair property estimation device 100 is a device that extracts a hair contour 112 of a subject 111 from an image (subject image 110) including the head of the subject 111 (contour image 113) and estimates the hair property of the subject 111. Note that the hair property estimation device 100 of this embodiment includes a server device, a PC (Personal Computer), and a mobile terminal such as a smartphone or tablet terminal. The same applies to other embodiments below.

[0015] The hair condition estimating device 100 extracts a hair contour 112 of the subject 111 from a subject image 110 that shows the subject 111 from the top of the head to, for example, the ends of the hair. The extracted contour 112 is displayed superimposed on the subject image 110, and a contour image 113 that represents the outer shape of the hair of the subject 111 is obtained. The hair condition estimating device 100 then estimates which type (first type 121 to fourth type 124) in the condition classification 120 the hair condition of the subject 111 falls into, based on the shape of the contour portion of the obtained contour image 113 (such as the degree of hair spread).

[0016] As described above, the contour line 112 extracted in this embodiment can be superimposed on the two-dimensional subject image 110 and can be represented as a line in the two-dimensional image. When representing the contour line 112 as a line in the two-dimensional image, the contour line 112 may be displayed as a thick line or a colored line to emphasize that it is the contour line 112.

[0017] Here, the hair property is, for example, the degree of curliness of the subject's 111 hair, and the property classification 120 is classified into four types according to the degree of curliness. The property classification 120 is divided into a first type 121, a second type 122, a third type 123, and a fourth type 124 in order of hair curliness. Note that the classification of hair curls is not limited to four, and may be two to three types, or five or more types.

[0018] The first type 121 is a silhouette that spreads out in an A-line, for example, with the volume gradually increasing from the top of the head to the ends due to the overlapping of curls, resulting in a cut silhouette that appears as an A-line. The second type 122 is a silhouette that spreads out from below the ears, for example, with the width from the top of the head to the ears being somewhat smaller, but the curls becoming more pronounced from around ear height.

[0019] The third type 123 is a silhouette in which the roots stand up and spread out slightly, for example, a type that looks like it could be straightened with a hair dryer, with the roots standing up and air getting in and slightly puffed up. The fourth type 124 is a silhouette that does not spread out, for example, a type in which the roots do not stand up much and the cut silhouette does not puff up. The hair condition estimating device 100 then estimates which type the subject person's 111 hair condition falls into.

[0020] 2A to 2Q, the configuration of the hair condition estimating device 100 will be described. The hair condition estimating device 100 includes a subject image acquiring unit 201, a contour line extracting unit 202, and a hair condition estimating unit 203.

[0021] The subject image acquisition unit 201 acquires a subject image 110 including the head of the subject 111. The subject image 110 is, for example, an image captured by a mobile device such as a smartphone or tablet device carried by the subject 111, or a digital camera. The subject image acquisition unit 201 acquires the subject image 110 from the mobile device or the like via wired communication or wireless communication. The subject image acquisition unit 201 may also acquire the subject image 110 stored in a memory device attached to the mobile device or the like directly from the memory device. Furthermore, the subject image acquisition unit 201 may also acquire the subject image 110 captured by the mobile device or the like and stored in cloud storage from the cloud storage.

[0022] As described above, the subject image 110 includes the head of the subject 111, and preferably includes from the top of the head to at least the chin of the subject 111. The subject image 110 may or may not include the ends of the subject's 111 hair.

[0023] In the example shown in FIG. 1 , the subject image 110 is an image of the subject 111 captured from the front. However, the direction from which the subject 111 is captured is not limited thereto. For example, an image captured from behind (back view), an oblique direction, an obliquely upward direction, or an obliquely downward direction may also be used. However, an image captured from the front is preferable from the perspective of understanding the relationship between the face and the position of each part of the face. Furthermore, the image may be a selfie taken by the subject 111 or an image captured by a camera installed in a store such as a beauty salon or cosmetics store. Note that if the subject image is an image of the subject 111 captured from a direction other than the front, it is not necessarily necessary for the subject 111's chin to be captured in the subject image. For example, if the subject image is an image of the subject 111 captured from behind, it is sufficient that the subject image captures a portion of the subject 111's head from the top of the head to the chin when viewed from behind.

[0024] Furthermore, the subject image 110 may be an image in any state as long as it shows the hair properties of the subject 111, but a preferred image is selected depending on the hair properties to be estimated. For example, when estimating the degree of hair curl, an image showing the state of bare hair is preferred, such as an image taken one hour after towel drying after washing or an image taken after air drying after washing. On the other hand, when estimating the finished styling, an image taken immediately after styling is preferred.

[0025] The subject image 110 acquired by the subject image acquisition unit 201 may be one or multiple for the same subject 111. Furthermore, when the subject image acquisition unit 201 acquires multiple subject images 110, the images may include images taken in different shooting directions, or multiple images taken in the same shooting direction. Note that the subject image 110 may be a still image or a video.

[0026] The contour line extraction unit 202 extracts the hair contour line 112 of the subject 111 from the acquired subject image 110. The "contour line" as used herein refers to the outer edge of a silhouette, i.e., the boundary between the hair and the surrounding area, i.e., areas other than the head. To extract the contour line 112, for example, as shown in FIG. 2B( a), an artificial intelligence may be trained on the learning data 230 to generate a trained contour line extraction model, and the generated trained contour line extraction model may be used to extract the hair contour line 112 of the subject 111. In other words, the contour line extraction unit 202 trains an artificial intelligence to learn the relationship between the subject image and the subject's hair contour line 112, generates a trained contour line extraction model, and can extract the hair contour line 112 of the subject 111 using the generated trained contour line extraction model. Artificial intelligence is a computer system equipped with intelligent functions such as inference and judgment similar to those performed by humans, and is a system that performs deep learning and determines correlation equations. The artificial intelligence used in the contour extraction unit 202 preferably uses deep learning.

[0027] Here, the learning data 230 is a set of a sample image 231 obtained by capturing a sample of hair, and a binarized image 232 obtained by binarizing the sample image 231 so that the hair region (hair portion) is emphasized. Then, this learning data 230 is subjected to machine learning by an artificial intelligence to obtain a trained contour extraction model. The sample image 231 and the binarized image 232 can be images of hair captured from various angles, such as a front image, a rear image, a side image, a diagonal front image, and a diagonal rear image.

[0028] As described above, the contour 112 output from the trained contour extraction model is the boundary between the hair region and other parts, and is represented as a line in the two-dimensional image. In generating the trained contour extraction model, the sample image 231 is binarized and used as training data (binarized image 232) so that the boundary can be correctly recognized, i.e., so that the hair region can be distinguished from other parts.

[0029] Then, for example, semantic segmentation is used to perform deep learning on the training data 230, thereby generating a trained contour extraction model. For deep learning, for example, a fully convolutional network (FCN), a convolutional neural network (CNN), a recurrent neural network (RNN), or the like can be used.

[0030] Semantic segmentation is an algorithm used to recognize groups of pixels that form distinctive labels or categories by associating labels or categories with all pixels in an image. This makes it possible to reliably extract the outline (silhouette) of hair even if the training data 230 is, for example, an image taken from behind or in front, or an image with a cluttered background.

[0031] The contour 112 extracted by the contour extraction unit 202 will be further described with reference to FIG. 2C . First, the contour extraction unit 202 extracts a hair region from the subject image 110 to generate image 233. Then, the contour extraction unit 202 extracts pixels constituting the outer edge of the hair region, i.e., contour 112, from image 233 to generate image 234. In this manner, the contour 112 can be represented as an image in which only the pixels constituting the contour 112 are filled in and the other pixels are not. Note that the method of representing the contour 112 is not limited to this method. For example, the pixels constituting the contour 112 may be displayed in the form of coordinates 235, or any other well-known method of representing a line may be used. Note that FIG. 2C is merely for illustrative purposes, and those skilled in the art will readily appreciate that the number of pixels actually used will be much greater.

[0032] Although the method for extracting the hair contour 112 has been described above, the method for extracting the contour 112 is not limited to the above method. For example, an infrared image may be used. Near-infrared light can be used as the infrared light. That is, hair and skin have different reflectances to near-infrared light. Hair's reflectance tends to increase as the wavelength increases in the 850-940 nm range, while skin's reflectance tends to decrease as the wavelength increases within the same wavelength range. By utilizing these characteristics of near-infrared light, near-infrared light in the 850-940 nm wavelength range is irradiated onto the subject 111, and the reflectance at 850 nm is subtracted from the reflectance at 940 nm to extract the area where the value is positive. This area then becomes the hair area. The outer edge of the hair area extracted in this manner can be extracted as the contour 112.

[0033] In addition to the method using the wavelength dependency of near-infrared reflectance, the hair contour 112 may also be extracted using, for example, thermography. That is, the hair contour 112 can be extracted based on the difference in temperature distribution between the face (skin), hair, and surrounding environment of the subject 111. The temperature distribution between the face and surrounding environment is more uniform than that of the hair (hair) region (peaks occur in the temperature zones of the face and surrounding environment), so this can be utilized. The hair contour 112 can be extracted by performing threshold processing according to the hair region zone, extracting a region excluding the face and surrounding environment, and further extracting the outer edge of the extracted region.

[0034] Alternatively, the contour 112 may be extracted using a typical image processing method without using machine learning. For example, an area of ​​the same color as the hair color may be extracted and its outline may be extracted as the contour 112, or the contour 112 may be extracted from the difference between the hair color and the background color.

[0035] Furthermore, the contour line 112 may be extracted by applying a filter process (edge ​​processing) to the subject image 110. Edge processing is a process that emphasizes portions (edges) of an image where brightness changes suddenly, such as from black to white or white to black, and includes processing that emphasizes vertical, horizontal, or both vertical and horizontal directions. For example, hair regions contain a lot of high-frequency components, while skin regions contain a lot of low-frequency components. Furthermore, hair regions have unevenness, which causes light reflection to vary depending on the location, resulting in a lot of high-frequency components, while skin regions have small unevenness, which causes light reflection to be almost constant regardless of the location, resulting in a lot of low-frequency components.

[0036] Furthermore, the contour line 112 may be extracted by combining information input by the subject 111 or other users with an image processing method. For example, some kind of interface may be used to set points and areas necessary for extracting hair color, the hair color may be extracted based on the set points, an area having a hue similar to the extracted hair color may be extracted, and the outer edge of the extracted area may be further extracted to extract the hair contour line 112.

[0037] Alternatively, assuming that the hair color is black or brown, a black or brown area may be extracted from the subject image 110, and the outer edge of the extracted area may be further extracted to extract hair contour 112. Alternatively, a location where the black or brown area changes suddenly to the background color may be searched for and extracted as the hair area, and the outer edge of the extracted hair area may be extracted as contour 112.

[0038] Furthermore, facial feature points such as the eyes and mouth of the subject 111 are extracted using face recognition technology. For example, a pixel located a few pixels above the apex of the forehead is assumed to be a hair pixel. Using this pixel as a reference, if the color of an adjacent pixel is close to that of the reference pixel, it is determined to be a pixel in the hair region. The hair region can then be extracted by repeating the above process until no pixels are determined to be pixels in the hair region. The outer edge of the extracted hair region may then be extracted as the contour line 112.

[0039] Furthermore, in addition to the method using the trained model described above, the contour line 112 may also be extracted by the subject 111 or another user tracing the hair contour line 112 using a finger or an appropriate input device on the subject image 110 displayed on a display device such as a touch panel display.

[0040] The hair property estimation unit 203 estimates the hair property of the subject 111 based on the extracted contour line 112. More specifically, the hair property estimation unit 203 estimates the hair property of the subject 111 using, for example, a learned hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line 112 and the property. In this way, a combination of the extracted contour line 112 and the property estimated from the contour line 112 is provided as learning data to the artificial intelligence for learning. The learning data is assigned a hair property type (first type 121 to fourth type 124), and the hair property type is assigned by a hair expert such as a hairdresser or barber. Note that in this embodiment, hair properties are categorized by the degree of hair curl as the hair property, but hair properties are not limited to this. Hair properties may be, for example, shape (finished hairstyle and how messy it is, bedhead, how wavy the perm is (length), etc.), ease of management, volume (actual hair volume), hair quality (hardness, smoothness, damage, etc.), appearance, i.e., how it appears to a third party (curly hair, volume, messiness, appearance of the ends, etc.), etc.

[0041] In addition to the degree of curliness described above, the following properties (1) to (8) may be estimated as hair properties. (1) As hair properties, frizziness may be estimated. Here, frizziness is different from the "degree of curliness" or "strength of curliness" determined from the state of bare hair after washing, and refers to the "curly appearance" of a style in a situation where the hair is not in its natural state, such as during the day after styling. Silhouettes that widen around the temples (from the top of the head to the ears) and around the face (from the ears to the chin) (silhouettes like the first type 121 and the second type 122) indicate a strong frizzy appearance. By achieving a silhouette that does not widen (the fourth type 124), the frizzy appearance gradually decreases. A user for whom frizziness is estimated is, for example, a Japanese woman between the ages of 10 and 70 (including those with hair straightening and perm treatments). The image used as the subject image 110 is preferably an image of a scene where the subject's hair is not in a natural state, such as during the day after styling, but any image that shows the subject's 111 hair condition may be used. The classification and evaluation of frizziness can be performed on a four-level scale: 1 = very frizzy, 2 = slightly frizzy, 3 = slightly frizzy, and 4 = slightly frizzy. A four-level frizziness score is assigned to the data for machine learning in advance, and the score is assigned by a hair expert such as a hairdresser or barber. An example of a hair care plan proposal, which will be described later, is, for example, a suggestion of when to re-do a hair straightening treatment for someone who is undergoing hair straightening.

[0042] (2) Untidyness may be estimated as a hair characteristic. Here, untidyness refers to the "untidy appearance" of a style in a situation where hair is not in its natural state, such as during the day after styling. For example, a silhouette with a wide head and face (such as the first type 121 or the second type 122) has a strong untidy appearance. A silhouette that does not widen (such as the fourth type 124) gradually reduces the untidy appearance. The target users for untidyness estimation are Japanese women aged 10 to 70 (including those with hair straightening or perm treatment). The image used as the subject image 110 is preferably one taken during the day after styling, when hair is not in its natural state. However, any image that shows the hair characteristics of the subject 111 may be used. Untidyness can be classified and evaluated on a four-level scale: 1 = very untidy, 2 = slightly untidy, 3 = slightly untidy, and 4 = slightly untidy. The data used for machine learning is assigned a four-level score in advance, and the score is assigned by hair experts such as hairdressers and barbers. An example of a hair care plan, which will be described later, is a recommendation to get a haircut at a beauty salon if your hair starts to look untidy.

[0043] (3) The degree of wave perm (length) may be estimated as hair characteristics. The degree of wave perm refers to the degree to which the wave shape of hair remains after a certain period of time has passed since a wave perm was performed at a beauty salon. As time passes after a perm, the hair grows unevenly, causing the phase of the curls to shift, resulting in a silhouette that expands, for example, as in Type 2 122 or Type 3 123. Furthermore, as the perm wears off, the curls straighten out, approaching the silhouette of Type 4 124 (a silhouette that does not expand). The users for whom the degree of wave perm is estimated are Japanese women aged 10 to 70 who have undergone wave / curl perm treatment. The image used as the subject image 110 is preferably one that shows the condition of the natural hair. For example, an image taken one hour after towel drying after washing or after natural drying after washing is preferable. However, any image that shows the hair characteristics of the subject 111 may be used. The strength of a wave perm can be classified and evaluated on a four-level scale: 1 = strong, 2 = slightly strong, 3 = slightly weak, and 4 = weak. Data for machine learning is assigned a four-level score in advance, and the scores are assigned by hair experts such as hairdressers and barbers. An example of a suggested hair care plan, which will be described later, is a suggestion for when to reapply a perm.

[0044] (4) The hair quality may also be estimated as a styling result. The styling result refers to the state of the hair immediately after styling, such as before going out in the morning. For example, when styling straight hair, a good result results in a flat silhouette, similar to the fourth type 124. A poor result results in a silhouette with a wide temples and face, similar to the first to third types 121 to 123. The users whose styling results are to be estimated are Japanese women aged 10 to 70. The image used as the subject image 110 is preferably an image taken immediately after styling. The styling result can be classified and evaluated on a four-level scale: 1 = good, 2 = somewhat good, 3 = somewhat poor, and 4 = poor. The data for machine learning is assigned a four-level score in advance, and the scores are assigned by hair experts such as hairdressers and barbers.

[0045] (5) The degree of disruption from styling may be estimated as a hair property. The degree of disruption from styling refers to the degree of change in the style from the hairstyle set before going out in the morning, etc., over time as the hairstyle progresses through daily activities. For example, if there is little change from the finished silhouette described in (4) above, the style is considered to be intact. If there is a significant change, approaching a silhouette with a wider temples or face, such as types 121 to 123, the style is considered to be disrupted. The users targeted for estimating the degree of disruption from styling are Japanese women aged 10 to 70. The image used as the subject image 110 is preferably an image taken more than eight hours after styling and engaging in daily activities, but any time after styling is acceptable. The degree of disruption from styling can be classified and evaluated using four levels: 1 = not disrupted, 2 = slightly disrupted, 3 = slightly disrupted, and 4 = disrupted. Data for machine learning is assigned a four-level score in advance, and the scores are assigned by hair experts such as hairdressers and barbers.

[0046] (6) The appearance of hair tips may be estimated as a hair characteristic. The appearance of hair tips refers to the appearance of the shape of the hair tips in a silhouette. A poor appearance is indicated when the hair tips of the silhouette are splayed or uneven on both sides, whereas a good appearance is indicated when the hair tips of the silhouette are uniform and do not splay. The target users for estimating the appearance of hair tips are Japanese women between the ages of 10 and 70 (including those who have undergone hair straightening or perming). The image used as the subject image 110 is preferably an image after styling, but any image in a state that allows the subject's 111 hair characteristics to be seen may be used. When estimating the appearance of hair tips, the subject image 110 is used to include the subject's 111 hair from the top of the head to the hair tips. The appearance of hair tips is classified and evaluated on a four-level scale: 1 = good, 2 = slightly good, 3 = slightly bad, and 4 = bad. Data for machine learning is assigned a four-level score in advance, and the scores are assigned by hair experts such as hairdressers and barbers.

[0047] (7) Volume (actual hair volume) may be estimated as a hair characteristic. Volume refers to the actual amount of hair. Even for the same volume (hair volume), the perceived volume varies depending on styling and the original shape of the hair. For curly hair, when straightened with a straightening iron or similar, a larger bulge in the silhouette of the crown indicates a larger amount of hair volume, while a smaller bulge indicates a smaller amount of hair volume. The target users for volume estimation are Japanese women aged 10 to 70 (including those with hair straightening and perm treatments). For straight hair, an image showing the natural state of the hair is preferable, while for curly hair, an image straightened with a straightening iron or similar is preferable. Volume is classified and evaluated on a four-level scale: 1 = high, 2 = slightly high, 3 = slightly low, and 4 = low. Alternatively, instead of sensory evaluation, the actual hair density on the crown may be measured and classified and evaluated. Data for machine learning is assigned a four-level score in advance, and the scores are assigned by hair experts such as hairdressers and barbers.

[0048] (8) Volume may be estimated as a hair characteristic. Volume does not refer to the actual amount of hair, but rather to the "voluminous appearance" of a style in a situation where the hair is not in its natural state, such as during the day after styling. A larger bulge in the silhouette of the top of the head indicates a larger volume, while a smaller bulge indicates a smaller volume. The target users for volume estimation are Japanese women aged 10 to 70 (including those with straightened or permed hair). The image used as the subject image 110 is preferably an image after styling, but any image in a state that shows the hair characteristics of the subject 111 may be used. Volume is classified and evaluated on a four-level scale: 1 = present, 2 = somewhat present, 3 = somewhat absent, and 4 = absent. Data for machine learning is assigned a four-level score in advance, and the scores are assigned by hair experts such as hairdressers and barbers.

[0049] The hair property type assigned to the data for machine learning, i.e., the training data, may be other than the hair property type assigned by a hair expert as described above. For example, in addition to the hair property (hair curl) classification by a hair expert as described above, hair property classification may be performed using the following method. From the viewpoint of ease of acquisition and data accuracy, it is preferable to measure preferably 1 to 1,000 hairs, more preferably 3 to 500 hairs, and even more preferably 5 to 250 hairs for each user, and use the average value as the degree of curl for that user. Note that when measuring the hair, it is not necessary to pluck or cut the hair from the user. Specific methods for measuring the degree of curl can include, for example, the following methods (I) to (VII).

[0050] (I) An image of the hair is scanned with an image scanner, and the radius of curvature of measurement points at predetermined intervals on each scanned image of the hair is calculated using image processing software, and the average value of these values ​​is used as the degree of curl. In this case, an upper limit may be set for the measured radius of curvature; for example, a radius of curvature of 20 cm or more can be set as 20 cm. In addition, it is preferable that the number of measurement points per hair is three or more.

[0051] (II) A transparent film with concentric circles (radius 1 mm to 200 mm) with different radii (for example, 1 mm) is placed on the hair, and the radius of curvature of the hair is read at multiple locations (for example, 4 locations at 1 cm intervals) at predetermined intervals from the root of the hair, and the reciprocal (curvature) is calculated. However, if the curvature is 0.05 (unit: cm －１ ) (when the radius of curvature is greater than 20 cm), the curvature can be approximated as 0.05. The average value of the curvatures at the multiple locations is calculated, and the reciprocal of this average value is used as the radius of curvature of the hair.

[0052] (III) An image of the hair is read in with an image scanner, and multiple reference curves with known average radii of curvature are stored in image processing software. Each read-in hair image is compared with the reference curve, and the average radius of curvature of the reference curve that best matches the two is determined as the degree of curl.

[0053] (IV) One end of the hair is held and hung, and its shape is read using a 3D scanner. The radius of curvature of measurement points at predetermined intervals on the read hair image is calculated using image processing software, and the average value is used as the degree of curl.

[0054] (V) The natural length (LO) of the hair and the length (L) of the hair stretched without stretching the hair itself and straightening out any undulations are measured, and (L - LO) / L is calculated, which is the degree of curl. For example, to measure the length, first fix one end of the hair and hang the other end free, and measure the distance between the fixed end and the other end (natural length LO). Next, the hair is stretched straight by hand without stretching the hair itself and the lengths (L) of both ends are measured, or a weight (e.g., 0.1 g to 0.01 g) with a weight that can be ignored as stress is hung from one end and the length (L) from the fixed end to the weight is measured. When measuring LO and L, it does not matter which end the root of the hair is at.

[0055] (VI) One end of the hair is held and hung, and its shape is photographed with a digital camera. The extent to which the hair spreads to the left and right of a straight line connecting one end of the hair to the other is measured at measurement points at specified intervals on the straight line, and the average of these measurements is taken as the degree of curl.

[0056] (VII) The hair is placed on a smooth metal plate (to prevent static electricity) and the spread, i.e., the angle from one end, of the triangle containing the hair 250 is measured, as shown in Figure 2B(b), and the average value of these angles is taken as the degree of curl. The angle can be extracted by image processing.

[0057] The hair condition estimation unit 203 preferably estimates the hair condition using the feature quantities of the extracted contour line 112. Specifically, to improve the accuracy of hair condition estimation, it is preferable to provide a combination of the feature quantities of the contour line 112 and the hair condition as learning data to an artificial intelligence for learning, as shown in FIG. 2D . FIG. 2D illustrates an example in which the angle of the contour line extracted from each predetermined region is used as the feature quantity. The angle of the contour line extracted from each region can be, for example, the average value of the angle between the contour line 112 and a line parallel to the facial midline in each region, such as a line extending straight down from the top of the head toward the chin. For example, when a selfie image is used as the subject image 110, the subject image 110 may be an image in which the facial midline of the subject 111 is tilted relative to the vertical direction of the subject image 110. Even in such a case, the angle of the contour line 112 can be derived with high accuracy by setting the average value of the angles between the contour line 112 and a line drawn straight down from the top of the head toward the tip of the chin as the angle of the contour line 112. The feature amount is not limited to the angle of the contour line extracted from each predetermined region, and for example, the length of the contour line extracted from each predetermined region can also be used.

[0058] The feature extraction may be performed by the contour line extraction unit 202 or the hair property estimation unit 203, but in this description, it is assumed that the feature extraction is performed by the contour line extraction unit 202. Therefore, the hair property estimation unit 203 obtains data related to the extracted feature amounts from the contour line extraction unit 202. Note that the hair property estimation device 100 may further include a contour feature amount extraction unit that extracts feature amounts.

[0059] For example, the contour line extraction unit 202 generates a contour line image 240, then divides it into several regions (S1 to S6), extracts features for each region, and uses the relationship between each extracted feature and the assigned hair property type as learning data for the artificial intelligence to learn.

[0060] Here, as shown in FIG. 2D , the regions are divided as follows: S1 = region from the parietal line to the brow line; S2 = region from the brow line to the eye line; S3 = region from the eye line to the upper lip line; S4 = region from the upper lip line to the chin line; S5 = region from the brow line to the upper lip line; and S6 = region from the brow line to the chin line. In this way, the subject image 110 can be divided into multiple regions based on facial features of the subject 111, such as the eyes, upper lip, eyebrows, and chin. In addition to the criteria shown here, the nose may also be included as one of the criteria. Note that the "chin" here refers to the lowest point of the chin, i.e., the tip of the chin, as can be seen in FIG. 2D . Furthermore, the "line" refers to a horizontal line, i.e., a line drawn parallel to the line connecting the pupils of both eyes or the tops of both eyebrows. In other words, the upper lip line refers to a horizontal line passing through the upper lip, and the chin line refers to a horizontal line passing through the bottom of the chin.

[0061] When estimating hair properties, it is preferable to use at least the contour of the head, that is, the area between the top of the head and the bottom of the chin, more specifically, the area from the top of the head to the tip of the chin. That is, a preferable subject image is one that includes the area from the top of the head to at least the tip of the chin. That is, it is preferable to obtain a subject image that includes the area from the top of the head to at least the tip of the chin, extract the hair contour, and use that contour for estimation.

[0062] The feature amount of the hair contour line 112 may be, for example, at least one of the angle and the length of the contour line 112 .

[0063] The contour line extraction unit 202, for example, calculates the angle of the hair contour line 112 (outer edge of the silhouette) in each region, and sets the calculated angle as the feature amount of the hair contour line 112. Note that here, the region for extracting feature amounts is divided into six, but the method of dividing the regions is not limited to the method shown here, and the regions may be divided more finely.

[0064] Next, calculation of the angle of each region will be described in detail with reference to Figure 2E. As shown, the contour image 240 is divided into left and right halves, and for each region, the angles of the left contour 112 and the right contour 112 are plotted, with the distance [pixels (px)] from the edge of the image on the vertical axis and the height [px] on the horizontal axis. Note that left and right here refer to the left and right sides of the subject 111, so in the contour image 240 shown in the figure, the left side corresponds to the right side of the subject 111, and the right side corresponds to the left side of the subject 111. Plot 241 represents the right contour 112, and plot 242 represents the left contour 112.

[0065] Then, by fitting the contour line 112 with a linear function (graph 243) and converting the slope of the linear function into an angle, the angle = arctan (slope), where the stronger the degree of hair curl, the larger the angle, and the weaker the degree of hair curl, the smaller the angle. For the section of region S1, the contour line 112 may be fitted with a quadratic function, and the quadratic coefficient may be used as the feature. Alternatively, for each region, the angular difference between the left and right silhouettes may be used as the feature. Here, the angular difference between the left and right silhouettes is the absolute value of the difference between the angle of the right contour line 112 and the angle of the left contour line 112. As described above, the angle of the contour line 112 can be the angle of the contour line 112 with respect to a line drawn straight down from the top of the head toward the tip of the chin.

[0066] The contour extraction unit 202 may also calculate the length of the hair contour 112 in each region and use the calculated length as a feature of the hair contour 112. For example, after resizing the image so that the face size is consistent, the number of pixels of the contour 112 (outer edge of the silhouette) in each region may be counted, and the counted number of pixels may be divided by the height of each region (each section) to determine the feature. In this case, the stronger the curl, the more curved the contour 112 (outer edge of the silhouette) and the greater the number of pixels. The weaker the curl, the closer the silhouette becomes to verticality and the fewer pixels. Any method may be used to resize the image so that the face size is consistent. For example, the positions of the forehead and chin may be detected and their lengths may be matched. As can be easily understood from the above, the feature may be the left and right angles, the average absolute value of the left and right angles, the difference between the left and right angles, or the tilt.

[0067] Alternatively, the contour 112 may be extracted after normalizing the image size so that the face size is constant. For example, a method of normalizing the subject image 110 may be used, for example, based on the length from the eyebrow line to the chin of the subject 111 in the subject image 110. The contour 112 extracted from the normalized subject image 110 may then be divided into multiple regions, as described above, and the number of pixels of the contour 112 in each region may be counted. The number of pixels counted in this manner may be used as the feature value. This is because the number of pixels counted in this manner is the length of the normalized contour 112 in each region.

[0068] Furthermore, as another feature, for example, the number of flyaway hairs that are not included in the outline may be detected and used. The flyaway hair detection method may use a typical image processing method or deep learning. In this case, for example, it is possible to evaluate that the more flyaway hairs there are, the stronger the degree of curliness of the hair.

[0069] The above-mentioned "within the outline" refers to the area surrounded by the outline 112, and not the outline 112 itself. In other words, a flyaway hair that is not included in the outline refers to a flyaway hair that protrudes from the area surrounded by the outline 112.

[0070] The hair property estimation unit 203 provides the feature amount of the extracted contour line 112, for example, the relationship between the angle and the hair property, to an artificial intelligence for machine learning, thereby generating a trained hair property estimation model. Machine learning by the artificial intelligence can use, for example, a support vector machine (SVM), a k-nearest neighbor method, a random forest, etc., but is not limited to these. In other words, any known method, such as deep learning or regression analysis, can be used.

[0071] By using a trained hair property estimation model that has learned the relationship between the feature amount of the contour line 112 and the hair property, it is possible to estimate the hair property with a small amount of calculation.

[0072] Furthermore, it is also possible to estimate the hair property directly from the contour line image 240 without extracting features. That is, it is also possible to estimate the hair property from the contour line image by using a trained hair property estimation model obtained by performing machine learning on an artificial intelligence using a combination of the contour line image 240 and the hair property as training data.

[0073] The trained hair property estimation model that has learned the relationship between the contour line image 240 and hair properties is preferably generated by deep learning. By using the trained hair property estimation model that has learned the relationship between the contour line image 240 and hair properties, the amount of calculation can be reduced compared to when estimating hair properties from the entire hair image.

[0074] Here, a support vector machine is a pattern recognition model that uses supervised learning, and is an algorithm that performs class classification and regression by determining, for example, a boundary line or hyperplane that divides data groups into two classes. The k-nearest neighbor method is an algorithm used for data classification, and when there is unknown data, it determines the classification of the unknown data from the classes of the surrounding training data. The k in the k-nearest neighbor method indicates the number of classes of training data that exist near the unknown data. The random forest is an algorithm that combines two techniques, decision trees and ensemble learning (bagging), and is an algorithm that performs class classification and regression.

[0075] A method for generating a trained hair property estimation model will be described with reference to FIGS. 2F to 2H, taking as an example a case where a support vector machine is used.

[0076] First, the features are converted into principal component scores using principal component analysis (PCA). Specifically, the axis PC1 with the largest variance is calculated. Then, the data is projected from the feature space 270 to the principal component space 271. Note that "PC2" in FIG. 2F is the axis with the second largest variance.

[0077] Next, a hyperplane 272 for classifying data in the principal component space 271 is derived using a support vector machine. Specifically, equation (1) representing the hyperplane 272 is calculated. The weight w and bias b in equation (1) can be calculated by solving equation (2). In equation (2), the slack variable is a variable that penalizes misclassified data, and the regularization parameter is a parameter whose larger value means that it tolerates less misclassification and whose smaller value means that it tolerates more misclassification. When multiple types can be classified, hyperplanes capable of classifying each type are calculated. By deriving hyperplanes capable of classifying data in this way, a learned hair property estimation model is generated.

[0078] Then, by using the generated trained hair property estimation model, it is possible to estimate the hair property of the subject 111. In addition to the method of estimating the hair property using the trained model, the hair property may also be estimated using a correlation, as shown in FIG.

[0079] For example, as shown in Figure 2G, when the hair contour 112 is divided into multiple regions and the angle of each region is calculated, it is found that a correlation is observed for some regions. Note that the vertical axis represents the coefficient of a quadratic function for region S1, and the angles for regions S2 to S6. The horizontal axis represents the degree of curl (type) for each of regions S1 to S6, and is expressed as first type 121 = 1, second type 122 = 2, third type 123 = 3, and fourth type 124 = 4. Then, the R of each region is ２ Looking at the coefficient of determination, in the areas S3, S5, and S6 close to the center of the face, R ２ is close to 1.0, and it can be seen that there is a high correlation between the angle and the strength of the curl. On the other hand, in areas S1, S2, and S4 far from the center of the face, R ２ is close to 0, indicating a low correlation. Therefore, the hair properties of the subject 111 may be estimated using such a correlation.

[0080] <First Verification> Here, the accuracy of the hair property estimation by the above-mentioned machine learning was verified, for example, as follows. Since the verification method uses a small amount of data, the estimation accuracy was verified by leave-one-out cross-validation. In this verification method, one piece of data is used as test data, and the remaining data is divided into learning data to verify whether the test data can be estimated. This is repeated until all data is test data. The number of images used in the verification was 11 images of the first type 121, 10 images of the second type 122, 11 images of the third type 123, and 11 images of the fourth type 124, and the degree of curl was estimated. Then, when the silhouette angle of the images used in the verification was used as a feature value to perform estimation using a support vector machine (SVM), the accuracy of the estimation of the degree of curl was 72% (31 / 43).

[0081] <Second Verification> In order to further improve the accuracy of estimating the degree of curl, a second verification was conducted by increasing the number of test data and the types of features used in the verification. In the second verification, the number of images was 30 for the first type 121, 35 for the second type 122, 39 for the third type 123, and 28 for the fourth type 124. Both the angle and the length of the contour line 112 were used as feature quantities. Specifically, eight types of angles were used: the angle of each of regions S1 to S6, the angular difference between the angle of region S2 and the angle of region S3, and the angular difference between the angle of region S3 and the angle of region S4. Eight types of lengths were used: the length of each of regions S1 to S6, the difference between the length of region S2 and the length of region S3, and the difference between the length of region S3 and the length of region S4.

[0082] Figure 2H shows the correlation between angle and degree of curl in the data used in the second verification. The vertical axis represents the coefficient of a quadratic function for region S1, and the vertical axis represents the angle for regions S2 to S6. The vertical axis for "S2-S3" represents the difference in angle between region S2 and region S3, while the vertical axis for "S3-S4" represents the difference in angle between region S3 and region S4. The horizontal axis represents the degree (type) of curl in regions S1 to S6, "S2-S3," and "S3-S4," with type 1 (121) = 1, type 2 (122) = 2, type 3 (123) = 3, and type 4 (124) = 4. Looking at the R2 (coefficient of determination) for each region, regions S3 and S5 have relatively large R2 values, indicating a high correlation between angle and strength of curl. In contrast, in the areas S1, S2, S4, S6, "S2-S3" and "S3-S4", R2 is relatively small, indicating a low correlation. Thus, even in the data used in the second verification, it can be seen that there is a correlation between the angle and the degree of curl.

[0083] Figure 2I shows the correlation between length and curl intensity for the data used in the second verification. The vertical axis for regions S1 to S6 represents the length of the contour line 112. The vertical axis for "S2-S3" represents the difference in length of the contour line 112 between regions S2 and S3, while the vertical axis for "S3-S4" represents the difference in length of the contour line 112 between regions S3 and S4. The horizontal axis for regions S1 to S6, "S2-S3," and "S3-S4" represents the degree (type) of curl, with type 1 121 = 1, type 2 122 = 2, type 3 123 = 3, and type 4 124 = 4. Looking at the R2 (coefficient of determination) for each region, regions S3, S5, and S6 have relatively large R2 values, indicating a high correlation between the angle and the strength of curl. In contrast, in the regions S1, S2, S4, "S2-S3" and "S3-S4", R2 is relatively small, indicating a low correlation. This shows that there is a correlation between length and degree of curl.

[0084] The degree of curl was estimated for all 20 subject images 110 using a support vector machine trained on the data used in the second verification. The 20 subject images 110 consisted of five first type 121, five second type 122, six third type 123, and four fourth type 124. Features, including both angle and length—specifically, the eight angles and eight lengths mentioned above—were extracted for a total of 16 features. The 16 features were then converted into seven principal component scores, and the seven principal component scores were used to estimate the degree of curl using a support vector machine. The results are shown in Figure 2J. The vertical axis represents the actual degree (type) of curl, and the horizontal axis represents the degree (type) of curl estimated by the support vector machine. Both the vertical and horizontal axes are represented as follows: first type 121 = 1, second type 122 = 2, third type 123 = 3, and fourth type 124 = 4.

[0085] As shown in FIG. 2J , the estimated type matched the actual type for 16 of the 20 subject images 110, resulting in an estimation accuracy of 80%. Specifically, for the first type 121 images, four of the five were estimated to be the first type 121 and one was estimated to be the second type 122. For the second type 122 images, all five were estimated to be the second type 122. For the third type 123 images, three of the six were estimated to be the third type 123 and three were estimated to be the fourth type 124. For the fourth type 124 images, all four were estimated to be the fourth type 124.

[0086] To verify whether angle or length is a more useful feature in estimating the degree of curl using a support vector machine, we conducted a verification test using the eight types of angle and the eight types of length as feature quantities. The results are shown in Figure 2K.

[0087] When only angle was used as a feature, as shown in graph 273 of FIG. 2K , the estimated type matched the actual type for 12 of the 20 subject images 110, resulting in an estimation accuracy of 60%. Specifically, for the first type 121 images, three of the five were estimated to be the first type 121, one was estimated to be the second type 122, and one was estimated to be the third type 123. For the second type 122 images, four of the five were estimated to be the second type 122 and one was estimated to be the third type 123. For the third type 123 images, three of the six were estimated to be the third type 123 and three were estimated to be the fourth type 124. For the fourth type 124 images, two of the four were estimated to be the third type 123 and two were estimated to be the fourth type 124.

[0088] When only length was used as a feature, as shown in graph 274 of FIG. 2K, the estimated type matched the actual type for 15 of the 20 subject images 110, resulting in an estimation accuracy of 75%. Specifically, for the images of type 121, all five were estimated to be type 1 121. For the images of type 2 122, four of the five were estimated to be type 2 122 and one was estimated to be type 3 123. For the images of type 3 123, two of the six were estimated to be type 3 123 and four were estimated to be type 4 124. For the images of type 4 124, all four were estimated to be type 4 124.

[0089] In this way, when only length is used as a feature, the estimation accuracy is higher than when only angle is used as a feature. The reason for this is thought to be as follows: The degree of curl is affected by both the width and wavy of the hair silhouette. The angle increases when the width of the silhouette increases. On the other hand, the length increases when both the width and wavy of the silhouette increase. Therefore, it is thought that length, which is affected by both the width and wavy of the silhouette, is more useful for estimating curl than angle, which is affected only by the width.

[0090] 2J and 2K, the estimation accuracy is higher when both the angle and the contour line are used than when only the angle or only the length is used. It is believed that the combination of the length, which involves both the silhouette's width and undulation, and the angle, which involves only the width, is more useful for estimating curl.

[0091] Furthermore, to verify which regions of the subject image 110 have useful feature values ​​for estimating the degree of habit using a support vector machine, verification was performed using only the feature values ​​of each region. Specifically, habit estimation was performed using feature values ​​from regions S1 to S6, the region around the ears (regions S2, S3, S5), the region below the eyebrows (regions S2, S3, S4, S5, S6), and the region above the upper lip (regions S1, S2, S3, S5). The results are shown in Figures 2L to 2M.

[0092] When the feature values ​​of region S1 were used, as shown in graph 281 of FIG. 2L , the estimated type matched the actual type for four of the 20 subject images 110, resulting in an estimation accuracy of 20%. Specifically, for the first type 121 images, two of the five were estimated to be the first type 121, one was estimated to be the second type 122, and two were estimated to be the third type 123. For the second type 122 images, three of the five were estimated to be the first type 121 and two were estimated to be the third type 123. For the third type 123 images, one of the six was estimated to be the first type 121, two were estimated to be the third type 123, and three were estimated to be the fourth type 124. For the fourth type 124 images, one of the four was estimated to be the first type 121 and three were estimated to be the third type 123.

[0093] When the feature values ​​of region S2 were used, as shown in graph 282 of FIG. 2L , the estimated type matched the actual type for 10 of the 20 subject images 110, resulting in an estimation accuracy of 50%. Specifically, for the first type 121 images, two of the five were estimated to be the first type 121, two were estimated to be the second type 122, and one was estimated to be the fourth type 124. For the second type 122 images, four of the five were estimated to be the second type 122 and one was estimated to be the fourth type 124. For the third type 123 images, one of the six was estimated to be the second type 122 and five were estimated to be the fourth type 124. For the fourth type 124 images, all four were estimated to be the fourth type 124.

[0094] When the feature values ​​of region S3 were used, as shown in graph 283 of FIG. 2L , the estimated type matched the actual type for 12 of the 20 subject images 110, resulting in an estimation accuracy of 60%. Specifically, for the first type 121 images, four of the five were estimated to be the first type 121 and one was estimated to be the second type 122. For the second type 122 images, three of the five were estimated to be the second type 122 and two were estimated to be the third type 123. For the third type 123 images, one of the six was estimated to be the second type 122, one was estimated to be the third type 123, and four were estimated to be the fourth type 124. For the fourth type 124 images, all four were estimated to be the fourth type 124.

[0095] When the feature values ​​of region S4 were used, as shown in graph 284 of FIG. 2M , the estimated type matched the actual type for 11 of the 20 subject images 110, resulting in an estimation accuracy of 55%. Specifically, for the first type 121 images, three of the five were estimated to be the first type 121, one was estimated to be the second type 122, and one was estimated to be the fourth type 124. For the second type 122 images, one of the five was estimated to be the second type 122 and four were estimated to be the third type 123. For the third type 123 images, three of the six were estimated to be the third type 123 and three were estimated to be the fourth type 124. For the fourth type 124 images, all four were estimated to be the fourth type 124.

[0096] When the feature values ​​of region S5 were used, as shown in graph 285 of FIG. 2M , the estimated type matched the actual type for 13 of the 20 subject images 110, resulting in an estimation accuracy of 65%. Specifically, for the first type 121 images, four of the five were estimated to be the first type 121 and one was estimated to be the third type 123. For the second type 122 images, four of the five were estimated to be the second type 122 and one was estimated to be the third type 123. For the third type 123 images, one of the six were estimated to be the second type 122, one was estimated to be the third type 123, and four were estimated to be the fourth type 124. For the fourth type 124 images, all four were estimated to be the fourth type 124.

[0097] When the feature values ​​for region S6 were used, as shown in graph 286 in FIG. 2M , the estimated type matched the actual type for 15 of the 20 subject images 110, resulting in an estimation accuracy of 75%. Specifically, for the images of type 121, all five were estimated to be type 121. For the images of type 222, one of the five was estimated to be type 2 122, and four were estimated to be type 2 122. For the images of type 3 123, two of the six were estimated to be type 3 123, and four were estimated to be type 4 124. For the images of type 4 124, all four were estimated to be type 4 124.

[0098] When the feature quantities of the areas around the ears (areas S2, S3, and S5) were used, as shown in graph 287 of FIG. 2N , the estimated type matched the actual type for 12 of the 20 subject images 110, resulting in an estimation accuracy of 60%. Specifically, for the images of type 121, four of the five were estimated to be type 1 121 and one was estimated to be type 2 122. For the images of type 2 122, three of the five were estimated to be type 2 122 and two were estimated to be type 3 123. For the images of type 3 123, one of the six was estimated to be type 2 122, one was estimated to be type 3 123, and four were estimated to be type 4 124. For the images of type 4 124, all four were estimated to be type 4 124.

[0099] When the feature quantities of the areas below the eyebrows (areas S2, S3, S4, S5, and S6) were used, as shown in graph 288 of FIG. 2N , the estimated type matched the actual type for 15 of the 20 subject images 110, resulting in an estimation accuracy of 75%. Specifically, for the first type 121 images, four of the five were estimated to be the first type 121 and one was estimated to be the second type 122. For the second type 122 images, all five were estimated to be the second type 122. For the third type 123 images, two of the six were estimated to be the third type 123 and four were estimated to be the fourth type 124. For the fourth type 124 images, all four were estimated to be the fourth type 124.

[0100] When using the feature values ​​of the area above the upper lip (areas S1, S2, S3, and S5), as shown in graph 289 of FIG. 2N , the estimated type matched the actual type for 14 of the 20 subject images 110, resulting in an estimation accuracy of 70%. Specifically, for the first type 121 images, three of the five were estimated to be the first type 121 and two were estimated to be the second type 122. For the second type 122 images, all five were estimated to be the second type 122. For the third type 123 images, two of the six were estimated to be the third type 123 and four were estimated to be the fourth type 124. For the fourth type 124 images, all four were estimated to be the fourth type 124.

[0101] The estimation accuracy shown in Figure 2J and Figures 2L to 2N is summarized in Figure 2O. As shown in Table 291 in Figure 2O, the estimation accuracy is particularly high when using the feature quantities of all regions (S1 to S2), the feature quantities of the regions below the eyebrows (S2 to S6), and the feature quantities of region S6 alone. Therefore, region S6, which is included in all of these, is considered important for improving estimation accuracy.

[0102] 2O, when using the feature amount of only one region (when using the feature amounts of each of regions S1, S2, S3, and S4), the estimation accuracy is higher when using the feature amount of region S3 and when using the feature amount of region S4. The reason for this is thought to be that regions S3 and S4 are regions where volume begins to appear due to the overlap of hair curls, and are points that hair experts such as hairdressers and barbers focus on when estimating the degree of curl.

[0103] The hair condition estimating device 100 may further include a normalization unit. The normalization unit normalizes the subject image 110 based on the length from the eyebrows to the chin of the subject 111 in the subject image 110. Normalizing the subject image 110 makes it possible to compare multiple subject images 110 with each other. In this embodiment, the size of the subject image 110 is normalized by making the lengths from the eyebrows to the chin of the subject 111 in the subject image 110 equal.

[0104] The method by which the normalization unit normalizes the subject image 110 will be described with reference to FIG. 2P . First, the size of the margins in the subject image 110 is adjusted to generate a margin-adjusted image 260. The margin-adjusted image 260 is an image whose vertical and horizontal lengths are, for example, four times the length L from the eyebrows to the chin of the subject 111 in the subject image 110. If the subject image 110 has little margin, the margin-adjusted image 260 is generated by adding margin to the subject image 110 and lengthening the vertical and horizontal lengths, as shown in FIG. 2P . On the other hand, if the subject image 110 has a lot of margin, the margins of the subject image 110 are reduced by trimming or the like, and the vertical and horizontal lengths are shortened to generate the margin-adjusted image 260. Note that the vertical and horizontal lengths of the margin-adjusted image 260 may be one to three times the length L, or may be five or more times the length L.

[0105] Next, the margin-adjusted image 260 is resized to a predetermined size to generate a normalized image 261. The predetermined size is, for example, a vertical length Dx of 1024 pixels and a horizontal length Dy of 1024 pixels.

[0106] Next, an example of the hair property type table 301 of the hair property estimating device 100 will be described with reference to Fig. 3. The hair property type table 301 stores hair contour shapes 312 in association with hair property types 311. The hair property types 311 are four types determined according to the degree of curliness of the hair, and are classified into four types, first to fourth, in order of decreasing strength of curliness. The hair contour shapes 312 indicate the characteristics of the shape of the hair contour, and shape characteristics are stored for each hair property type. The hair property estimating unit 203 then estimates the hair property type by, for example, referring to the hair property type table 301.

[0107] The hardware configuration of the hair property estimating device 100 will be described with reference to FIG. 4. The CPU (Central Processing Unit) 410 is a processor for arithmetic and control, and executes programs to realize the various functional components of the hair property estimating device 100 shown in FIG. 2. The CPU 410 may have multiple processors and execute different programs, modules, tasks, threads, etc. in parallel. The ROM (Read Only Memory) 420 stores fixed data such as initial data and programs, as well as other programs. The network interface 430 communicates with other devices via a network. The CPU 410 is not limited to one, and may be multiple CPUs or may include a GPU (Graphics Processing Unit) for image processing. Furthermore, it is preferable that the network interface 430 has a CPU independent of the CPU 410 and writes or reads transmitted or received data to or from an area of ​​the RAM (Random Access Memory) 440. It is also preferable to provide a DMAC (Direct Memory Access Controller) (not shown) that transfers data between the RAM 440 and the storage 450. Furthermore, the CPU 410 recognizes that data has been received or transferred to the RAM 440 and processes the data. The CPU 410 also prepares the processing results in the RAM 440 and leaves subsequent transmission or transfer to the network interface 430 or the DMAC.

[0108] The RAM 440 is a random access memory used by the CPU 410 as a temporary storage work area. The RAM 440 has a storage area reserved for storing data necessary for implementing this embodiment. The subject image data 441 is data relating to an image of the subject 111. The contour shape data 442 is data relating to the contour of the subject 111's hair extracted from the subject image 110. The feature amount data 443 is data relating to the feature amount of the extracted hair contour. The hair property type data 444 is data relating to the property of the hair, for example, data relating to a classification according to the degree of curl.

[0109] The transmitted / received data 445 is data transmitted and received via the network interface 430. The RAM 440 also has an application execution area 446 for executing various application modules.

[0110] The storage 450 stores a database, various parameters, and the following data or programs required to implement this embodiment. The storage 450 stores a hair property type table 301. The hair property type table 301 is a table that manages the relationship between the hair property type 311 and the hair contour shape 312 shown in FIG. 3.

[0111] The storage 450 further stores a subject image acquisition module 451, a contour line extraction module 452, and a hair property estimation module 453. The subject image acquisition module 451 is a module that acquires a subject image 110 including the head of the subject 111. The contour line extraction module 452 is a module that extracts the hair contour line 112 of the subject 111 in the acquired subject image 110. The hair property estimation module 453 is a module that estimates the hair property of the subject 111 based on the extracted hair contour line 112 of the subject 111. These modules 451 to 453 are read into the application execution area 446 of the RAM 440 by the CPU 410 and executed. The control program 454 is a program for controlling the entire hair property estimation device 100.

[0112] The input / output interface 460 interfaces input / output data with input / output devices. A display unit 461 and an operation unit 462 are connected to the input / output interface 460. A storage medium 464 may also be connected to the input / output interface 460. A speaker 463 serving as an audio output unit, a microphone (not shown) serving as an audio input unit, or a GPS position determination unit may also be connected. Note that the RAM 440 and storage 450 shown in FIG. 4 do not include programs or data related to the general-purpose functions of the hair condition estimating device 100 or other feasible functions.

[0113] Next, a processing procedure of the hair property estimating device 100 will be described with reference to the flowchart shown in Fig. 5A. This flowchart is executed by the CPU 410 in Fig. 4 using the RAM 440, and realizes each functional configuration of the hair property estimating device 100 in Fig. 2A.

[0114] To implement this flowchart in an information processing device that operates as the hair property estimating device 100, it is necessary to copy and store a program for executing this flowchart, which is stored in an exchangeable storage medium such as a USB memory or a CD-ROM (not shown), into the storage 450. The program may be copied to the storage 450, for example, from a program server connected to the network to the storage 450 via the network.

[0115] In step S501 (subject image acquisition step), the subject image acquisition unit 201 acquires the subject image 110 including the head of the subject 111. In step S503 (contour line extraction step), the contour line extraction unit 202 extracts the hair contour line 112 of the subject 111 in the acquired subject image 110. In step S505 (hair property estimation step), the hair property estimation unit 203 estimates the hair property of the subject 111 based on the extracted contour line 112. As described above, these steps are realized by causing the computer (CPU 410) to execute the program stored in the storage 450.

[0116] 5B , after executing step 503 (contour line extraction step), feature amounts of the contour 112 may be extracted in step 504 (contour feature amount extraction step). As described above, feature amount extraction may be performed in the contour line extraction unit 202, the hair condition estimating unit 203, or the contour feature amount extraction unit. However, in this example, the feature amounts of the contour 112 are extracted in an independent contour feature amount extraction unit. Specifically, the contour feature amount extraction unit divides the contour 112 extracted in the contour line extraction step 503 into regions according to the height of each part of the face, such as the eyes and mouth, and extracts feature amounts of the contour 112 in each region, such as the angle and length.

[0117] According to this embodiment, the hair characteristics are determined from the hair outline (silhouette), so the degree of curliness of the entire head of hair can be accurately estimated. Even if the hair is only curled on the inside, the degree of curliness of the hair can be estimated from the outline, so the degree of curliness of the entire head of hair can be reliably estimated. Furthermore, since the hair characteristics are determined from the hair outline, the degree of curliness of the entire head of hair, which cannot be estimated from the shape of a single hair, can be reliably estimated.

[0118] Furthermore, according to this embodiment, the contour line is extracted from the entire hair and the degree of curl of the hair is estimated from the feature amount, so the amount of data directly used for estimation is small and the time required to obtain the estimation result is short.

[0119] Furthermore, because the hair condition estimating device 100 estimates the degree of curl from the entire contour, it is possible to finely distinguish between weak curls, which was previously difficult to distinguish. This makes it easier to determine whether it is better to keep the curl or to straighten the hair, and it allows for more appropriate suggestions for hairstyles and hair cosmetics.

[0120] Second Embodiment Next, a hair property estimating device according to a second embodiment of the present invention will be described with reference to Figs. 6 to 9. Fig. 6 is a block diagram for explaining the configuration of a hair property estimating device 600 according to this embodiment. The hair property estimating device 600 according to this embodiment differs from the first embodiment in that it includes a display control unit and an input receiving unit. As the other configurations and operations are the same as those of the first embodiment, the same configurations and operations are denoted by the same reference numerals and detailed descriptions thereof will be omitted.

[0121] The hair condition estimating device 600 has a display control unit 601 and an input receiving unit 602. The display control unit 601 displays the contour 112 extracted by the contour extraction unit 202 superimposed on the subject image 110 (superimposed image). For example, the display control unit 601 displays the contour 112 of the subject's 111 hair superimposed on the subject image 110, but may display a thick line or a colored line as the contour 112 to emphasize that it is a hair contour. By displaying it in this way, the hair contour is emphasized, allowing the subject 111 and other users to easily recognize the contour 112 of the subject's 111 hair.

[0122] The input receiving unit 602 receives input regarding the suitability of the superimposed displayed contour line 112. That is, the input receiving unit 602 receives input regarding whether the hair contour line 112 highlighted and displayed by the display control unit 601 is applicable as the hair contour line 112 of the subject 111. The input of suitability may be performed using a mobile terminal such as a smartphone owned by the subject 111, or may be performed by the subject 111 or another person using a predetermined input device or the like. That is, whether the hair contour line 112 has been correctly extracted is checked by a person.

[0123] If the received input result is "applicable," the input receiving unit 602 sends the input result to the hair property estimation unit 203. If the input result is "not applicable," the input receiving unit 602 sends the input result to the contour line extraction unit 202. Upon receiving the input result indicating "not applicable," the contour line extraction unit 202 may again acquire the subject image 110 and extract the contour line 112. Alternatively, the contour line extraction unit 202 may perform multiple contour line extractions in advance, display these in an array, and allow the user to input whether the contour line is appropriate or not, thereby enabling selection of the most appropriate contour line 112. It is preferable to perform multiple contour line extractions by acquiring multiple subject images in advance and performing contour line extraction for each of the acquired multiple subject images. Alternatively, multiple contour line extractions can be performed by performing image processing, such as different filtering, on a single subject image before contour line extraction, thereby acquiring multiple different images, and performing contour line extraction on each of the multiple image-processed subject images. Of course, it is also possible to use a mixture of these two methods for extracting multiple contour lines.

[0124] The hair property estimation unit 203 then estimates the hair property of the subject 111 based on the contour lines that are determined to be "applicable." For contour lines that are determined to be "inapplicable," the contour line extraction unit 202 redoes the extraction of the hair contour lines of the subject 111. The input receiving unit 602 then receives input regarding the suitability of the contour lines again. In this way, by receiving input regarding the suitability of the contour lines using the input receiving unit 602, it is possible to estimate the hair property based on a hair contour line that is thought to be more accurate, thereby improving the accuracy of the estimation.

[0125] The hair property estimation unit 203 estimates the property of the subject's hair using a trained hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line and the property. Preferably, feature quantities of the contour line are extracted, and the property is estimated from the extracted feature quantities. Furthermore, the feature quantities are preferably extracted from one or more predetermined regions of the contour line. For example, either the angle or the length of the contour line may be used as the feature quantity, or both the angle and the length of the contour line may be used.

[0126] Next, an example of the action table 701 of the hair property estimating device 600 will be described with reference to Fig. 7. The action table 701 stores an action 712 in association with an input result 711. The input result 711 is an input regarding the suitability of the superimposed outline. The action 712 is the next action determined according to the input result. The hair property estimating device 600 then refers to the action table 701 to determine the next action.

[0127] The hardware configuration of the hair condition estimating device 600 will be described with reference to Figure 8. The RAM 840 is a random access memory used by the CPU 410 as a work area for temporary storage. The RAM 840 has a storage area reserved for storing data necessary for implementing this embodiment. The input result 841 is an input regarding the suitability of the displayed outline. The action candidate data 842 is data regarding the next action determined according to the input result.

[0128] The storage 850 stores a database, various parameters, or the following data or programs required to implement this embodiment. The storage 850 stores an action table 701. The action table 701 is a table that manages the relationship between the input result 711 and the action 712 shown in FIG. 7.

[0129] The storage 850 further stores a display control module 851 and an input receiving module 852. The display control module 851 is a module that displays the extracted contour line superimposed on the subject person image 110. The input receiving module 852 is a module that receives input regarding the suitability of the superimposed displayed contour line.

[0130] Next, the processing procedure of the hair property estimating device 600 will be described with reference to the flowchart shown in Figure 9. This flowchart is executed by the CPU 410 in Figure 8 using the RAM 840, and realizes each functional configuration of the hair property estimating device 600 in Figure 6. The program required to realize this flowchart is stored in the storage 850 by copying it from an exchangeable storage medium such as a USB memory or CD-ROM (not shown) to the storage 850, or by receiving it from a server via a communication line and copying it to the storage 850. In this way, this flowchart is realized in the hair property estimating device 600. The same applies to the necessary data and tables.

[0131] In step S901 (superimposed display step), the display control unit 601 displays the extracted contour line superimposed on the subject image 110. In step S903 (suitability input receiving step), the input receiving unit 602 receives input regarding the suitability of the superimposed contour line. In step S905, the hair property estimating device 600 determines whether the input result is "suitable." If the input result is "suitable" (YES in step S905), the hair property estimating device 600 proceeds to step S505. If the input result is "inapplicable (not suitable)" (NO in step S905), the hair property estimating device 600 returns to step S503 and repeats the subsequent processes. Having received the input result indicating inapplicability, the contour line extraction unit 202 again acquires the subject image 110 and extracts the contour line in step S503. Also, in step S503, the contour line extraction unit 202 may extract multiple contour lines in advance, and then display them side by side, prompting the user to input whether they are suitable or not, allowing the user to select the contour line that is most suitable.

[0132] According to this embodiment, input regarding the suitability of the extracted contour line is accepted, so that the hair condition can be estimated based on a more accurate hair contour line, thereby further improving the accuracy of the condition estimation.

[0133] [Third Embodiment] Next, a hair property estimating device according to a third embodiment of the present invention will be described with reference to Figures 10 to 13. Figure 10 is a block diagram illustrating the configuration of a hair property estimating device 1000 according to this embodiment. The hair property estimating device 1000 according to this embodiment differs from the first or second embodiment in that it includes a hair care plan generating unit, a presenting unit, and a question presenting / answer receiving unit 1003. The other configurations and operations are the same as those of the first or second embodiment, so the same configurations and operations are denoted by the same reference numerals and detailed description thereof will be omitted.

[0134] The hair property estimation device 1000 has a hair care plan generation unit 1001, a presentation unit 1002, and a question presentation / answer reception unit 1003. The hair care plan generation unit 1001 generates a hair care plan according to the hair properties of the subject 111 based on the estimation results by the hair property estimation unit 203. The hair care plan may include, for example, product suggestions, product usage suggestions, hairstyle recommendations, salon treatment menu suggestions, and salon treatment timing suggestions. In addition to these, the hair property estimation device 1000 also has a subject image acquisition unit 201, a contour line extraction unit 202, a hair property estimation unit 203, a display control unit 601, and an input reception unit 602, similar to those in the first or second embodiment.

[0135] When the subject's 111 hair type is, for example, a strongly curled type 121 or a strongly curled type 122 (when the hair has a large width or volume), the hair care plan generation unit 1001 generates a hair care plan that makes use of the subject's naturally strong curl shape, such as a perm. When the subject's 111 hair type is, for example, a weakly curled type 123 or a weakly curled type 124 (when the hair has a small width or volume), the hair care plan generation unit 1001 generates a hair care plan that straightens the hair, a hair care plan that includes products and styling methods that make styling easier, or a style that makes use of the naturally curl shape of each type. In this way, the hair care plan generation unit 1001 generates a hair care plan according to the strength and degree of curl of the hair.

[0136] For example, in the case of subject 111 with very curly hair, the hair care plan generation unit 1001 generates a hair care plan that makes use of the curly hair and turns the curly hair into a personality or strength of subject 111, rather than regarding the curly hair as a complex. For subject 111 who feels complex about their curly hair, even if a hair care plan is generated to straighten their curly hair, the complex about having curly hair will not go away and will remain. In such a case, the hair care plan generation unit 1001 generates a plan that allows subject 111 to have confidence in their hair or to love their hair. For example, the hair care plan generation unit 1001 generates a hair care plan that recommends hairstyles that allow the subject 111 to enjoy curl styles or wavy styles that make the most of their curls and waves, recommended hair care products, etc.

[0137] Furthermore, for the subject 111 with weakly curly hair, the hair care plan generation unit 1001 generates a hair care plan to eliminate (reduce) the curl, for example, a plan to make the hair straight and manageable by simply washing the hair (without using a straightening iron, etc.), or a plan to make it easier to straighten the curls with a straightening iron, etc. Furthermore, the hair care plan generation unit 1001 generates a hair style that straightens out the curls, recommended hair care products, etc. as a hair care plan.

[0138] The presentation unit 1002 presents the generated hair care plan to the subject 111. For example, the presentation unit 1002 transmits data related to the display of the generated hair care plan to a mobile device such as a smartphone held by the subject 111 or a predetermined monitor, and displays the data on the display of the mobile device. The presentation unit 1002 also transmits data related to the display of the hair care plan to a monitor or the like connected to the hair property estimating device 1000, and displays the hair care plan. The presentation unit 1002 may also transmit the original hair properties and the analysis results of the hair properties to a predetermined display device along with the presentation of the hair care plan, and display the original hair properties and the analysis results of the hair properties. The presentation unit 1002 may also transmit only the estimated hair properties to a predetermined display device and display the original hair properties and the analysis results of the hair properties.

[0139] The question presenting / answer receiving unit 1003 presents a question to the subject 111 and receives an answer to the question. The question is provided to the subject 111 from the hair condition estimating device 1000, for example. The provided question may be a one-off question, or may be in an interview format (question and answer) in which the next question changes depending on the answer of the subject 111. The content of the question may be, for example, but is not limited to, daily feelings about one's own hair quality and condition, parts that the subject would like to fix, hairstyles that the subject would like to try, etc. The question presenting / answer receiving unit 1003 may present a plurality of possible answer candidates to the question to the subject 111, allowing the subject to select an appropriate answer from among them.

[0140] Next, an example of a hair care plan table 1101 included in the hair property estimating device 1000 will be described with reference to FIG. 11 . The hair care plan table 1101 stores answer details 1112 and hair care plans 1113 in association with hair properties 1111. The hair properties 1111 are estimated hair properties of the subject 111 and are classified into a first type 121, a second type 122, a third type 123, and a fourth type 124. The answer details 1112 are answers from the subject 111 to questions. The hair care plan 1113 is derived from a combination of the subject 111's hair properties 1111 and the answer details 1112. The example shown here is an example of the hair care plan table 1101, and it is not necessary for one hair care plan to correspond to a combination of a hair property and an answer detail. In other words, multiple hair care plans may correspond to a combination of a hair property and an answer detail. The hair property estimating device 1000 then generates a hair care plan by referring to the hair care plan table 1101. It is also possible to configure the hair care plan generating unit 1001 to create a hair care plan solely from the hair properties estimated from the contour by the hair property estimating unit 203 without receiving answers to the subject's questions, i.e., without having the question presenting / answer receiving unit 1003. In this case, the hair care plan table 1101 is composed of only the hair properties 1111 and the hair care plan 1113.

[0141] The hardware configuration of the hair property estimating device 1000 will be described with reference to Fig. 12. The RAM 1240 is a random access memory used by the CPU 410 as a temporary storage work area. The RAM 1240 has a storage area reserved for storing data necessary for implementing this embodiment. The hair care plan data 1241 is data related to the hair care plan to be generated. The question data 1242 is data related to questions presented to the subject 111. The answer data 1243 is data related to the subject 111's answers to the questions.

[0142] Storage 1250 stores a database, various parameters, and the following data or programs required to implement this embodiment. Storage 1250 stores a hair care plan table 1101. Hair care plan table 1101 is a table that manages the relationship between hair properties 1111 and hair care plans 1113, etc., as shown in FIG. 11 . The data or programs required to implement this embodiment are stored in storage 1250 by copying them from an exchangeable storage medium such as a USB memory or CD-ROM (not shown) to storage 1250, or by copying them from a server or the like to storage 1250 via a communication line (not shown).

[0143] Storage 1250 further stores a hair care plan generation module 1251, a presentation module 1252, and a question presentation / answer reception module 1253. Hair care plan generation module 1251 is a module that generates a hair care plan according to the hair properties of subject 111 based on the estimation result by hair property estimation unit 203. Presentation module 1252 is a module that presents the generated hair care plan to subject 111. Question presentation / answer reception module 1253 is a module that presents questions to subject 111 and receives answers to the questions.

[0144] Next, the processing procedure of the hair property estimating device 1000 will be described with reference to the flowchart shown in Fig. 13. This flowchart is executed by the CPU 410 in Fig. 12 using the RAM 1240, and realizes each functional configuration of the hair property estimating device 1000 in Fig. 10.

[0145] In step S501 (subject image acquisition step), the subject image acquisition unit 201 acquires a subject image 110 including the head of the subject 111, more specifically, from the top of the head to the chin. In step S503 (contour line extraction step), the contour line extraction unit 202 extracts a hair contour line 112 of the subject 111 in the acquired subject image 110. In step S901 (superimposed display step), the display control unit 601 displays the extracted contour line superimposed on the subject image 110. In step S903 (suitability input reception step), the input reception unit 602 receives input regarding the suitability of the superimposed contour line. In step S905, the hair property estimating device 600 determines whether the input result is "suitable." If the input result is "suitable" (YES in step S905), the hair property estimating device 600 proceeds to step S505. If the input result is "not suitable" (NO in step S905), the hair condition estimating device 600 returns to step S503 and repeats the subsequent processes. In step S505 (hair condition estimating step), the hair condition estimating unit 203 estimates the hair condition of the subject 111 based on the extracted contour line 112.

[0146] In step S1301 (question presentation / answer reception step), the question presentation / answer reception unit 1003 asks a question to the subject 111 and receives an answer to the question. In step S1303 (hair care plan generation step), the hair care plan generation unit 1001 generates a hair care plan 1113 according to the hair characteristics of the subject 111 from a combination of the hair characteristics 1111 of the subject 111 and the answer content 1112.

[0147] According to this embodiment, a hair care plan that matches the characteristics of the subject's hair can be proposed. In addition, since the hair care plan is generated taking into account the subject's answers to questions, a hair care plan that matches the subject's preferences and requests and provides a high level of satisfaction can be proposed.

[0148] [Fourth Embodiment] Next, a hair property estimating system according to a fourth embodiment of the present invention will be described with reference to Figs. 14 and 15. Fig. 14 is a diagram illustrating the configuration of a hair property estimating system 1400 according to this embodiment. The hair property estimating system 1400 according to this embodiment includes a mobile terminal 1401 and a hair property estimating device 1402. The mobile terminal 1401 captures an image including the head of a subject 111 and acquires the captured image as a subject image 110. After acquiring the subject image 110, the mobile terminal 1401 extracts a contour line 112 of the subject's 111 hair (hair) and displays the contour line 112 superimposed on the subject image 110 to generate a contour line image 113. The mobile terminal 1401 then transmits the generated contour line image 113 to the hair property estimating device 1402. The hair condition estimating device 1402 receives the contour line image 113 and estimates the hair condition of the subject 111 based on the contour line 112 (hair contour) shown in the contour line image 113. The hair condition is estimated by determining which of four types (first type 121 to fourth type 124 in order of curliness) the hair falls into, for example.

[0149] Next, with reference to Fig. 15 , the configurations of the mobile terminal 1401 and hair property estimation device 1402 included in the hair property estimation system 1400 will be described. The mobile terminal 1401 is a smartphone, tablet terminal, or the like, and has a subject image acquisition unit 1501 and a contour line extraction unit 1502. The mobile terminal 1401 and the hair property estimation device 1402 can communicate with each other via wireless communication or wired communication.

[0150] <Mobile terminal 1401> The mobile terminal 1401 has a camera, a subject image acquisition unit 1501 that acquires a subject image 110 captured by the camera, a contour line extraction unit 1502 that extracts a hair contour line 112 of the subject 111 from the acquired subject image, a display control unit 1503 that displays the contour line 112 extracted by the contour line extraction unit superimposed on the subject image, a display unit (not shown) that displays a suitability confirmation image, and a transmission unit that transmits the hair contour line 112 extracted by the contour line extraction unit to the hair condition estimation device 1402.

[0151] The subject image acquisition unit 1501 of the mobile terminal 1401 acquires an image including the head of the subject 111 as the subject image 110. The subject image 110 may be captured by a camera provided in the mobile terminal 1401, or may be captured by an imaging device other than the mobile terminal 1401. The subject image 110 may also be an image (selfie image) captured using an in-camera provided in the mobile terminal 1401, for example.

[0152] When the subject image 110 is captured by a camera or the like provided in the mobile terminal 1401, the captured subject image 110 is temporarily stored in a storage area of ​​the camera or the like. Then, the subject image acquisition unit 1501 may acquire the stored subject image 110, or the subject image acquisition unit 1501 may acquire the subject image 110 by receiving the subject image 110 transmitted from the camera or the like.

[0153] The contour line extraction unit 1502 extracts a contour line 112 of the hair of the subject 111 in the subject image 110. The contour line extraction unit 1502 also extracts a feature amount of the contour line 112. The feature amount of the contour line 112 is, for example, the extent (angle) of hair spread in each of the divided regions obtained by dividing the head of the subject 111 into several regions from the top of the head to the lower jaw.

[0154] When transmitting the contour line image 113 from the mobile terminal 1401 to the hair property estimating device 1402, only information about the contour line 112 may be transmitted to the hair property estimating device 1402. This is because, from the viewpoint of protecting personal information, it may not be appropriate for an enterprise operating the hair property estimating device 1402 to possess an image showing the face of the subject 111.

[0155] Furthermore, the contour line extraction unit 1502 extracts features of the contour line 112. The contour line extraction unit 1502 divides the subject image 110 into several regions from the top of the head to the lower jaw of the subject 111, and extracts feature amounts of the contour line 112 for each divided region. The feature amount of the contour line 112 is, for example, the angle of the contour line 112. The contour line extraction unit 1502 then transmits the feature amount of the contour line 112 together with the contour line image 113 to the hair condition estimating device 1402.

[0156] As described above, by transmitting only the data related to the contour line 112 and the data related to the feature quantities of the contour line 112 from the mobile terminal 1401 to the hair property estimation device 1402, personal information such as a facial photograph of the subject 111 is not passed to the operator of the hair property estimation system 1400. Therefore, the operator does not need to take measures to protect personal information, and the burden on the system operation can be reduced. Note that instead of transmitting both the data related to the contour line 112 and the data related to the feature quantities of the contour line 112 to the hair property estimation device 1402, it is also possible to configure the system so that only one of the data, i.e., only the data representing the contour line 112 or only the data related to the feature quantities of the contour line 112, is transmitted from the mobile terminal 1401 to the hair property estimation device 1402.

[0157] When transmitting only data related to the feature of the contour 112, the feature may be, for example, the length of the contour 112 in a predetermined area, rather than the angle of the contour 112. Alternatively, it is also possible to transmit both the angle and the length of the contour 112. By transmitting only the data related to the feature of the contour 112, the amount of data can be reduced compared to transmitting data representing the contour 112.

[0158] The display control unit 1503 displays the extracted contour (contour line 112) superimposed on the subject image 110. This allows the subject 111 to check the extracted contour line 112 on the display of the mobile terminal 1401. After checking the contour line 112, the subject 111 can perform an operation such as capturing a face image again, if necessary.

[0159] It should be noted that the mobile terminal 1401 of this embodiment does not necessarily require the display control unit 1503, which displays the contour 112 extracted by the contour extraction unit superimposed on the subject image, as described above. If the mobile terminal 1401 does not have the display control unit 1503, the display unit need not have a function for displaying a suitability confirmation image. Even in this case, by limiting the data transmitted from the mobile terminal 1401 to the hair condition estimating device 1402 to only data representing the contour 112 or only data related to the feature quantities of the contour 112, or both, a technical effect is achieved in which personal information such as a facial photograph of the subject 111 is not provided to the operator of the hair condition estimating device 1402, and the amount of data to be transmitted can be reduced.

[0160] The hair property estimation system 1400 includes a display control unit 1503 that superimposes the contour line 112 extracted by the contour line extraction unit 1502 on the subject image and displays it as a suitability confirmation image, and an input receiving unit 1511 that receives input regarding the suitability of the superimposed contour line 112, and the hair property estimation unit 1510 estimates the property of the subject's hair based on the received input.

[0161] The program used in the mobile terminal of the hair property estimating system 1400 can be configured to cause the mobile terminal to execute a contour line extraction step, in which the contour line extraction unit 1502 extracts a hair contour line 112 from a subject image, and a transmission control step, in which the mobile terminal transmits the hair contour line 112 extracted in the contour line extraction step to the hair property estimating device 1402. The program may also be configured to cause the mobile terminal 1401 to execute a display control step, in which the contour line 112 extracted by the contour line extraction unit is superimposed on the subject image and displayed as a suitability check image, an input receiving step, in which an input regarding the suitability of the superimposed contour line 112 is received, and a transmission control step, in which the contour line 112 is transmitted to the hair property estimating device 1402 in accordance with the input in the input receiving step. In this case, a plurality of contour line images in which the subject image and the contour line 112 are superimposed may be displayed, and the contour line 112 to be transmitted may be selectable from the plurality of superimposed images in the input step. It is preferable that the program used in the above-mentioned mobile terminal 1401 be stored, for example, on a server accessible from the Internet, and sent to the user's mobile terminal 1401 via a communication line such as the Internet so that it can be downloaded.

[0162] <Hair Property Estimating Device 1402 > The hair property estimating device 1402 includes a hair property estimating unit 1510 , an input receiving unit 1511 , a hair care plan generating unit 1512 , a presenting unit 1513 , and a question presenting / answer receiving unit 1514 .

[0163] The hair property estimation unit 1510 estimates the hair property of the subject 111 using a trained hair property estimation model generated by having an artificial intelligence learn the relationship between the hair outline 112 and the hair property. The hair property is, for example, the degree of hair curl, and the hair property is estimated based on the degree of spread of the subject 111's hair.

[0164] The input receiving unit 1511 receives an input regarding the suitability of the superimposed contour line 112. The subject 111 inputs the suitability by looking at the display of the mobile terminal 1401. The suitability may be input using, for example, an input device provided in the mobile terminal 1401 or a connected input device.

[0165] Furthermore, input regarding suitability may be made using an input device connected to the hair property estimating device 1402. The hair property estimating unit 1510 estimates the hair property of the subject 111 based on the received input result. That is, the system administrator or the like checks whether the extracted contour (contour line 112) is applicable on the display of the hair property estimating device 1402. If the system administrator or the like determines that it is applicable, the hair property estimating device 1402 performs the subsequent processing.

[0166] More specifically, the subject 111 or the like may input the suitability of the contour 112 by pressing a "YES" or "NO" button displayed on the mobile terminal 1401 while viewing the display of the contour 112 (suitability confirmation image) displayed on the display of the mobile terminal 1401. Then, the mobile terminal 1401 transmits the input information of suitability to the hair property estimation device 1402. Note that the input information of suitability may be transmitted to the hair property estimation device 1402 together with the contour image 113 or information only of the contour 112 (information of the contour 112). The input receiving unit 1511 receives the input regarding the suitability of the contour 112 transmitted in this manner.

[0167] Here, the information about only the contour 112 refers to information about the shape and position of the contour 112, i.e., coordinates 235. For example, the amount of data can be compressed by sending only the position information of the pixels that make up the contour 112, rather than sending the two-dimensional image data in which the contour 112 appears. Alternatively, the amount of data can be compressed by sending only information about the positions and shapes of the curves and straight lines that make up the contour 112. These data compression methods can be performed using techniques well known to those skilled in the art without any restrictions.

[0168] The hair care plan generation unit 1512 generates a hair care plan according to the hair properties of the subject 111 based on the hair property estimation result. The hair care plan generation unit 1512 generates a plan according to the hair properties (degree of curl) of the subject 111. If the hair properties are of a strong curl type (first type 121 and second type 122), for example, a plan that makes use of the strength of the curl is generated. Also, if the hair properties are of a weak curl type (third type 123 and fourth type 124), for example, a plan to straighten the curl or a plan that makes use of the weak curl is generated.

[0169] The presentation unit 1513 presents the generated hair care plan to the subject 111. For example, the presentation unit 1513 transmits data regarding the generated hair care plan to the mobile terminal 1401, and the mobile terminal 1401, upon receiving the data, displays the hair care plan on a display, thereby presenting the hair care plan to the subject 111.

[0170] The question presenting / answer receiving unit 1514 asks questions to the subject 111 and receives answers to those questions. The questions are transmitted from the hair property estimating device 1402 to the mobile terminal 1401. The transmitted questions are displayed on the display of the mobile terminal 1401, and when the subject 111 answers the questions and transmits data related to the answers from the mobile terminal 1401 to the hair property estimating device 1402, the question presenting / answer receiving unit 1514 receives the data related to the answers. The content of the questions may be, for example, questions about a hairstyle that the subject 111 wants to try, how the subject 111 usually cares for their hair, any problems or complaints about their hair properties, etc., but is not limited to these.

[0171] Then, the hair care plan generation unit 1512 generates a hair care plan taking into account the answers received by the question presentation / answer reception unit 1514. The generated hair care plan may be a plan that makes the most of the hair's characteristics (curls), a plan that straightens the hair's characteristics (curls), or the like, and the plan may include styling products, styling methods, and the like required to achieve this.

[0172] According to this embodiment, information related to the subject's personal information remains on the mobile terminal, and only information not related to personal information is sent to the hair condition estimating device, so that companies operating hair condition estimating devices can estimate hair properties without handling personal information.

[0173] <Other Forms> The functional components of the mobile terminal 1401 and the hair property estimation device 1402 included in the hair property estimation system 1400 according to this embodiment are not limited to the examples described above. Each of the functional components of the mobile terminal 1401 and the hair property estimation device 1402 may be included in any terminal or device.

[0174] For example, the contour line extraction unit 1502 may be included in the hair property estimation device 1402. In this case, the subject image 110 is transmitted from the mobile terminal 1401 to the hair property estimation device 1402, and the transmitted subject image 110 has been subjected to processing such that the face of the subject 111 is not recognizable to an operator of the hair property estimation device 1402. Alternatively, data in which only the face of the subject 111 has been removed from the subject image 110 may be transmitted to the hair property estimation device 1402.

[0175] Furthermore, the input receiving unit 1511 may be provided in the mobile terminal 1401. In this case, the mobile terminal 1401 receives an input regarding the suitability of the extracted contour, and the input result is transmitted from the mobile terminal 1401 to the hair property estimation unit 1510 of the hair property estimation device 1402. The hair property estimation unit 1510 estimates the hair property of the subject 111 based on the received input result. That is, the subject 111 checks on the display of the mobile terminal 1401 whether the extracted contour (contour line 112) is applicable. If the subject 111 determines that the contour is applicable, the mobile terminal 1401 transmits data regarding the extracted contour to the hair property estimation device 1402, and the hair property estimation device 1402 performs the subsequent processing. Note that the input regarding the suitability is not necessarily required. That is, the input receiving unit 1511 may not be provided, and the information relating to the contour line extracted in the mobile terminal 1401 may not be judged as to suitability in either the mobile terminal 1401 or the hair condition estimating device, and the feature values ​​may be extracted and the subsequent processing may be performed in the hair condition estimating device 1402. Alternatively, the mobile terminal 1401 may extract the contour line and the feature values ​​of the contour line from the captured image without judging the suitability, and the hair condition estimating device 1402 may perform the subsequent processing.

[0176] The questions for the subject 111 may not be transmitted from the hair property estimating device 1402 to the mobile terminal 1401, but may be stored in advance in the mobile terminal 1401 and presented to the subject 111. Specifically, a program executed on the mobile terminal may have the content of the questions and a function for presenting them. That is, the mobile terminal may be provided with a question presenting / answer receiving unit 1514. In this case, the content of the questions may be constant regardless of the hair property, or may vary depending on the hair property. In the latter case, information on the estimated hair property is transmitted from the hair property estimating device 1402 to the mobile terminal 1401.

[0177] Fifth Embodiment Next, a hair property estimating system 1600 according to a fifth embodiment of the present invention will be described with reference to FIGS. 16 to 18. FIG. 16 is a block diagram illustrating the configurations of a mobile terminal 1601 and a hair property estimating server 1602 included in the hair property estimating system 1600 according to this embodiment. The hair property estimating system 1600 according to this embodiment differs from the fourth embodiment in that the input receiving unit 1511 is included in the mobile terminal 1601 rather than the hair property estimating server 1602. Since the other configurations and operations are the same as those of the fourth embodiment, the same configurations and operations are denoted by the same reference numerals and detailed descriptions thereof will be omitted. In this embodiment, the amount of information for hair property estimation sent from the mobile terminal 1601 to the hair property estimating server 1602 is minimized.

[0178] <Mobile terminal 1601> The mobile terminal 1601 has a camera (not shown), a subject image acquisition unit 1501, a contour line extraction unit 1502, a display control unit 1503, an input reception unit 1511, and a transmission unit (not shown).

[0179] The subject image acquisition unit 1501 acquires an image including at least the head of the subject 111, more specifically, from the top of the head to the tip of the chin, as the subject image 110. The subject image acquisition unit 1501 acquires the subject image 110 that is temporarily stored in a storage area of ​​a camera or the like. The subject image 110 is an image that includes at least the head of the subject 111, more specifically, from the top of the head to the tip of the chin.

[0180] The contour line extraction unit 1502 extracts the outer contour line of the subject's 111 hair as the contour line 112. Here, the outer contour line refers to the side of the hair contour line that is not in contact with the face. The contour line extraction unit 1502 then extracts feature quantities from the extracted outer contour line. In this embodiment, to reduce the amount of calculations, the outer contour line is extracted only from the area from the top of the head to the chin in the subject's image 110. The feature quantities extracted by the contour line extraction unit 1502 are the feature quantities of the outer contour line. For example, the head of the subject 111 is divided into several areas based on the facial parts between the top of the head and the chin, and the feature quantities of the outer contour line are the extent (angle) of the hair in the divided areas, i.e., the extent of the contour line, or the length of the contour line 112 in the divided areas, or a multidimensional quantity (vector quantity) appropriately selected and combined from these angles or lengths. The feature amount of the outer contour line from the top of the head to the tip of the chin extracted by the contour line extraction unit 1502 is transmitted to the hair condition estimation server 1602 via a transmission unit (not shown).

[0181] It is also possible to configure the contour line extraction unit 1502 to extract the contour line 112 but not to extract the feature amounts. In this case, the hair property estimation server 1602 extracts the feature amounts. When the contour line extraction unit 1502 does not extract the feature amounts of the contour line, the contour line extraction unit 1502 extracts data related to the contour line 112. Here, the data related to the contour line 112 is data that represents at least the shape of the contour line 112. From the viewpoint of enabling the hair property estimation server 1602 to correctly define the above-mentioned divided area when extracting the feature amounts of the contour line 112, it is preferable that the data related to the contour line 112 further include data that represents the position of the contour line 112 relative to the eyes, chin, and other feature points of the face, in addition to the shape of the contour line 112.

[0182] As described above, by transmitting only the data related to the contour line 112 or the feature values ​​of the contour line 112 from the mobile terminal 1601 to the hair property estimation server 1602, personal information such as a facial photograph of the subject 111 is not passed to the operator of the hair property estimation server 1602. Therefore, the operator does not need to take measures to protect personal information, and the burden on the system operation can be reduced. Furthermore, compared to sending an image containing the face of the subject 111, transmitting only the data related to the contour line 112 or the feature values ​​of the contour line 112 from the mobile terminal 1601 to the hair property estimation server 1602 is preferable because it reduces the amount of data transmitted from the mobile terminal 1601 to the hair property estimation server 1602.

[0183] The display control unit 1503 displays the extracted contour (contour line 112) superimposed on the subject image 110. The display control unit 1503 also displays information received from the hair condition estimating server 1602. The display control unit 1503 displays images and the like on the display of the mobile terminal 1601. Note that the display control unit 1503 may also display images and the like on a display connected to the mobile terminal 1601 by wire or wirelessly.

[0184] <Hair Property Estimation Server 1602 > The hair property estimating server 1602 includes a hair property estimation unit 1510 , a question presentation / answer reception unit 1514 , a hair care plan generation unit 1512 , and a presentation unit 1513 .

[0185] The question presentation / answer reception unit 1514 creates a question for the subject 111 based on the hair property estimated by the hair property estimation unit 1510, sends the created question to the mobile terminal 1601, and receives an answer to the question sent from the mobile terminal 1601.

[0186] The hair care plan generation unit 1512 creates a hair care plan based on the estimated hair properties and the response sent from the mobile terminal 1601.

[0187] The presentation unit 1513 presents the hair care plan created by the hair care plan generation unit 1512 to the mobile terminal 1601 to the subject 111.

[0188] Next, the processing procedures of the mobile terminal 1601 and the hair property estimation server 1602 will be described with reference to the flowcharts shown in Figures 17A and 17B. This flowchart is executed by storing a program stored in an exchangeable storage medium such as a USB memory or a CD-ROM in a storage device in the mobile terminal 1601 or the hair property estimation server 1602, and starting the program by an operation by the subject 111, for example. The program may be stored in a storage device in the mobile terminal 1601 or the hair property estimation server 1602 by downloading it via a network from a server on which the program is saved.

[0189] <Processing Procedure of Portable Terminal 1601> First, the processing procedure of portable terminal 1601 will be described with reference to the flowchart shown in FIG. 17A . In step S501, subject image acquisition unit 1501 acquires subject image 110 including the head of subject 111. Specifically, subject image 110 is captured by the camera of portable terminal 1601. In step S503, contour line extraction unit 1502 extracts hair contour line 112 from the acquired subject image 110. In step S901, display control unit 1503 superimposes extracted hair contour line 112 on subject image 110 and displays an appropriateness check image on the display of portable terminal 1601. In this embodiment, the appropriateness check image is contour line image 113. In step S901, a display prompting the operator of portable terminal 1601, for example, subject 111, to input whether the contour line is appropriate is also displayed on the display of portable terminal 1601. In step S903, input receiving unit 1511 receives input regarding the suitability of the superimposed contour line. In step S905, portable terminal 1601 determines whether the input result is "suitable." If the input result is "suitable" (YES in step S905), portable terminal 1601 proceeds to step S1701. If the input result is "inappropriate (not suitable)" (NO in step S905), portable terminal 1601 returns to step S503 and repeats the subsequent processes.

[0190] In step S1701 (feature extraction step), the contour extraction unit 1502 extracts contour feature values ​​using information (contour data) about the hair contour 112 judged to be "suitable." In step S1703 (feature transmission step), the transmission unit transmits the extracted contour feature values ​​to the hair condition estimation server 1602.

[0191] In step S1705 (question receiving step), the mobile terminal 1601 receives a question corresponding to the hair property of the subject 111 from the hair property estimation server 1602. In step S1707, the display control unit 1503 displays the received question on the display of the mobile terminal 1601. Also in step S1707, the input accepting unit 1511 accepts an answer to the question entered by the subject 111. The question is a question that provides reference for determining what kind of hair care plan should be provided to the subject 111, such as about lifestyle habits, specifically the number of times and duration of hair washing, or hair care products used daily, and may be one or more questions. After accepting all answers to the questions, the mobile terminal 1601 proceeds to step S1709 (answer sending step).

[0192] In step S1709, the transmitting unit transmits the responses received by the input receiving unit 1511 to the hair condition estimating server 1602. When transmission of all received responses has been completed, the process proceeds to step S1711 (hair care plan receiving step).

[0193] In step S1711 , the mobile terminal 1601 receives the hair care plan from the hair condition estimating server 1602 .

[0194] In step S1713 (hair care plan display step), the display control unit 1503 displays the received hair care plan on the display of the mobile terminal 1601 .

[0195] <Processing Procedure of Hair Property Estimating Server 1602> Next, the processing procedure of the hair property estimating server 1602 will be described with reference to the flowchart shown in Fig. 17B. In step S1715, the hair property estimating server 1602 receives the feature amount of the hair contour line 112 transmitted from the mobile terminal 1601. Upon receiving the feature amount, the process proceeds to step S505.

[0196] In step S505, the hair property estimation unit 1510 estimates the hair property of the subject 111 based on the received feature amount. When the hair property estimation is completed, the process proceeds to step S1717 (question generation / transmission step).

[0197] In step S1717, the question presenting / answer receiving unit 1514 creates a question corresponding to the estimated hair property. One method for creating the question is to select a question from a plurality of questions prepared in advance based on the estimated hair property, the age of the subject, etc. In step S1717, the hair property estimation server 1602 also transmits the created question to the mobile terminal 1601 to present to the subject 111. After transmitting the question, the process proceeds to step S1719 (answer receiving step). In step S1719, the question presenting / answer receiving unit 1514 receives and accepts an answer entered in response to the presented question from the mobile terminal 1601. The answer is entered by the subject 111, transmitted from the mobile terminal 1601, and received by the hair property estimation server 1602. Once the answer reception is complete, the process proceeds to step S1303.

[0198] In step S1303, the hair care plan generation unit 1512 creates a hair care plan based on the hair properties estimated in step S505 and the response received in step S1719. The hair care plan is created by first selecting multiple candidate hair care plans to present to the subject 111 based on the estimated hair shape, and then selecting a hair care plan that is suitable for the subject 111's lifestyle, for example, from the selected multiple hair care plans, or eliminating incompatible hair care plans, based on the subject 111's response, i.e., the response received in step S1719. Specifically, for example, if the subject 111 responds that they do not usually perform care when bathing, a hair care plan that involves shampooing or other treatments performed when bathing would be excluded. Once the creation of the hair care plan is complete in step S1303, the process proceeds to step S1721 (hair care plan transmission step).

[0199] In step S1721, the presenting unit 1513 transmits the created hair care plan to the mobile terminal 1601. The transmitted hair care plan is displayed on the display of the mobile terminal 1601, as described above.

[0200] <Other Forms> In this embodiment, the display control unit 1503 controls the creation and display of the suitability check image so that the subject 111 can determine whether the acquired image is appropriate. However, it is also possible to configure the system so that the subject does not have to determine whether the image is appropriate. In this case, step S901 (superimposed display step), step S903 (suitability input acceptance step), and step S905 (suitability judgment step) are unnecessary. Alternatively, the acquired image may be displayed as is without being superimposed on a contour line, and used to determine whether the subject 111 can use the image for estimation. In this case, as shown in FIG. 18 , after acquiring the subject image 110 in step S501, the display control unit 1503 displays the subject image 110 on the display of the mobile terminal 1601 in step S1801. In step S903, the input acceptance unit 1511 accepts input regarding the suitability of the contour line displayed on the subject image 110 displayed on the display. In step S905, the mobile terminal 1601 determines whether the input result is "correct." If the input result is "correct" (YES in step S905), the mobile terminal 1601 proceeds to step S503. If the input result is "incorrect (not correct)" (NO in step S905), the mobile terminal 1601 returns to step S501 and repeats the subsequent processes.

[0201] As can be easily understood from the above explanation, even if the operation of the mobile terminal 1601 changes in this way, the operation of the hair condition estimating server 1602 does not change, so multiple types of programs may be prepared for the mobile terminal 1601.

[0202] Note that the determination of whether the area from the top of the head to the chin is included in the acquired image is merely an application of conventional face recognition technology. Therefore, the suitability determination can be performed by a program rather than by the subject 111. In this case, instead of step S903 (suitability input reception step), a photographing range determination step (not shown) for determining whether the area from the top of the head to the chin is included in the acquired image can be configured to be performed by the mobile terminal 1801 without the involvement of the subject 111. This makes it even easier to use the present system. In this case, there is no need to change the operation of the hair condition estimation server 1602, and therefore there is no need to change the program used in the hair condition estimation server 1602.

[0203] In this specification, "hair" means human hair. "Headwear" means, for example, hair wigs, hairpieces, weaving, hair extensions, braided hair, hair accessories, doll hair, etc. "Fiber for headwear" means fiber used in the headwear, excluding hair. In other words, the headwear uses either hair or fiber for headwear.

[0204] The present invention can be applied to hair mixed with fibers for head accessories or to hair with a head accessory attached, but from the viewpoint of accurately estimating the original properties of hair, application to hair alone or hair with a head accessory attached is preferred, and application to hair alone is more preferred. Fibers for head accessories may be either naturally occurring fibers or synthetic fibers, but naturally occurring fibers are preferred. Naturally occurring fibers refer to fibers (excluding hair) collected from natural plants and animals, or fibers artificially produced using proteins, polysaccharides, etc. as raw materials. Among these, preferred are artificially produced fibers made from animal hair, or proteins or polysaccharides such as proteins derived from soybeans, peanuts, corn, silk, etc., as well as keratin, collagen, casein, and the like; more preferred are regenerated protein fibers made from keratin, collagen, casein, soybean protein, peanut protein, corn protein, silk protein (e.g., silk fibroin), etc.; even more preferred are regenerated protein fibers such as regenerated collagen fibers made from collagen and regenerated silk fibers made from silk fibroin, with regenerated collagen fibers being even more preferred.

[0205] Regenerated collagen fibers can be produced by known techniques. The composition of regenerated collagen fibers does not need to be 100% collagen, and may contain natural polymers, synthetic polymers, additives, etc. to improve quality. Furthermore, regenerated collagen fibers may be post-processed or post-treated. Filaments are preferred as the form of regenerated collagen fibers. Filaments are generally removed from bobbins or boxes. Furthermore, filaments emerging from the drying process in the regenerated collagen fiber production process can also be used directly.

[0206] Examples of synthetic fibers include fibers containing a synthetic resin as a main component. From the viewpoint of ease of production of synthetic fibers and of obtaining a texture similar to that of human hair, the synthetic resin is preferably a thermoplastic resin, more preferably at least one selected from the group consisting of polyester resin, polyamide resin, polyimide resin, polyamideimide resin, vinyl chloride resin, polycarbonate resin, polyphenylene sulfide resin, and modacrylic resin (a copolymer of acrylonitrile and vinyl chloride). The term "main component" as used herein refers to a component whose content in the synthetic fiber is preferably 50% by mass or more, more preferably 60% by mass or more, even more preferably 70% by mass or more, still more preferably 80% by mass or more, and even more preferably 90% by mass or more, but not more than 100% by mass.

[0207] In addition to the above synthetic resins, the synthetic fibers may further contain various components such as flame retardants, flame retardant auxiliaries, light or heat stabilizers, fluorescent agents, antioxidants, antistatic agents, and ultraviolet absorbers, as long as the effects of the present invention are not impaired.

[0208] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments and can be modified as appropriate. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. Furthermore, systems or devices that combine separate features included in each embodiment in any manner are also included in the scope of the present invention.

[0209] The present invention may also be applied to a system consisting of multiple devices or to a single device. Furthermore, the present invention may also be applied when an information processing program that realizes the functions of the embodiments is supplied to a system or device in a format recorded on a removable storage medium such as a USB memory or CD-ROM and executed by an embedded processor. Therefore, the technical scope of the present invention also includes a program installed on a computer to realize the functions of the present invention, a storage medium storing the program, a WWW (World Wide Web) server from which the program can be downloaded, a processor that executes the program, and a program product. In particular, the technical scope of the present invention also includes a non-transitory computer-readable medium storing a program that causes a computer to execute at least the processing steps included in the above-described embodiments.

[0210] The following supplementary notes are further disclosed regarding the above-described embodiments of the present invention. <1> A hair property estimation device comprising: a subject image acquisition unit that acquires a subject image including the subject's head from the top to at least the tip of the chin; a contour line extraction unit that extracts a contour line of the subject's hair in the acquired subject image; and a hair property estimation unit that estimates the property of the subject's hair based on the extracted contour line. <2> The hair property estimation device according to <1>, wherein the hair property estimation unit estimates the property of the subject's hair based on at least one of the angle and the length of the extracted contour line. <3> The hair property estimation device according to <1> or <2>, wherein the contour line extraction unit further extracts feature amounts of the contour line, and the hair property estimation unit estimates the property of the subject's hair using the extracted feature amounts. <4> The hair property estimation device according to <3>, wherein the feature amounts are extracted from a predetermined region of the contour line. <5> The hair property estimation device according to <3> or <4>, wherein the feature amount is at least one of the angle and length of the contour line. <6> The hair property estimation device according to any one of <1> to <5>, wherein the hair property estimation unit estimates the property of the subject's hair using a trained hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line and the property. <7> The hair property estimation device according to <6>, wherein the trained hair property estimation model is generated by having an artificial intelligence learn the relationship between the feature amount of the contour line and the property. <8> The hair property estimation device according to any one of <1> to <7>, wherein the property includes the degree of hair curl. <9> The hair property estimation device according to any one of <1> to <8>, wherein the contour line extraction unit extracts the contour line of the subject's hair using a trained contour line extraction model generated by having an artificial intelligence learn the relationship between the subject's image and the contour line.<10> The hair property estimating device according to any one of <1> to <9>, further comprising: a display control unit that displays the contour extracted by the contour extraction unit superimposed on the image of the subject; and an input receiving unit that receives input regarding the suitability of the superimposed contour, wherein the hair property estimating unit estimates the hair property of the subject based on the received input. <11> The hair property estimating device according to any one of <1> to <10>, further comprising: a hair care plan generating unit that generates a hair care plan according to the hair property of the subject based on an estimation result by the hair property estimating unit; and a presentation unit that presents the generated hair care plan to the subject. <12> The hair property estimating device according to <11>, further comprising: a question presenting / answer receiving unit that receives an answer to a question from the subject, wherein the hair care plan generating unit generates the hair care plan from a combination of the answer and the hair property. <13> The hair property estimation device according to any one of <1> to <12>, further comprising a normalization unit that normalizes the subject image based on a length from the eyebrows to the chin of the subject in the subject image. <14> The hair property estimation device according to any one of <1> to <13>, wherein the subject image includes from the top of the subject's head to the ends of the hair, and the hair property estimation unit estimates an appearance of the hair ends as the property. <15> A hair property estimation method comprising: a subject image acquisition step of acquiring a subject image including from the top of the subject's head to at least the end of the chin, a contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image, and a hair property estimation step of estimating the property of the subject's hair based on the extracted contour line. <16> The hair property estimation method according to <15>, wherein the hair property estimation step estimates the property of the subject's hair based on at least one of the angle and the length of the extracted contour line. <17> The hair property estimation method according to <15> or <16>, wherein the contour line extraction step further extracts feature amounts of the contour line, and the hair property estimation step estimates the property of the subject's hair using the extracted feature amounts.<18> The hair property estimation method according to <17>, wherein the feature amount is extracted from a predetermined region of the contour line. <19> The hair property estimation method according to <17> or <18>, wherein the feature amount is at least one of the angle and the length of the contour line. <20> The hair property estimation method according to any one of <15> to <19>, wherein in the hair property estimation step, the property of the subject's hair is estimated using a trained hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line and the property. <21> The hair property estimation method according to <20>, wherein the trained hair property estimation model is generated by having an artificial intelligence learn the relationship between the feature amount of the contour line and the property. <22> The hair property estimation method according to any one of <15> to <21>, wherein the property includes the degree of hair curl. <23> The hair property estimating method according to any one of <15> to <22>, wherein in the contour line extraction step, the contour line of the subject's hair is extracted using a learned contour line extraction model generated by having artificial intelligence learn the relationship between the subject's image and the contour line. <24> The hair property estimating method according to any one of <15> to <23>, further comprising: a display control step of superimposing the contour line extracted in the contour line extraction step on the subject's image and displaying it; and an input receiving step of receiving input regarding the suitability of the superimposed displayed contour line, wherein in the hair property estimating step, the property of the subject's hair is estimated based on the received input. <25> The hair property estimating method according to any one of <15> to <24>, further comprising: a hair care plan generating step of generating a hair care plan according to the hair properties of the subject based on a result of the estimation in the hair property estimating step; and a presentation step of presenting the generated hair care plan to the subject. <26> The hair property estimating method according to <25>, further comprising a question presenting / answer receiving step of receiving an answer to a question of the subject, wherein in the hair care plan generating step, the hair care plan is generated from a combination of the answer and the hair properties.<27> The hair property estimation device according to any one of <15> to <26>, further comprising a normalization step of normalizing the subject image based on a length from the eyebrows to the chin of the subject in the subject image. <28> The hair property estimation method according to any one of <15> to <27>, wherein the subject image includes the area from the top of the subject's head to the ends of the hair, and the hair property estimation step estimates an appearance of the hair ends as the property. <29> A hair property estimation program that causes a computer to execute the following steps: a subject image acquisition step of acquiring a subject image including the area from the top of the subject's head to at least the end of the chin, a contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image, and a hair property estimation step of estimating the property of the subject's hair based on the extracted contour line. <30> The hair property estimation program according to <29>, wherein the hair property estimation step estimates the property of the subject's hair based on at least one of the angle and length of the extracted contour line. <31> The hair property estimation program according to <29> or <30>, wherein the contour line extraction step further extracts feature amounts of the contour line, and the hair property estimation step estimates the property of the subject's hair using the extracted feature amounts. <32> The hair property estimation program according to <31>, wherein the feature amounts are extracted from a predetermined region of the contour line. <33> The hair property estimation program according to <31> or <32>, wherein the feature amount is at least one of the angle and length of the contour line. <34> The hair property estimation program according to any one of <29> to <33>, wherein in the hair property estimation step, the property of the subject's hair is estimated using a trained hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line and the property. <35> The hair property estimation program according to <34>, wherein the trained hair property estimation model is generated by having an artificial intelligence learn the relationship between features of the contour line and the property. <36> The hair property estimation program according to any one of <29> to <35>, wherein the property includes a degree of hair curl.<37> The hair property estimating program according to any one of <29> to <36>, wherein in the contour line extraction step, the contour line of the subject's hair is extracted using a learned contour line extraction model generated by having artificial intelligence learn the relationship between the subject's image and the contour line. <38> The hair property estimating program according to any one of <29> to <37>, further causing a computer to execute: a display control step of displaying the contour line extracted in the contour line extraction step superimposed on the subject's image; and an input receiving step of receiving input regarding the suitability of the superimposed displayed contour line; and wherein in the hair property estimating step, the hair property of the subject is estimated based on the received input. <39> The hair property estimation program according to any one of <29> to <38>, further causing a computer to execute: a hair care plan generation step of generating a hair care plan according to the hair properties of the subject based on a result of estimation in the hair property estimation step; and a presentation step of presenting the generated hair care plan to the subject. <40> The hair property estimation program according to <39>, further causing a computer to execute a question presentation / answer reception step of receiving an answer to a question of the subject, and generating the hair care plan from a combination of the answer and the hair properties in the hair care plan generation step. <41> The hair property estimation program according to any one of <29> to <40>, further causing a computer to execute a normalization step of normalizing the subject's image based on the length from the eyebrows to the chin of the subject in the subject image. <42> The hair property estimation program according to any one of <29> to <41>, wherein the subject image includes the subject's head from the top to the ends of the hair, and in the hair property estimation step, an appearance of the hair ends is estimated as the property.<43> A storage medium storing a hair property estimation program that causes a computer to execute the following steps: a subject image acquisition step of acquiring a subject image including the subject's head from the top to at least the tip of the chin; a contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image; and a hair property estimation step of estimating the property of the subject's hair based on the extracted contour line. <44> The storage medium described in <43>, in which the hair property estimation step estimates the property of the subject's hair based on at least one of the angle and length of the extracted contour line. <45> The storage medium described in <43> or <44>, in which the contour line extraction step further extracts feature amounts of the contour line, and in which the hair property estimation step estimates the property of the subject's hair using the extracted feature amounts. <46> The storage medium described in <45>, in which the feature amounts are extracted from a predetermined region of the contour line. <47> The storage medium according to <45> or <46>, wherein the feature is at least one of the angle and length of the contour line. <48> The storage medium according to any one of <43> to <47>, wherein, in the hair property estimation step, the property of the subject's hair is estimated using a trained hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line and the property. <49> The storage medium according to <48>, wherein the trained hair property estimation model is generated by having an artificial intelligence learn the relationship between the feature amount of the contour line and the property. <50> The storage medium according to any one of <43> to <49>, wherein the property includes the degree of hair curl. <51> The storage medium according to any one of <43> to <50>, wherein, in the contour line extraction step, the contour line of the subject's hair is extracted using a trained contour line extraction model generated by having an artificial intelligence learn the relationship between the subject's image and the contour line.<52> The storage medium of any one of <43> to <51>, further causing a computer to execute: a display control step of superimposing the contour extracted in the contour extraction step on the subject image and displaying it; and an input receiving step of receiving input regarding the suitability of the superimposed contour, wherein the hair property estimation step estimates the hair property of the subject based on the received input. <53> The storage medium of any one of <43> to <52>, further causing a computer to execute: a hair care plan generation step of generating a hair care plan according to the hair property of the subject based on the estimation result in the hair property estimation step; and a presentation step of presenting the generated hair care plan to the subject. <54> The storage medium of <53>, further causing a computer to execute a question presentation / answer reception step of receiving an answer to a question from the subject, wherein the hair care plan generation step generates the hair care plan from a combination of the answer and the hair property. <55> The storage medium according to any one of <43> to <54>, further causing a computer to execute a normalization step of normalizing the subject image based on a length from the eyebrows to the chin of the subject in the subject image. <56> The storage medium according to any one of <43> to <55>, wherein the subject image includes a portion from the top of the subject's head to the ends of the hair, and wherein the hair property estimating step estimates an appearance of the hair ends as the property. <57> A hair property estimating system including a hair property estimating device and a mobile terminal, comprising: a subject image acquisition unit that acquires a subject image including a portion from the top of the subject's head to at least the end of the chin, a contour line extraction unit that extracts a contour line of the subject's hair in the acquired subject image, and a hair property estimating unit that estimates the property of the subject's hair based on the extracted contour line. <58> The hair property estimating system according to <57>, wherein the hair property estimating unit estimates the property of the subject's hair based on at least one of an angle and a length of the extracted contour line.<59> The hair property estimation system according to <57> or <58>, wherein the contour line extraction unit further extracts feature amounts of the contour line, and the hair property estimation unit estimates the property of the subject's hair using the extracted feature amounts. <60> The hair property estimation system according to <59>, wherein the feature amounts are extracted from a predetermined region of the contour line. <61> The hair property estimation system according to <60>, wherein the feature amounts are at least one of the angle and length of the contour line. <62> The hair property estimation system according to any one of <57> to <61>, wherein the hair property estimation unit estimates the property of the subject's hair using a trained hair property estimation model generated by training an artificial intelligence on the relationship between the contour line and the property. <63> The hair property estimation system according to <62>, wherein the trained hair property estimation model is generated by training an artificial intelligence on the relationship between the feature amounts of the contour line and the property. <64> The hair property estimating method according to any one of <57> to <63>, wherein the property includes a degree of hair curl. <65> The hair property estimating system according to any one of <57> to <64>, wherein the contour line extraction unit extracts the contour line of the subject's hair using a learned contour line extraction model generated by having artificial intelligence learn the relationship between the subject's image and the contour line. <66> The hair property estimating system according to any one of <57> to <65>, comprising: a display control unit that displays the contour line extracted by the contour line extraction unit as a suitability confirmation image by superimposing it on the subject's image; and an input receiving unit that receives input regarding the suitability of the superimposed contour line, wherein the hair property estimating unit estimates the property of the subject's hair based on the received input. <67> The hair property estimating system according to any one of <57> to <66>, further comprising: a hair care plan generating unit that generates a hair care plan according to the hair properties of the subject based on the estimation result by the hair property estimating unit; and a presentation unit that presents the generated hair care plan to the subject.<68> The hair property estimating system according to <67>, further comprising a question presenting / answer receiving unit that receives an answer to a question from the subject, wherein the hair care plan generating unit generates the hair care plan from a combination of the answer and the hair property. <69> The hair property estimating system according to any one of <57> to <68>, further comprising a normalizing unit that normalizes the subject image based on a length from the eyebrows to the chin of the subject in the subject image. <70> The hair property estimating system according to any one of <57> to <69>, wherein the subject image includes the subject's head from the top to the ends of the hair, and the hair property estimating unit estimates the appearance of the hair ends as the property. <71> A mobile terminal comprising: a camera; a subject image acquisition unit that acquires an image of a subject captured by the camera; a contour line extraction unit that extracts a contour line of the subject's hair from the acquired image of the subject; a display control unit that displays the contour line extracted by the contour line extraction unit so as to be superimposed on the image of the subject; a display unit that displays a suitability check image; and a transmission unit that transmits the contour line of the hair extracted by the contour line extraction unit to a hair property estimation device. <72> A program for a mobile terminal used in a hair property estimation system, the program causing the mobile terminal to execute: a contour line extraction step that extracts a contour line of the hair from the image of the subject; and a transmission control step that transmits the contour line of the hair extracted by the contour line extraction unit to a hair property estimation device. <73> The program for a mobile terminal according to <72>, which causes a mobile terminal to execute the following steps: a display control step of superimposing the contour extracted by the contour extraction unit on the image of the subject and displaying it as a suitability confirmation image, an input receiving step of receiving input regarding the suitability of the superimposed and displayed contour, and a transmission control step of transmitting the contour to the hair condition estimating device in accordance with the input received in the input receiving step. <74> The program for a mobile terminal according to <73>, which is configured such that the display control step can display a plurality of superimposed images, and the input receiving step can select the contour to be transmitted from the plurality of superimposed images.

[0211] According to the present invention, hair properties can be estimated.

Claims

1. A hair property estimation device comprising: a subject image acquisition unit that acquires a subject image including the head of a subject; a contour line extraction unit that extracts a contour line of the hair of the subject from the acquired subject image; and a hair property estimation unit that estimates the property of the hair of the subject based on the extracted contour line.

2. The hair property estimation device according to claim 1, wherein the hair property estimation unit estimates the property of the hair of the subject based on at least one of the angle and length of the extracted contour line.

3. The hair property estimation device according to claim 1 or 2, wherein the contour line extraction unit further extracts a feature amount of the contour line, and the hair property estimation unit estimates the property of the hair of the subject using the extracted feature amount.

4. The hair property estimation device according to claim 3, wherein the feature amount is extracted from a predetermined region in the contour line.

5. The hair property estimation device according to claim 3, wherein the feature amount is at least one of the angle and length of the contour line.

6. The hair property estimation device according to claim 1, wherein the hair property estimation unit estimates the property of the hair of the subject using a learned hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line and the property.

7. The hair property estimation device according to claim 6, wherein the learned hair property estimation model is generated by having an artificial intelligence learn the relationship between the feature amount of the contour line and the property.

8. The hair property estimation device according to claim 1 or 2, wherein the property includes the degree of hair curl.

9. The hair property estimation device according to claim 1, wherein the contour line extraction unit extracts the contour line of the hair of the subject using a learned contour line extraction model generated by having an artificial intelligence learn the relationship between the subject image and the contour line.

10. The hair property estimation device according to claim 1, further comprising: a display control unit that superimposes and displays the contour line extracted by the contour line extraction unit on the subject image; and an input reception unit that receives an input regarding the suitability of the superimposed and displayed contour line, wherein the hair property estimation unit estimates the property of the hair of the subject based on the received input.

11. A hair care plan generation unit that generates a hair care plan according to the hair properties of the subject based on the estimation result by the hair property estimation unit, and a presentation unit that presents the generated hair care plan to the subject. The hair property estimation device according to claim 1, further comprising:

12. Further comprising a question presentation / answer reception unit that receives an answer to a question from the subject, and the hair care plan generation unit generates the hair care plan from the combination of the answer and the hair properties. The hair property estimation device according to claim 11.

13. The hair property estimation device according to claim 1, further comprising a normalization unit that normalizes the subject image based on the length from the subject's eyebrow line to the chin tip in the subject image.

14. A hair property estimation method including: a subject image acquisition step of acquiring a subject image including the head of the subject; a contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image; and a hair property estimation step of estimating the properties of the subject's hair based on the extracted contour line.

15. A hair property estimation program for causing a computer to execute: a subject image acquisition step of acquiring a subject image including the head of the subject; a contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image; and a hair property estimation step of estimating the properties of the subject's hair based on the extracted contour line.

16. A storage medium storing a hair property estimation program for causing a computer to execute: a subject image acquisition step of acquiring a subject image including the head of the subject; a contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image; and a hair property estimation step of estimating the properties of the subject's hair based on the extracted contour line.

17. A hair property estimation system including a hair property estimation device and a mobile terminal, the hair property estimation system comprising: a subject image acquisition unit that acquires a subject image including the head of the subject; a contour line extraction unit that extracts a contour line of the subject's hair in the acquired subject image; and a hair property estimation unit that estimates the properties of the subject's hair based on the extracted contour line.

18. A display control unit that overlays the contour line extracted by the contour line extraction unit on the subject image and displays it as an appropriateness confirmation image; and an input reception unit that receives an input regarding the appropriateness of the overlaid and displayed contour line. The hair property estimation system according to claim 17, wherein the hair property estimation unit estimates the properties of the subject's hair based on the received input.

19. A mobile terminal comprising: a camera; a subject image acquisition unit that acquires a subject image captured by the camera; a contour line extraction unit that extracts a contour line of the subject's hair from the acquired subject image; a display control unit that overlays the contour line extracted by the contour line extraction unit on the subject image and displays it; a display unit that displays an appropriateness confirmation image; and a transmission unit that transmits the contour line of the hair extracted by the contour line extraction unit to a hair property estimation device.

20. A program for a mobile terminal used in a hair property estimation system, the program causing the mobile terminal to execute: a contour line extraction step of extracting a contour line of hair from a subject image; and a transmission control step of transmitting the contour line of the hair extracted by the contour line extraction unit to a hair property estimation device.

21. The program for a mobile terminal according to claim 20, causing the mobile terminal to execute: a display control step of overlaying the contour line extracted by the contour line extraction unit on the subject image and displaying it as an appropriateness confirmation image; an input reception step of receiving an input regarding the appropriateness of the overlaid and displayed contour line; and a transmission control step of transmitting the contour line to the hair property estimation device according to the input in the input reception step.

22. The program for a mobile terminal according to claim 21, wherein in the display control step, a plurality of overlaid images can be displayed, and in the input reception step, the contour line to be transmitted can be selected from the plurality of overlaid images.

Images