Hair property estimation device, hair property estimation method, hair property estimation program, hair property estimation system, portable terminal, program for portable terminal and storage medium

Abstract

To provide a hair property estimation device for estimating hair properties.SOLUTION: A hair property estimation device comprises a subject image acquisition unit for acquiring a subject image including from a parietal region to at least a jaw tip of the subject, a contour line extraction unit for extracting a contour line of the subject's hair in the acquired subject image, and a hair property estimation unit for estimating properties of the subject's hair on the basis of the extracted contour line.SELECTED DRAWING: Figure 2A

Description

Technical Field

[0001] The present invention relates to a hair trait estimation device, a hair trait estimation method, a hair trait estimation program, a hair trait estimation system, a mobile terminal, a program for a mobile terminal, and a storage medium.

Background Art

[0002] In the above technical field, Patent Document 1 discloses that a person passing in front of an image display device is photographed by an image sensor, the person is extracted from the photographed image, the characteristics of the person (characteristics as a customer) are determined, and an advertisement corresponding to the determined characteristics is displayed on the image display device (paragraph

[0023] etc. of the same document). Patent Document 2 discloses grasping the facial contour and image brought about by a hairstyle formed by an inner line which is an inner boundary line between the face and the hair and an outer line which is an outer line of the hairstyle, and analyzing whether the hairstyle fits based on shape, balance, and image, and evaluating the hairstyle (paragraphs

[0014] to

[0029] , claim 1, etc. of the same document). Patent Document 3 discloses extracting feature amounts such as a hairstyle, hair length, and hair color from a photographed image of an identification target person, and determining whether the identification target person is a registered person based on the similarity between the extracted feature amounts and the registered feature amounts (paragraphs

[0011] to

[0012] , claim 1, etc. of the same document).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technologies described in Patent Documents 1 to 3 above, advertisement display corresponding to the characteristics of a person, hairstyle evaluation from the facial contour, and determination of the presence or absence of registration of the person to be identified from feature amounts such as hairstyle, hair length, and hair color were performed, but the hair properties could not be estimated.

Means for Solving the Problems

[0005] To achieve the above object, the hair property estimation device according to the present invention includes a subject image acquisition unit that acquires a subject image including the head of the subject, a contour line extraction unit that extracts the contour line of the hair of the subject in the acquired subject image, a hair property estimation unit that estimates the properties of the hair of the subject based on the extracted contour line, and is provided with.

[0006] Also, to achieve the above object, the hair property estimation method according to the present invention includes a subject image acquisition step of acquiring a subject image including at least from the top of the head to the tip of the chin of the subject, a contour line extraction step of extracting the contour line of the hair of the subject in the acquired subject image, a hair property estimation step of estimating the properties of the hair of the subject based on the extracted contour line, and includes.

[0007] Furthermore, to achieve the above object, the hair property estimation program according to the present invention causes a computer to execute a subject image acquisition step of acquiring a subject image including the head of the subject, a contour line extraction step of extracting the contour line of the hair of the subject in the acquired subject image, a hair property estimation step of estimating the properties of the hair of the subject based on the extracted contour line, and.

[0008] Furthermore, to achieve the above object, the storage medium according to the present invention A subject image acquisition step of acquiring a subject image including the head of a subject; In the acquired subject image, a contour line extraction step of extracting the contour line of the subject's hair; Based on the extracted contour line, a hair property estimation step of estimating the property of the subject's hair; A hair property estimation program for causing a computer to execute is stored.

[0009] Furthermore, in order to achieve the above object, a hair property estimation system according to the present invention is A hair property estimation system including a hair property estimation device and a mobile terminal, A subject image acquisition unit that acquires a subject image including the head of a subject; In the acquired subject image, a contour line extraction unit that extracts the contour line of the subject's hair; Based on the extracted contour line, a hair property estimation unit that estimates the property of the subject's hair; It is provided with.

[0010] Furthermore, in order to achieve the above object, a mobile terminal according to the present invention is A camera, A subject image acquisition unit that acquires a subject image photographed by the camera; In the acquired subject image, a contour line extraction unit that extracts the contour line of the subject's hair; A display control unit that superimposes and displays the contour line extracted by the contour line extraction unit on the subject image; A display unit that displays an appropriateness confirmation image; A transmission unit that transmits the contour line of the hair extracted by the contour line extraction unit to a hair property estimation device; It is provided with.

[0011] Furthermore, in order to achieve the above object, a program for a mobile terminal according to the present invention is A program for a mobile terminal used in a hair property estimation system, A contour line extraction step of extracting the contour line of hair from a subject image; A transmission control step of transmitting the outline of the hair extracted by the outline extraction unit to the hair property estimation device, is executed on the mobile terminal.

Advantages of the Invention

[0012] According to the present invention, the hair properties can be estimated.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 2E

Figure 2F

Figure 2G

Figure 2H

Figure 2I

Figure 2J

Figure 2K

Figure 2L

Figure 2M

Figure 2N

Figure 2O

Figure 2P

Figure 3

Figure 4

Figure 5A

Figure 5B

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17A

Figure 17B

Figure 18

Embodiments for Carrying Out the Invention

[0014] Hereinafter, embodiments for carrying out the present invention will be illustratively and specifically described with reference to the drawings. However, the configurations, numerical values, processing flows, functional elements, etc. described in the following embodiments are merely examples, and modifications and changes thereof are free, and are not intended to limit the technical scope of the present invention to the following description.

[0015] [First Embodiment] The hair property estimation device according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 5. FIG. 1 is a diagram for explaining an outline of the operation of the hair property estimation device 100 according to the present embodiment. The hair property estimation device 100 is a device that extracts the contour line 112 of the hair of the subject 111 from an image (subject image 110) including the head of the subject 111 (contour line image 113), and estimates the properties of the hair of the subject 111. Note that the hair property estimation device 100 of the present embodiment includes a server device, a PC (Personal Computer), a mobile terminal such as a smartphone or a tablet terminal. The same applies to other embodiments hereinafter.

[0016] The hair property estimation device 100 extracts the hair contour line 112 of the subject 111 from the subject image 110 in which, for example, from the top of the head of the subject 111 to the hair tips are shown. The extracted contour line 112 is displayed superimposed on the subject image 110, and a contour line image 113 representing the outer shape of the hair of the subject 111 is obtained. Then, the hair property estimation device 100 estimates from the shape of the contour line portion of the obtained contour line image 113 (such as the spreading state of the hair) which type (the first type 121 to the fourth type 124) in the property classification 120 the hair property of the subject 111 corresponds to.

[0017] The contour line 112 extracted in this embodiment can be superimposed on the two-dimensional subject image 110 as described above and can be represented as a line in the two-dimensional image. When representing the contour line 112 as a line in the two-dimensional image, in order to emphasize that it is the contour line 112, the contour line 112 may be displayed as a thick line or a colored line.

[0018] Here, the hair property is, for example, the degree of the curl of the hair of the subject 111, and the property classification 120 is classified into four types according to the degree of the curl. And the property classification 120 is the first type 121, the second type 122, the third type 123, and the fourth type 124 in the order of the strength of the hair curl. Note that the classification of the hair curl is not limited to four, and it may be two or three types, or five or more types.

[0019] The first type 121 is a silhouette that spreads to the A-line. For example, due to the overlapping of the curls, the volume gradually increases from the top of the head to the hair tip, and the cut silhouette directly becomes the A-line. The second type 122 is a silhouette that spreads from below the ears. For example, the spread from the top of the head to the ears is slightly reduced, but the curl greatly appears from around the height of the ears.

[0020] The third type 123 is a silhouette with a slightly rising and spreading root. For example, it is a curl that can be stretched if a dryer is used, with the root rising and air entering to make it slightly swollen. The fourth type 124 is a non-spreading silhouette. For example, the rising of the root is weak and the cut silhouette is not swollen. And the hair property estimation device 100 estimates which type the hair property of the subject 111 corresponds to.

[0021] Next, with reference to FIGS. 2A to 2Q, the configuration of the hair property estimation device 100 will be described. The hair property estimation device 100 includes a subject image acquisition unit 201, a contour line extraction unit 202, and a hair property estimation unit 203.

[0022] The subject image acquisition unit 201 acquires a subject image 110 including the head of the subject 111. The subject image 110 is, for example, an image captured by a mobile terminal such as a smartphone or a tablet terminal that the subject 111 has, or a digital camera. Then, the subject image acquisition unit 201 acquires the subject image 110 from a mobile terminal or the like via wired communication or wireless communication. Further, the subject image acquisition unit 201 may directly acquire the subject image 110 stored in a memory device attached to a mobile terminal or the like from the memory device. Furthermore, the subject image acquisition unit 201 may acquire the subject image 110 captured by a mobile terminal or the like and stored in a storage on the cloud from the storage on the cloud.

[0023] As described above, the subject image 110 includes the head of the subject 111. Preferably, the subject image 110 includes at least from the top of the head to the tip of the chin of the subject 111. Also, the subject image 110 may or may not include the tips of the hair of the subject 111.

[0024] In the example shown in FIG. 1, the subject image 110 is an image of the subject 111 captured from the front. However, the direction of capturing the subject 111 is not limited to this. For example, it may be an image captured from the back (rear view), an oblique direction, an oblique upward direction, or an oblique downward direction. From the viewpoint of grasping the relationship with the position of the face and its respective parts, an image captured from the front is preferred. Also, the subject image 110 may be an image taken by the subject 111 himself / herself, or an image captured by a camera or the like installed in a store such as a beauty salon or a cosmetics store. Note that when the subject image is an image of the subject 111 captured from a direction other than the front, it is not always necessary for the tip of the chin of the subject 111 to be shown in the subject image. For example, when the subject image is an image of the subject 111 captured from the back, it is sufficient if the subject image shows a portion corresponding to the region from the top of the head to the tip of the chin when the head of the subject 111 is viewed from the back side.

[0025] In addition, the subject image 110 may be an image in any state as long as the hair characteristics of the subject 111 can be understood. However, a preferable image is selected according to the hair characteristics to be estimated. For example, when estimating the degree of hair curl, an image showing the state of natural hair is preferable. For example, an image one hour after towel drying after washing or an image after natural drying after washing is preferable. On the other hand, when estimating the styling finish, an image immediately after styling is preferable.

[0026] The subject image acquisition unit 201 may acquire one or a plurality of subject images 110 for the same subject 111. Further, when the subject image acquisition unit 201 acquires a plurality of subject images 110, the images may include images in different shooting directions, or a plurality of images in the same shooting direction may be included. Note that the subject image 110 may be a still image or a moving image.

[0027] The contour extraction unit 202 extracts the contour line 112 of the hair of the subject 111 from the acquired subject image 110. The "contour line" referred to in the present invention is the outer edge of the silhouette and refers to the boundary between the hair and the surroundings, that is, other than the head. For example, as shown in FIG. 2B(a), for the extraction of the contour line 112, the learning data 230 is learned by artificial intelligence to generate a learned contour extraction model, and the generated learned contour extraction model is used to extract the contour line 112 of the hair of the subject 111. That is, the contour extraction unit 202 can learn the relationship between the subject image and the contour line 112 of the subject's hair by artificial intelligence, generate a learned contour extraction model, and use the generated learned contour extraction model to extract the contour line 112 of the hair of the subject 111. Note that artificial intelligence is a computer system having intelligent functions such as inference and judgment performed by humans, and is a system that performs deep learning and determines correlation formulas. However, the artificial intelligence used in the contour extraction unit 202 preferably uses deep learning.

[0028] Here, the learning data 230 is a set of a sample image 231 obtained by imaging a sample hair and a binarized image 232 obtained by binarizing the sample image 231 so that the hair region (hair part) is emphasized. Then, such learning data 230 is used to train an artificial intelligence through machine learning to obtain a learned contour extraction model. The sample image 231 and the binarized image 232 can be images of the hair taken from various angles, such as a front image, a back image, a side image, a diagonal front image, and a diagonal back image.

[0029] As described above, the contour line 112 output by the learned contour extraction model is the boundary between the hair region and the other parts, and is represented as a line in a two-dimensional image. When generating the learned contour extraction model, the binarized sample image 231 is used as the teacher data (binarized image 232) so that the boundary can be correctly recognized, that is, the hair part and the other parts can be distinguished.

[0030] Then, for such learning data 230, for example, by using semantic segmentation to perform deep learning on the learning data 230, a learned contour extraction model is generated. For deep learning, for example, FCN (Fully Convolutional Network), CNN (Convolutional Neural Network), RNN (Recurrent neural network), etc. can be used.

[0031] Here, semantic segmentation is an algorithm used to recognize a collection of pixels that form a characteristic label or category by associating a label or category with all pixels in an image. As a result, it becomes possible to reliably extract the contour (silhouette) of the hair even if the learning data 230 is, for example, an image from behind or from the front, or an image with a complicated background.

[0032] The contour line 112 extracted by the contour extraction unit 202 will be further described with reference to FIG. 2C. First, the contour extraction unit 202 extracts the hair region from the subject image 110, generates the image 233, and then extracts the pixels constituting the outer edge of the hair region, that is, the contour line 112, from the image 233 to generate the image 234. In this way, the contour line 112 can be represented as an image in which only the pixels constituting the contour line 112 are filled, and the other pixels are not filled. Note that the representation method of the contour line 112 is not limited to this method. For example, the pixels constituting the contour line 112 may be displayed in the form of coordinates 235, or any other well-known method for representing a line may be used. Note that FIG. 2C is for explaining the concept, and those skilled in the art can easily infer that the actual number of pixels used is much larger.

[0033] As described above, the method for extracting the contour line 112 of the hair has been described, but the method for extracting the contour line 112 is not limited to the method described above. For example, an infrared image may be used. As the infrared ray, near-infrared rays can be used. That is, between hair and skin, the reflectance with respect to near-infrared rays is different. For hair, the reflectance tends to increase as the wavelength becomes longer in the range of 850 to 940 [nm], and for skin, in the same wavelength range, the reflectance tends to decrease as the wavelength becomes longer. By using such characteristics of near-infrared rays, the subject 111 is irradiated with near-infrared rays in the wavelength range of 850 to 940 [nm], the reflectance at 850 [nm] is subtracted from the reflectance at 940 [nm], and a region where a positive value is obtained is extracted. Then, this region becomes the hair region. The outer edge of the hair region extracted in this way can be extracted as the contour line 112.

[0034] In addition to the method using the wavelength dependence of the reflectance of near-infrared light, for example, thermography may be used to extract the contour line 112 of the hair. That is, based on the differences in the temperature distributions of the face (skin), hair, and surrounding environment of the subject 111, the contour line 112 of the hair can be extracted. Since the temperature distributions of the face and the surrounding environment are more uniform than those in the hair region (peaks occur in the temperature bands of the face and the surrounding environment), this can be utilized. By performing threshold processing on the hair region band, extracting the region excluding the face and the surrounding environment, and further extracting the outer edge of the extracted region, the contour line 112 of the hair can be extracted.

[0035] Also, without using machine learning, the contour line 112 may be extracted by typical image processing techniques. For example, the region of the same color as the hair color may be extracted, and its outer contour may be extracted as the contour line 112, or the contour line 112 may be extracted from the difference between the hair color and the background color, etc.

[0036] Furthermore, the contour line 112 may be extracted by subjecting the subject image 110 to a filter process (edge process). The edge process is a process that emphasizes the portions (edges) where the brightness of the image changes rapidly, such as from black to white or from white to black, and there are processes that emphasize the vertical direction, the horizontal direction, and both the vertical and horizontal directions. For example, the hair region has many high-frequency components, and the skin region has many low-frequency components. Also, since the hair region has irregularities, the light reflection varies depending on the location, resulting in many high-frequency components, while the skin region has small irregularities, so the light reflection is almost constant regardless of the location, resulting in many low-frequency components.

[0037] Furthermore, the contour line 112 may be extracted by combining the information input by the subject 111 or other users with the image processing method. For example, using some interface to set the points and regions necessary for extracting the hair color, extracting the hair color with reference to the set points, extracting the region having the same hue as the extracted hair color, and further extracting the outer edge of the extracted region, the contour line 112 of the hair may be extracted.

[0038] Also, assuming that the hair color is black, brown, etc., the black or brown regions are extracted from the target person image 110, and the outer edge of the extracted region is further extracted, so that the hair contour line 112 may be extracted. Alternatively, a location where the color suddenly changes from a black or brown region to the background color may be searched and extracted as the hair region, and the outer edge of the extracted hair region may be extracted as the contour line 112.

[0039] Furthermore, facial feature points such as the eyes and mouth of the target person 111 are extracted by face recognition technology. For example, a pixel located several pixels above the apex of the forehead is assumed to be a hair pixel, and based on this pixel, if the color of an adjacent pixel is close to the reference pixel, it is determined to be a pixel in the hair region. Then, by repeating the above process until there are no more pixels determined to be in the hair region, the hair region can be extracted. And the outer edge of the extracted hair region may be extracted as the contour line 112.

[0040] Furthermore, the extraction of the contour line 112 may be performed by the target person 111 or other users tracing the part of the hair contour line 112 in the target person image 110 displayed on a display device such as a touch panel display using a finger or an appropriate input device, in addition to the method using the above-mentioned pre-trained model.

[0041] The hair property estimation unit 203 estimates the properties of the hair of the subject 111 based on the extracted contour line 112. More specifically, the hair property estimation unit 203 estimates the properties of the hair of the subject 111 using, for example, a learned hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line 112 and the properties. In this way, a combination of the extracted contour line 112 and the properties estimated from the contour line 112 is given to the artificial intelligence as learning data for learning. The learning data is given the type of hair property (first type 121 to fourth type 124), and the type of hair property is given by a hair expert such as a beautician or a barber. In the present embodiment, the hair properties are classified by the degree of hair curl, but the hair properties are not limited to this. The hair properties may be, for example, shape (the finish and looseness of the hairstyle, sleeping kinks, the degree of wave perm (curl)), ease of handling, volume (actual hair volume), hair quality (hardness / softness, smoothness, damage, etc.), appearance, that is, how it appears to a third party (curly hair feel, volume feel, slovenly feel, appearance of the hair tips, etc.).

[0042] As the hair properties, in addition to the degree of curl described above, the following properties (1) to (8) may be estimated. (1) As a hair trait, the wavy hair feel may be estimated. Here, the wavy hair feel refers to the "appearance like wavy hair" of the style in a scene other than the state of natural hair during the day after styling, which is different from the "degree of wave" or "strength of wave" judged from the state of natural hair after shampooing. Silhouettes that spread around the head (around from the top of the head to the ears) and around the face (around from the ears to the jaw) (such as the silhouettes of the first type 121 and the second type 122) have a strong wavy hair feel. By making a non-spreading silhouette (the fourth type 124), the wavy hair feel gradually decreases. The users for whom the wavy hair feel is to be estimated are, for example, Japanese women aged 10 to 70 (including hair straightening and perming). The conditions for the image used as the target person image 110 are preferably a scene other than the state of natural hair during the day after styling, but any image in a state where the hair traits of the target person 111 can be seen may be used. And the classification and evaluation of the wavy hair feel can be evaluated in four levels: 1 = strong wavy hair feel, 2 = slightly strong wavy hair feel, 3 = slightly weak wavy hair feel, 4 = weak wavy hair feel. In the data for machine learning, scores for the wavy hair feel at four levels are given in advance, and the scores are given by hair experts such as beauticians and barbers. In addition, as an example of the proposed case of the hair care plan described later, for example, it is a proposal for the timing of re-treatment for a person who has had hair straightening.

[0043] (2) As a hair trait, a slovenly feeling may be estimated. Here, the slovenly feeling refers to the "slovenly appearance" of the style in a scene other than the state of natural hair during the day after styling. For example, silhouettes that spread around the temples and face (such as the silhouettes of the first type 121 and the second type 122) have a strong slovenly feeling. By making the silhouette that does not spread (the fourth type 124), the slovenly feeling gradually decreases. The users for whom the slovenly feeling is to be estimated are Japanese women aged 10 to 70 (including hair straightening and perming). The condition of the image to be used as the target person image 110 is preferably a scene other than the state of natural hair during the day after styling, but any image can be used as long as the hair traits of the target person 111 can be seen. And the classification and evaluation of the slovenly feeling can be classified and evaluated in four levels: 1 = strong slovenly feeling, 2 = slightly strong slovenly feeling, 3 = slightly weak slovenly feeling, 4 = weak slovenly feeling. For the data for machine learning, scores at four levels are given in advance, and the scores are given by hair experts such as beauticians and barbers. Incidentally, as an example of a proposed hair care plan described later, there is a proposal to recommend a haircut at a beauty salon when the slovenly feeling increases.

[0044] (3) As a hair trait, the degree of wave perm (for whom) may be estimated. The degree of wave perm refers to the remaining condition of the wave shape of the hair after a certain period of time has passed since the wave perm treatment at a beauty salon. As time passes after perming, the hair grows unevenly, causing the curl phase to shift, and for example, the silhouette bulges like the second type 122 and the third type 123. Furthermore, when the perm is removed, the curl stretches and approaches the silhouette of the fourth type 124 (a non-spreading silhouette). The users for whom the degree of wave perm is to be estimated are Japanese women aged 10 to 70 who have had a wave / curl perm treatment. The condition for the image used as the target person image 110 is preferably an image in which the state of natural hair can be seen. For example, an image taken 1 hour after towel drying after washing or an image after natural drying after washing is preferred. However, any image in a state where the hair traits of the target person 111 can be seen may be used. The classification and evaluation of the degree of wave perm can be classified and evaluated in four levels: 1 = strong, 2 = slightly strong, 3 = slightly weak, 4 = weak. The data for machine learning is pre-assigned a score of four levels, and the score is assigned by hair experts such as beauticians and barbers. Note that the proposed case of the hair care plan described later is a proposal for a guideline for re-perming.

[0045] (4) As a hair trait, the styling finish may be estimated. The styling finish refers to the state immediately after styling the hair style, such as before going out in the morning. For example, when styling straight hair, a good finish results in a non-spreading silhouette similar to the fourth type 124. If the finish is bad, the silhouette around the top and face of the head spreads like the first type 121 to the third type 123. The users for whom the styling finish is to be estimated are Japanese women aged 10 to 70. The condition for the image used as the target person image 110 is preferably an image immediately after styling. The classification and evaluation of the styling finish can be classified and evaluated in four levels: 1 = good, 2 = slightly good, 3 = slightly bad, 4 = bad. The data for machine learning is pre-assigned a score of four levels, and the score is assigned by hair experts such as beauticians and barbers.

[0046] (5) As a hair property, the degree of loosening from the styling finish may be estimated. The degree of loosening from the styling finish refers to the degree of change in the style after performing daily activities from the styled hair style before going out in the morning or the like. For example, if there is little change from the silhouette of the finish in (4) above, the style has not loosened, and if the change is large and approaches a silhouette where the periphery of the head or the face spreads like the first type 121 to the third type 123, it is evaluated that the style has loosened. The users for whom the degree of loosening from the styling finish is to be estimated are Japanese women aged 10 to 70. As for the conditions of the image used as the subject image 110, an image after 8 hours or more of performing daily activities after the styling finish is preferable, but any image taken after the styling may be used. The classification and evaluation of the degree of loosening from the styling finish can be classified and evaluated in four levels: 1 = not loosened, 2 = slightly not loosened, 3 = slightly loosened, 4 = loosened. In the data for machine learning, scores for the four levels are given in advance, and the scores are given by hair experts such as beauticians and barbers.

[0047] (6) As a hair property, the appearance of the hair tip may be estimated. The appearance of the hair tip refers to the appearance of the shape of the hair tip part of the silhouette. If the hair tip of the silhouette spreads or is uneven on the left and right, it will have a bad appearance, and if the hair tip of the silhouette does not spread and is uniform on the left and right, it will have a good appearance. The users for whom the appearance of the hair tip is to be estimated are Japanese women aged 10 to 70 (including hair straightening and perming). As for the conditions of the image used as the subject image 110, an image after styling is preferable, but any image in a state where the hair properties of the subject 111 can be seen may be used. When estimating the appearance of the hair tip, as the subject image 110, one including from the top of the subject 111's head to the hair tip is used. The classification and evaluation of the appearance of the hair tip are classified and evaluated in four levels: 1 = good, 2 = slightly good, 3 = slightly bad, 4 = bad. In the data for machine learning, scores for the four levels are given in advance, and the scores are given by hair experts such as beauticians and barbers.

[0048] (7) As a hair property, the volume (actual hair quantity) may be estimated. The volume refers to the actual hair quantity. Even with the same volume (hair quantity), the volume feeling can vary depending on styling or the shape of the natural curl. In the case of curly hair, when straightened with a straightening iron or the like, if the bulge of the silhouette at the top of the head is large, the hair quantity is large, and if the bulge is small, the hair quantity is small. The users for whom the volume is to be estimated are Japanese women aged 10 to 70 (including hair straightening and perming). As for the conditions of the image used as the target person image 110, in the case of straight hair, an image where the state of natural hair can be seen is preferable, and in the case of curly hair, an image when straightened with a straightening iron or the like is preferable. The classification and evaluation of the volume are classified and evaluated in four levels: 1 = a lot, 2 = somewhat a lot, 3 = somewhat little, 4 = little. Alternatively, instead of sensory evaluation, the actual hair density at the top of the head may be measured for classification and evaluation. For the data for machine learning, four-level scores are pre-assigned, and the scores are assigned by hair experts such as beauticians and barbers.

[0049] (8) As a hair property, the volume feeling may be estimated. The volume feeling refers not to the actual hair quantity but to the "voluminous appearance" of the style in a scene other than the state of natural hair such as during the day after styling. If the bulge of the silhouette at the top of the head is large, the volume feeling is large, and if the bulge is small, the volume feeling is small. The users for whom the volume feeling is to be estimated are Japanese women aged 10 to 70 (including hair straightening and perming). As for the conditions of the image used as the target person image 110, an image after styling is preferable, but any image in a state where the hair property of the target person 111 can be seen is acceptable. The classification and evaluation of the volume feeling are classified and evaluated in four levels: 1 = present, 2 = somewhat present, 3 = somewhat absent, 4 = absent. For the data for machine learning, four-level scores are pre-assigned, and the scores are assigned by hair experts such as beauticians and barbers.

[0050] The type of hair trait assigned to the data for machine learning, i.e., the teacher data, may be other than the type of hair trait assigned by the above-mentioned hair experts. For example, in addition to the type classification of hair traits (hair curls) by the above-mentioned hair experts, the type classification of hair traits may be performed by the following method. For each user, from the viewpoints of ease of acquisition and data accuracy, preferably, 1 to 1000 hairs, more preferably 3 to 500 hairs, and even more preferably 5 to 250 hairs are measured, and the average value thereof is taken as the degree of hair curl of the user. Note that when measuring the hair, it is not necessary to pluck or cut the hair from the user. And specific measurement methods for the degree of curl can include, for example, the following methods (I) to (VII).

[0051] (I) Read the image of the hair with an image scanner, obtain the radius of curvature of the measurement points at predetermined intervals of each read hair image with image processing software, and take the average value thereof as the degree of curl. In this case, an upper limit may be set for the measured radius of curvature. For example, a radius of curvature of 20 cm or more can be set to 20 cm. Also, the number of measurement points per hair is preferably 3 or more.

[0052] (II) Place a transparent film with concentric circles (radius 1 mm to 200 mm) having different radii by a predetermined length (e.g., 1 mm) on the hair, and read the radius of curvature of the hair at a plurality of sites at a predetermined interval (e.g., 4 sites at 1 cm intervals) from the root of the hair, and obtain the reciprocal (curvature). However, when the curvature is less than 0.05 (unit: cm -1 ), i.e., when the radius of curvature is greater than 20 cm), the curvature can be approximated to 0.05. Obtain the average value of the curvatures at the above-mentioned plurality of sites, and take the reciprocal thereof as the radius of curvature of the hair.

[0053] (III) Read the image of the hair with an image scanner. On the other hand, accumulate a plurality of reference curves with known average radii of curvature in the image processing software, compare each read hair image with the reference curves, and take the average radius of curvature of the reference curve that best matches both as the degree of curl.

[0054] (IV) Hold one end of the hair and suspend it, read its shape with a 3D scanner, obtain the radius of curvature of the measurement points at predetermined intervals of the read hair image with image processing software, and take the average value thereof as the degree of the wave.

[0055] (V) Measure the natural length (LO) of the hair and the length (L) obtained by stretching the hair itself without stretching it and making the undulation straight, calculate (L - LO) / L, and take this as the degree of the wave. For example, for the measurement of the length, first fix one end of the hair, suspend the other end freely, and measure the distance (natural length LO) between the fixed part and the other end. Next, methods such as stretching the hair straight by hand without stretching the hair itself and measuring the length (L) at both ends, or hanging a weight (for example, 0.1 g to 0.01 g) that can be ignored as a stress at one end and measuring the length (L) from the fixed part to the weight are used. When measuring LO and L, it doesn't matter which end is the root of the hair.

[0056] (VI) Hold one end of the hair and suspend it, take a picture of its shape with a digital camera, measure how much the hair spreads left and right from the straight line connecting one end and the other end at the measurement points at predetermined intervals on the straight line, and take the average value thereof as the degree of the wave.

[0057] (VII) Place the hair on a smooth metal plate on the surface (for anti-static purposes), measure the spread, that is, as the angle from one end, as shown in FIG. 2B(b), measure the magnitude of the apex angle of the triangle sandwiching the hair 250, and take the average value thereof as the degree of the wave. The angle can be extracted by image processing.

[0058] The hair property estimation unit 203 preferably estimates the properties of the hair using the feature amounts of the extracted contour line 112. Specifically, in order to improve the accuracy of hair property estimation, for example, as shown in FIG. 2D, it is preferable to use the combination of the feature amounts of the contour line 112 and the hair properties as learning data and give it to artificial intelligence for learning. In FIG. 2D, the case where the angle of the contour line extracted from each predetermined region is used as the feature amount is given as an example. The angle of the contour line extracted from each region can be, for example, the average value of the angles formed by the line extending straight downward from the top of the head toward the chin tip side in each region, for example, a line parallel to the midline of the face and the contour line 112. For example, when the self-taken image is the subject image 110, the subject image 110 may be an image in which the midline of the face of the subject 111 is inclined with respect to the vertical direction of the subject image 110. Even in such a case, by setting the average value of the angles formed by the line extending straight downward from the top of the head toward the chin tip side and the contour line 112 as the angle of the contour line 112, the angle of the contour line 112 can be accurately derived. The feature amount is not limited to the angle of the contour line extracted from each predetermined region, and for example, the length of the contour line extracted from each predetermined region can be used.

[0059] The extraction of the feature amount may be performed in the contour line extraction unit 202 or in the hair property estimation unit 203. Here, however, it will be described assuming that the extraction of the feature amount is performed in the contour line extraction unit 202. Therefore, the hair property estimation unit 203 acquires data regarding the extracted feature amount from the contour line extraction unit 202. Note that the hair property estimation device 100 may further include a contour feature amount extraction unit that extracts the feature amount.

[0060] The contour line extraction unit 202, for example, after generating the contour line image 240, divides it into several regions (S1 to S6), extracts the feature amount for each region, and causes artificial intelligence to learn the relationship between each extracted feature amount and the type of the given hair property as learning data.

[0061] Here, the regions are divided, for example, as shown in FIG. 2D, into S1 = the region from the top-of-head line to the eyebrow line, S2 = the region from the eyebrow line to the eye line, S3 = the region from the eye line to the upper lip line, S4 = the region from the upper lip line to the jaw line, S5 = the region from the eyebrow line to the upper lip line, and S6 = the region from the eyebrow line to the jaw line. In this way, the subject image 110 can be divided into a plurality of regions based on parts of the face of the subject 111, such as the eyes, upper lip, eyebrows, jaw, etc. In addition to the criteria shown here, for example, the nose may be included as one of the criteria. Note that the "jaw" as used here refers to the lowermost end of the jaw, that is, the chin tip, as can be seen from FIG. 2D. Also, the line refers to a straight line drawn parallel to the horizontal direction, that is, a line connecting the pupils of both eyes or the vertices of both eyebrows. That is, the upper lip line refers to a horizontal straight line passing through the upper lip, and the jaw line refers to a horizontal straight line passing through the lower tip of the jaw.

[0062] In estimating the hair characteristics, it is preferable to use at least the outline of the head, that is, the range between the top of the head and the lower end of the jaw, and more specifically, the range from the top of the head to the chin tip. That is, a preferable subject image is one that includes the range from the top of the subject's head to at least the chin tip. That is, it is preferable to obtain a subject image including the range from the top of the subject's head to at least the chin tip, extract the outline of the hair, and use the outline for estimation.

[0063] Examples of the feature amount of the hair outline 112 include at least one of the angle and length of the outline 112.

[0064] The outline extraction unit 202 calculates, for example, the angle of the hair outline 112 (the outer edge of the silhouette) in each region, and uses the calculated angle as the feature amount of the hair outline 112. Here, the region for extracting the feature amount is divided into six, but the method of dividing the region is not limited to the method shown here, and the region may be divided more finely.

[0065] Next, with reference to FIG. 2E, the calculation of the angle of each region will be described in detail. As shown in the figure, the contour line image 240 is divided into left and right parts. For each region, with the distance from the image edge [pixels (px)] on the vertical axis and the height [px] on the horizontal axis, the angles of the left contour line 112 and the right contour line 112 are plotted. Here, the left and right here represent the left and right of the subject 111. In the illustrated contour line image 240, on the drawing, the left side corresponds to the right side of the subject 111, and the right side corresponds to the left side of the subject 111. Plot 241 represents the right contour line 112, and plot 242 represents the left contour line 112.

[0066] Then, the contour line 112 is fitted with a linear function (graph 243), and when the slope of the linear function is converted into an angle, the angle = arctan(slope). The stronger the degree of the hair curl, the larger the angle, and the weaker the degree of the hair curl, the smaller the angle. For the section of region S1, the contour line 112 may be fitted with a quadratic function, and the quadratic coefficient may be used as a feature amount. Also, for each region, the angle difference between the left and right silhouettes may be used as a feature amount. Here, the angle difference between the left and right silhouettes is the absolute value of the difference between the angle of the right contour line 112 and the angle of the left contour line 112. The angle of the contour line 112 can be the angle of the contour line 112 with respect to the line drawn straight down from the top of the head toward the chin tip as described above.

[0067] In addition, the contour extraction unit 202 may calculate the length of the hair contour line 112 in each region, and use the calculated length as a feature amount of the hair contour line 112. For example, after changing the image size so that the face size becomes a certain size, the number of pixels of the contour line 112 (outer edge of the silhouette) in each region is counted, and the counted number of pixels is divided by the height of each region (each section) may be used as a feature amount. In this case, the stronger the curl, the more the contour line 112 (outer edge of the silhouette) bends, so the number of pixels increases, and the weaker the curl, the closer the silhouette is to vertical and the number of pixels tends to decrease. The method of changing the image size so that the face size becomes a certain size may be any method, but for example, the positions of the forehead and the chin are detected and the lengths are made to match. As can be easily understood from the above description, as the feature amount, in addition to the respective left and right angles, the average of the absolute values of the left and right angles, and the left and right angle differences, the inclination may also be used.

[0068] Also, after normalizing the image size so that the face size becomes a certain size, the contour line 112 may be extracted. Examples of the normalization method include a method of normalizing the subject image 110 based on the length from the eyebrow line of the subject 111 to the chin tip in the subject image 110. Then, the contour line 112 extracted from the normalized subject image 110 is divided into a plurality of regions as described above, for example, the number of pixels of the contour line 112 in each region is counted, and the counted number of pixels may be used as the feature amount as it is. This is because the number of pixels counted in this way is the length of the normalized contour line 112 in each region.

[0069] Furthermore, as another feature amount, for example, stray hairs not included in the contour may be detected, and the number of detected stray hairs may be used. As the method for detecting stray hairs, typical image processing methods or deep learning may be used. Then, for example, it is possible to evaluate that the stronger the degree of hair curl, the larger the number of stray hairs.

[0070] The "inside of the contour" mentioned above means the area surrounded by the contour line 112, not the contour line 112 itself. That is, stray hairs not included in the inside of the contour refer to stray hairs protruding from the area surrounded by the contour line 112.

[0071] The hair property estimation unit 203 gives the feature amounts of the extracted contour line 112, such as the relationship between the angle and the hair properties, to artificial intelligence for machine learning to generate a learned hair property estimation model. For machine learning by artificial intelligence, for example, a support vector machine (SVM), k-nearest neighbor method, random forest, etc. can be used, but it is not limited to these. That is, known methods such as deep learning and regression analysis can be arbitrarily used.

[0072] By using the learned hair property estimation model that has learned the relationship between the feature amounts of the contour line 112 and the hair properties, the hair properties can be estimated with a small amount of calculation.

[0073] Also, without extracting the feature amounts, the hair properties can be directly estimated from the contour line image 240. That is, by using the learned hair property estimation model obtained by making artificial intelligence perform machine learning with the combination of the contour line image 240 and the hair properties as teacher data, the hair properties can be estimated from the contour line image.

[0074] As the learned hair property estimation model that has learned the relationship between the contour line image 240 and the hair properties, those generated by deep learning are preferable. By using the learned hair property estimation model that has learned the relationship between the contour line image 240 and the hair properties, the amount of calculation can be reduced compared to the case of estimating the hair properties from the entire hair image.

[0075] Here, the support vector machine is one of the pattern recognition models using supervised learning, and it is an algorithm for class classification and regression by determining a boundary line or hyperplane that divides data groups of two classes, for example. The k-nearest neighbor method is an algorithm used for data classification. When there is a certain unknown data, it is an algorithm that determines the classification of the unknown data from the classes of the surrounding learning data. The k in the k-nearest neighbor method indicates the number of classes of the learning data existing near the unknown data. The random forest is an algorithm that combines two techniques, decision trees and ensemble learning (bagging), and it is an algorithm for class classification and regression.

[0076] Regarding the method for generating a learned hair trait estimation model, taking the case of using a support vector machine as an example, it will be described with reference to FIGS. 2F to 2H.

[0077] First, the feature amount is converted into principal component scores by principal component analysis (PCA: Principal Component Analysis). Specifically, the axis PC1 with the maximum variance is calculated. Then, the data is projected from the feature amount space 270 to the principal component space 271. Note that "PC2" in FIG. 2F is the axis with the second largest variance.

[0078] Next, a hyperplane 272 for classifying data in the principal component space 271 is derived by the support vector machine. Specifically, the expression (1) representing the hyperplane 272 is calculated. The weights w and the bias b in the expression (1) can be calculated by solving the expression (2). In the expression (2), the slack variable is a variable that gives a penalty to misclassified data, and the regularization parameter is a parameter that allows less misclassification as its value is larger and allows more misclassification as its value is smaller. When it is possible to classify multiple types, hyperplanes that can classify each type are calculated respectively. By deriving a hyperplane capable of classifying data in this way, a learned hair trait estimation model is generated.

[0079] Then, using the generated learned hair trait estimation model, the traits of the hair of subject 111 can be estimated. In addition to the method of estimating hair traits using the learned model, as shown in Fig. 2G, the hair traits may be estimated using a correlation relationship.

[0080] For example, as shown in Fig. 2G, when the contour line 112 of the hair is divided into a plurality of regions and the angles of each region are calculated, it can be seen that there is a correlation for some regions. Note that for the vertical axis, region S1 takes the coefficient of the quadratic function, and regions S2 to S6 take the angles. Also, for the horizontal axis, in any of regions S1 to S6, the degree (type) of the wave is taken, and the first type 121 = 1, the second type 122 = 2, the third type 123 = 3, and the fourth type 124 = 4 are represented. And the R 2 (coefficient of determination) is observed. In regions S3, S5, and S6 close to the center of the face, R 2 is close to 1.0, indicating a high correlation between the angle and the strength of the wave. On the contrary, in regions S1, S2, and S4 far from the center of the face, R 2 is close to 0, indicating a low correlation. Therefore, by using such a correlation relationship, the hair traits of subject 111 may be estimated.

[0081] <First verification> Here, regarding the accuracy of the estimation of hair traits by the above-mentioned machine learning, for example, the verification was performed as follows. Since the number of data is small, the estimation accuracy was verified by leave-one-out cross-validation. In this verification method, one piece of data is used as test data, and the remaining data is divided into learning data, and it is verified whether the test data can be estimated. This is repeated until all the data becomes test data. The number of images used for the verification was 11 images of the first type 121, 10 images of the second type 122, 11 images of the third type 123, and 11 images of the fourth type 124, and the degree of the wave was estimated. Then, when the angle of the silhouette of the images used for the verification was estimated by a support vector machine (SVM) using it as a feature amount, the estimation accuracy of the degree of the wave was 72% (31 / 43).

[0082] <Second Verification> To further improve the estimation accuracy of the habit level, the second verification was conducted by increasing the number of test data and the types of feature quantities used for the verification. In the second verification, the number of images was set as follows: 30 images of the first type 121, 35 images of the second type 122, 39 images of the third type 123, and 28 images of the fourth type 124. For the feature quantities, both the angle and length of the contour line 112 were used. Specifically, as the angles, a total of 8 types were used, including the angles of regions S1 to S6 respectively, the angle difference between the angles of regions S2 and S3, and the angle difference between the angles of regions S3 and S4. Also, as the lengths, a total of 8 types were used, including the lengths of regions S1 to S6 respectively, the difference between the lengths of regions S2 and S3, and the difference between the lengths of regions S3 and S4.

[0083] Fig. 2H shows the correlation between the angle and the habit level in the data used for the second verification. Regarding the vertical axis, for region S1, the coefficient of the quadratic function is taken, and for regions S2 to S6, the angles are taken. The vertical axis of "S2 - S3" takes the angle difference between the angles of regions S2 and S3, and the vertical axis of "S3 - S4" takes the angle difference between the angles of regions S3 and S4. Also, regarding the horizontal axis, for all of regions S1 to S6, "S2 - S3", and "S3 - S4", the habit level (type) is taken, and it is represented as the first type 121 = 1, the second type 122 = 2, the third type 123 = 3, and the fourth type 124 = 4. Looking at the R2 (coefficient of determination) of each region, it can be seen that in regions S3 and S5, R2 is relatively large, indicating a high correlation between the angle and the strength of the habit. On the contrary, in regions S1, S2, S4, S6, "S2 - S3", and "S3 - S4", R2 is relatively small, indicating a low correlation. Thus, it can be seen that there is also a correlation between the angle and the habit level in the data used for the second verification.

[0084] Also, the correlation between the length and the degree of the habit in the data used for the second verification is shown in FIG. 2I. Regarding the vertical axis, for regions S1 to S6, the length of the contour line 112 is taken. For the vertical axis of "S2 - S3", the difference in the length of the contour line 112 between region S2 and region S3 is taken, and for the vertical axis of "S3 - S4", the difference in the length of the contour line 112 between region S3 and region S4 is taken. Regarding the horizontal axis, for any of regions S1 to S6, "S2 - S3", and "S3 - S4", the degree of the habit (type) is taken, and it is represented as the first type 121 = 1, the second type 122 = 2, the third type 123 = 3, and the fourth type 124 = 4. Looking at the R2 (coefficient of determination) of each region, it can be seen that in regions S3, S5, and S6, R2 is relatively large, indicating a high correlation between the angle and the strength of the habit. On the contrary, in regions S1, S2, S4, "S2 - S3", and "S3 - S4", R2 is relatively small, indicating a low correlation. Thus, it can be seen that there is also a correlation between the length and the degree of the habit.

[0085] Using a support vector machine trained with the data used for the second verification, the degree of the habit was estimated for all 20 subject images 110. The breakdown of the 20 subject images 110 is 5 of the first type 121, 5 of the second type 122, 6 of the third type 123, and 4 of the fourth type 124. As features, both the angle and the length were used. Specifically, a total of 16 types of features, namely the 8 types of angles and 8 types of lengths described above, were extracted. Then, the 16 types of features were converted into 7 principal component scores, and using the 7 principal component scores, the degree of the habit was estimated by a support vector machine. The result is shown in FIG. 2J. Regarding the vertical axis, the actual degree of the habit (type) is taken, and regarding the horizontal axis, the degree of the habit (type) estimated by the support vector machine is taken. Both the vertical axis and the horizontal axis are represented as the first type 121 = 1, the second type 122 = 2, the third type 123 = 3, and the fourth type 124 = 4.

[0086] As shown in FIG. 2J, among the total of 20 subject images 110, 16 images had the estimated type match the actual type, and the estimation accuracy was 80%. Specifically, for the images of the first type 121, out of the total 5 images, 4 were estimated to be of the first type 121 and 1 was estimated to be of the second type 122. For the images of the second type 122, all 5 images were estimated to be of the second type 122. For the images of the third type 123, out of the total 6 images, 3 were estimated to be of the third type 123 and 3 were estimated to be of the fourth type 124. For the images of the fourth type 124, all 4 images were estimated to be of the fourth type 124.

[0087] In the estimation of the degree of habit using a support vector machine, in order to verify which of the angle and the length is useful as a feature amount, verification was performed for the case where the above-described eight types of angles were extracted as feature amounts and the case where the above-described eight types of lengths were extracted as feature amounts. The results are shown in FIG. 2K.

[0088] When only the angle was used as the feature amount, as shown in the graph 273 of FIG. 2K, among the total 20 subject images 110, 12 images had the estimated type match the actual type, and the estimation accuracy was 60%. Specifically, for the images of the first type 121, out of the total 5 images, 3 were estimated to be of the first type 121, 1 was estimated to be of the second type 122, and 1 was estimated to be of the third type 123. For the images of the second type 122, out of the total 5 images, 4 were estimated to be of the second type 122 and 1 was estimated to be of the third type 123. For the images of the third type 123, out of the total 6 images, 3 were estimated to be of the third type 123 and 3 were estimated to be of the fourth type 124. For the images of the fourth type 124, out of the total 4 images, 2 were estimated to be of the third type 123 and 2 were estimated to be of the fourth type 124.

[0089] When only the length is used as the feature amount, as shown in the graph 274 of FIG. 2K, among all 20 target person images 110, 15 images had the estimated type match the actual type, and the estimation accuracy was 75%. Specifically, for the images of the first type 121, all 5 images were estimated to be of the first type 121. For the images of the second type 122, out of all 5 images, 4 were estimated to be of the second type 122 and 1 was estimated to be of the third type 123. For the images of the third type 123, out of all 6 images, 2 were estimated to be of the third type 123 and 4 were estimated to be of the fourth type 124. For the images of the fourth type 124, all 4 images were estimated to be of the fourth type 124.

[0090] Thus, when only the length is used as the feature amount, the estimation accuracy is higher than when only the angle is used as the feature amount. As reasons for this, the following reasons can be considered. In the degree of the habit, both the spread of the hair silhouette and the undulation are involved. The angle becomes large when the spread of the silhouette becomes large. On the other hand, the length becomes large when both the spread of the silhouette becomes large and when the undulation becomes large. Therefore, the length in which both the spread of the silhouette and the undulation are involved is considered to be more useful for estimating the habit than the angle in which only the spread is involved.

[0091] Also, as shown in FIGS. 2J and 2K, when both the angle and the contour line are used, the estimation accuracy is higher than in either the case of using only the angle or the case of using only the length. Combining the length in which both the spread of the silhouette and the undulation are involved with the angle in which only the spread is involved is considered to be even more useful for estimating the habit.

[0092] Furthermore, in the estimation of the degree of habit using a support vector machine, in order to verify which region features in the subject image 110 are useful, the verification was performed using only the feature amounts of each region. Specifically, the feature amounts of regions S1 to S6, the regions around the ears (regions S2, S3, S5), the regions below the eyebrows (regions S2, S3, S4, S5, S6), and the regions above the upper lip (regions S1, S2, S3, S5) were used to estimate the habit. The results are shown in FIGS. 2L to 2M.

[0093] When using the feature amount of region S1, as shown in graph 281 of FIG. 2L, out of the total 20 subject images 110, 4 had the estimated type match the actual type, and the estimation accuracy was 20%. Specifically, for the images of the first type 121, out of the total 5 images, 2 were estimated to be of the first type 121, 1 was of the second type 122, and 2 were of the third type 123. For the images of the second type 122, out of the total 5 images, 3 were estimated to be of the first type 121 and 2 were of the third type 123. For the images of the third type 123, out of the total 6 images, 1 was of the first type 121, 2 were of the third type 123, and 3 were of the fourth type 124. For the images of the fourth type 124, out of the total 4 images, 1 was of the first type 121 and 3 were of the third type 123.

[0094] When using the feature amount of region S2, as shown in graph 282 of FIG. 2L, out of the total 20 subject images 110, 10 had the estimated type match the actual type, and the estimation accuracy was 50%. Specifically, for the images of the first type 121, out of the total 5 images, 2 were of the first type 121, 2 were of the second type 122, and 1 was of the fourth type 124. For the images of the second type 122, out of the total 5 images, 4 were of the second type 122 and 1 was of the fourth type 124. For the images of the third type 123, out of the total 6 images, 1 was of the second type 122 and 5 were of the fourth type 124. For the images of the fourth type 124, all 4 images were estimated to be of the fourth type 124.

[0095] When using the feature amount of region S3, as shown in graph 283 of FIG. 2L, among all 20 subject images 110, 12 images had the estimated type match the actual type, and the estimation accuracy was 60%. Specifically, for the images of the first type 121, out of all 5 images, 4 were estimated to be of the first type 121 and 1 was estimated to be of the second type 122. For the images of the second type 122, out of all 5 images, 3 were estimated to be of the second type 122 and 2 were estimated to be of the third type 123. For the images of the third type 123, out of all 6 images, 1 was estimated to be of the second type 122, 1 was estimated to be of the third type 123, and 4 were estimated to be of the fourth type 124. For the images of the fourth type 124, all 4 images were estimated to be of the fourth type 124.

[0096] When using the feature amount of region S4, as shown in graph 284 of FIG. 2M, among all 20 subject images 110, 11 images had the estimated type match the actual type, and the estimation accuracy was 55%. Specifically, for the images of the first type 121, out of all 5 images, 3 were estimated to be of the first type 121, 1 was estimated to be of the second type 122, and 1 was estimated to be of the fourth type 124. For the images of the second type 122, out of all 5 images, 1 was estimated to be of the second type 122 and 4 were estimated to be of the third type 123. For the images of the third type 123, out of all 6 images, 3 were estimated to be of the third type 123 and 3 were estimated to be of the fourth type 124. For the images of the fourth type 124, all 4 images were estimated to be of the fourth type 124.

[0097] When using the feature amount of region S5, as shown in graph 285 of FIG. 2M, among all 20 subject images 110, 13 images had the estimated type match the actual type, and the estimation accuracy was 65%. Specifically, for the images of type 121, out of all 5 images, 4 were estimated to be of type 121 and 1 was estimated to be of type 123. For the images of type 122, out of all 5 images, 4 were estimated to be of type 122 and 1 was estimated to be of type 123. For the images of type 123, out of all 6 images, 1 was of type 122, 1 was of type 123, and 4 were estimated to be of type 124. For the images of type 124, all 4 images were estimated to be of type 124.

[0098] When using the feature amount of region S6, as shown in graph 286 of FIG. 2M, among all 20 subject images 110, 15 images had the estimated type match the actual type, and the estimation accuracy was 75%. Specifically, for the images of type 121, all 5 images were estimated to be of type 121. For the images of type 122, out of all 5 images, 1 was of type 122 and 4 were estimated to be of type 122. For the images of type 123, out of all 6 images, 2 were of type 123 and 4 were estimated to be of type 124. For the images of type 124, all 4 images were estimated to be of type 124.

[0099] When using the feature amounts of the regions around the ears (regions S2, S3, S5), as shown in graph 287 of FIG. 2N, out of all 20 subject images 110, 12 had the estimated type match the actual type, and the estimation accuracy was 60%. Specifically, for the images of the first type 121, out of all 5, 4 were estimated to be of the first type 121 and 1 was estimated to be of the second type 122. For the images of the second type 122, out of all 5, 3 were of the second type 122 and 2 were estimated to be of the third type 123. For the images of the third type 123, out of all 6, 1 was of the second type 122, 1 was of the third type 123, and 4 were estimated to be of the fourth type 124. For the images of the fourth type 124, all 4 were estimated to be of the fourth type 124.

[0100] When using the feature amounts of the regions below the eyebrows (regions S2, S3, S4, S5, S6), as shown in graph 288 of FIG. 2N, out of all 20 subject images 110, 15 had the estimated type match the actual type, and the estimation accuracy was 75%. Specifically, for the images of the first type 121, out of all 5, 4 were of the first type 121 and 1 was estimated to be of the second type 122. For the images of the second type 122, all 5 were estimated to be of the second type 122. For the images of the third type 123, out of all 6, 2 were of the third type 123 and 4 were estimated to be of the fourth type 124. For the images of the fourth type 124, all 4 were estimated to be of the fourth type 124.

[0101] When using the feature quantities of the regions above the upper lip (regions S1, S2, S3, S5), as shown in the graph 289 of FIG. 2N, among all 20 target person images 110, 14 images had the estimated type match the actual type, and the estimation accuracy was 70%. Specifically, for the images of the first type 121, out of all 5 images, 3 were estimated to be of the first type 121 and 2 were estimated to be of the second type 122. For the images of the second type 122, all 5 images were estimated to be of the second type 122. For the images of the third type 123, out of all 6 images, 2 were estimated to be of the third type 123 and 4 were estimated to be of the fourth type 124. For the images of the fourth type 124, all 4 images were estimated to be of the fourth type 124.

[0102] The estimation accuracies shown in FIGS. 2J and 2L - 2N are summarized in FIG. 2O. As shown in the table 291 of FIG. 2O, the estimation accuracies are particularly high when using the feature quantities of all regions (S1 - S2), the feature quantities of the regions below the eyebrows (S2 - S6), and the feature quantities of only region S6. Therefore, region S6, which is included in all of these, is considered important for improving the estimation accuracy.

[0103] Also, as shown in the table 291 of FIG. 2O, among the cases of using only the feature quantities of one region (the feature quantities of regions S1, S2, S3, and S4 respectively), the estimation accuracies are high when using the feature quantities of region S3 and when using the feature quantities of region S4. This is presumably because regions S3 and S4 are the regions where volume starts to appear due to the overlap of hair curls, and are the points that hair experts such as beauticians and barbers focus on when estimating the degree of curls.

[0104] The hair trait estimation device 100 may further include a normalization unit. The normalization unit normalizes the subject image 110 based on the length from the eyebrow portion to the chin tip of the subject 111 in the subject image 110. By normalizing the subject image 110, comparison between a plurality of subject images 110 becomes possible. In the present embodiment, the size of the subject image 110 is normalized by making the length from the eyebrow portion to the chin tip of the subject 111 in the subject image 110 equal.

[0105] A method by which the normalization unit normalizes the subject image 110 will be described with reference to FIG. 2P. First, the size of the margin in the subject image 110 is adjusted to generate a margin-adjusted image 260. The margin-adjusted image 260 is an image whose vertical and horizontal lengths are, for example, four times the length L from the eyebrow portion to the chin tip of the subject 111 in the subject image 110. When the margin in the subject image 110 is small, as shown in FIG. 2P, a margin is added to the subject image 110 and the vertical and horizontal lengths are increased to generate the margin-adjusted image 260. On the other hand, when the margin in the subject image 110 is large, the margin of the subject image 110 is reduced by trimming or the like and the vertical and horizontal lengths are shortened to generate the margin-adjusted image 260. Note that the vertical and horizontal lengths of the margin-adjusted image 260 may be one to three times the length L, or may be five times or more.

[0106] Next, the image size of the margin-adjusted image 260 is resized so as to be a predetermined size, and a normalized image 261 is generated. The predetermined size is, for example, a vertical length Dx of 1024 pixels and a horizontal length Dy of 1024 pixels.

[0107] Next, an example of the hair property type table 301 included in the hair property estimation apparatus 100 will be described with reference to FIG. 3. The hair property type table 301 stores a hair contour shape 312 in association with a hair property type 311. The hair property type 311 is four types defined according to the degree of hair curl, and is classified into four types from the first to the fourth in descending order of the strength of the degree of curl. The hair contour shape 312 indicates the shape characteristics of the hair contour, and the shape characteristics are stored for each type of hair property. Then, the hair property estimation unit 203 estimates the hair property type by referring to, for example, the hair property type table 301.

[0108] Referring to FIG. 4, the hardware configuration of the hair property estimation apparatus 100 will be described. The CPU (Central Processing Unit) 410 is a processor for arithmetic control, and realizes each functional configuration of the hair property estimation apparatus 100 in FIG. 2 by executing a program. The CPU 410 has a plurality of processors, and may execute different programs, modules, tasks, threads, etc. in parallel. The ROM (Read Only Memory) 420 stores fixed data such as initial data and programs, and other programs. Further, the network interface 430 communicates with other devices via a network. Note that the CPU 410 is not limited to one, and may be a plurality of CPUs, or may include a GPU (Graphics Processing Unit) for image processing. Further, the network interface 430 preferably has a CPU independent of the CPU 410 and writes or reads transmission / reception data to / from the area of the RAM (Random Access Memory) 440. Further, it is desirable to provide a DMAC (Direct Memory Access Controller) for transferring data between the RAM 440 and the storage 450 (not shown). Furthermore, the CPU 410 recognizes that data has been received or transferred to the RAM 440 and processes the data. Also, the CPU 410 prepares the processing result in the RAM 440, and leaves the subsequent transmission or transfer to the network interface 430 or the DMAC.

[0109] The RAM 440 is a random access memory that the CPU 410 uses as a temporary storage work area. A storage area for storing data necessary for the implementation of this embodiment is secured in the RAM 440. The target person image data 441 is data related to an image of the target person 111. The contour shape data 442 is data related to the contour of the hair of the target person 111 extracted from the target person image 110. The feature amount data 443 is data related to the feature amount of the extracted hair contour. The hair property type data 444 is data related to the property of the hair, for example, data related to classification according to the degree of curl.

[0110] The transmission / reception data 445 is data transmitted and received via the network interface 430. Also, the RAM 440 has an application execution area 446 for executing various application modules.

[0111] The storage 450 stores a database, various parameters, or the following data or programs necessary for the implementation of this embodiment. The storage 450 stores the hair property type table 301. The hair property type table 301 is a table that manages the relationship between the hair property type 311 and the hair contour shape 312 shown in FIG. 3.

[0112] Storage 450 further stores a subject image acquisition module 451, a contour extraction module 452, and a hair property estimation module 453. The subject image acquisition module 451 is a module that acquires a subject image 110 including the head of the subject 111. The contour extraction module 452 is a module that extracts the contour line 112 of the hair of the subject 111 in the acquired subject image 110. The hair property estimation module 453 is a module that estimates the properties of the hair of the subject 111 based on the extracted contour line 112 of the hair of the subject 111. These modules 451 to 453 are read into the application execution area 446 of the RAM 440 by the CPU 410 and executed. The control program 454 is a program for controlling the entire hair property estimation device 100.

[0113] The input / output interface 460 interfaces input / output data with input / output devices. A display unit 461 and an operation unit 462 are connected to the input / output interface 460. Further, a storage medium 464 may be connected to the input / output interface 460. Furthermore, a speaker 463 which is an audio output unit, a microphone (not shown) which is an audio input unit, or a GPS position determination unit may be connected. Note that programs and data related to general functions and other realizable functions of the hair property estimation device 100 are not shown in the RAM 440 and the storage 450 shown in FIG. 4.

[0114] Next, with reference to the flowchart shown in FIG. 5A, the processing procedure of the hair property estimation device 100 will be described. This flowchart is executed by the CPU 410 in FIG. 4 using the RAM 440, and realizes each functional configuration of the hair property estimation device 100 in FIG. 2A.

[0115] To implement this flowchart using an information processing apparatus that operates as the hair property estimation apparatus 100, for example, it is necessary to copy and store in the storage 450 a program for executing this flowchart, which is stored in a removable storage medium such as a USB memory or a CD-ROM (not shown). A method of copying the program to the storage 450 may be, for example, a method of copying it from a program server connected to a network to the storage 450 via the network.

[0116] In step S501 (subject image acquisition step), the subject image acquisition unit 201 acquires a subject image 110 including the head of the subject 111. In step S503 (contour line extraction step), the contour line extraction unit 202 extracts the contour line 112 of the hair of the subject 111 from the acquired subject image 110. In step S505 (hair property estimation step), the hair property estimation unit 203 estimates the property of the hair of the subject 111 based on the extracted contour line 112. These steps are realized by causing a computer (CPU 410) to execute the program stored in the storage 450 as described above.

[0117] As shown in FIG. 5B, after executing step 503 (contour line extraction step), in step 504 (contour feature amount extraction step), the feature amount of the contour line 112 may be extracted. As described above, the extraction of the feature amount may be performed by the contour line extraction unit 202, the hair property estimation unit 203, or a contour feature amount extraction unit. Here, the extraction of the feature amount of the contour line 112 is performed by an independent contour feature amount extraction unit. Specifically, the contour feature amount extraction unit divides the contour line 112 extracted in the contour line extraction step 503 into regions according to the heights of each part of the face such as the eyes and mouth, and extracts the feature amounts of the contour line 112 in each region, such as the angle and length.

[0118] According to this embodiment, since the properties of the hair are determined based on the contour line (silhouette) of the hair, the degree of the curl of the entire hair can be accurately estimated. Even when there is a curl only inside the hair, since the degree of the curl of the hair is estimated from the contour line, the degree of the curl of the entire hair can be surely estimated. Further, since the properties of the hair are determined from the contour line of the hair, the degree of the curl of the entire hair that cannot be estimated from the shape of a single hair can be surely estimated.

[0119] Also according to this embodiment, since the contour line is extracted from the entire hair and the degree of the curl of the hair is estimated from the feature amount, the amount of data directly used for the estimation is small and the time until the result of the estimation is obtained is short.

[0120] Furthermore, since the hair property estimation device 100 estimates the degree of the curl from the entire contour line, it is possible to finely distinguish the curl strength in the case of a weak curl that has been difficult to distinguish conventionally. Since the fine distinction has become possible, it becomes easier to determine whether it is better to make use of the curl or to make it straight, and the proposal of the hairstyle and hair cosmetics becomes more appropriate.

[0121] [Second Embodiment] Next, the hair property estimation device according to the second embodiment of the present invention will be described with reference to FIGS. 6 to 9. FIG. 6 is a block diagram for explaining the configuration of the hair property estimation device 600 according to this embodiment. The hair property estimation device 600 according to this embodiment is different from the first embodiment in that it has a display control unit and an input reception unit. Since the other configurations and operations are the same as those of the first embodiment, the same configurations and operations are denoted by the same reference numerals and the detailed description thereof is omitted.

[0122] The hair trait estimation device 600 includes a display control unit 601 and an input reception unit 602. The display control unit 601 superimposes and displays the contour line 112 extracted by the contour line extraction unit 202 on the subject image 110 (superimposed image). The display control unit 601, for example, superimposes and displays the contour line 112 of the hair of the subject 111 on the subject image 110. However, in order to emphasize that it is the contour line of the hair, a thick line or a colored line may be displayed as the contour line 112. In this way, by displaying, the contour portion of the hair is emphasized, and the subject 111 and other users can easily grasp the contour line 112 of the hair of the subject 111.

[0123] The input reception unit 602 receives an input regarding the appropriateness of the superimposed and displayed contour line 112. That is, the input reception unit 602 receives an input as to whether the contour line 112 of the hair emphasized and displayed by the display control unit 601 is applicable as the contour line 112 of the hair of the subject 111. The input regarding the appropriateness may be made using a mobile terminal such as a smartphone owned by the subject 111, or may be performed by the subject 111 or other persons using a predetermined input device or the like. That is, it is up to a person to check whether the contour line 112 of the hair is correctly extracted.

[0124] When the received input result is "applicable", the input reception unit 602 sends the input result to the hair property estimation unit 203. When the input result is "not applicable", the input reception unit 602 sends the input result to the contour extraction unit 202. The contour extraction unit 202 that has received the inapplicable input result may obtain the subject image 110 again and extract the contour line 112. Also, in the contour extraction unit 202, multiple contour extractions may be performed in advance, and for those arranged and displayed, by allowing the input of applicability, a more suitable one can be selected as the contour line 112. It is preferable to perform multiple contour extractions by obtaining a plurality of subject images in advance and performing contour extraction for each of the obtained plurality of subject images. Also, before contour extraction, by performing image processing such as different filtering on one subject image, a plurality of different images can be obtained, and by performing contour extraction for each of the subject images after the plurality of image processings, multiple contour extractions can also be performed. Note that it is of course possible to mix these two methods for multiple contour extractions.

[0125] Then, based on the contour line determined to be "applicable", the hair property estimation unit 203 estimates the hair properties of the subject 111. For the contour line determined to be "not applicable", the contour extraction unit 202 re-performs the extraction of the hair contour line of the subject 111. Then, the input reception unit 602 receives the input regarding the applicability of the contour line again. In this way, by receiving the input regarding the applicability of the contour line by the input reception unit 602, the hair properties can be estimated based on a hair contour line that seems to be more correct, so it becomes possible to improve the accuracy of the estimation.

[0126] The hair trait estimation unit 203 estimates the traits of the subject's hair using a learned hair trait estimation model generated by having an artificial intelligence learn the relationship between the contour line and the traits. Preferably, the feature amount of the contour line is extracted, and the traits are estimated from the extracted feature amount. Further, the feature amount is preferably extracted from a predetermined one or a plurality of regions in the contour line. As the feature amount, for example, either one of the angle or the length of the contour line may be used, or both the angle and the length of the contour line can be used.

[0127] Next, an example of the action table 701 included in the hair trait estimation device 600 will be described with reference to FIG. 7. The action table 701 stores an action 712 in association with an input result 711. The input result 711 is an input regarding the suitability of the superimposed contour line. The action 712 is the next operation determined according to the input result. Then, the hair trait estimation device 600 refers to the action table 701 to determine the next operation.

[0128] With reference to FIG. 8, the hardware configuration of the hair trait estimation device 600 will be described. The RAM 840 is a random access memory that the CPU 410 uses as a temporary storage work area. A storage area for storing data necessary for realizing the present embodiment is secured in the RAM 840. The input result 841 is an input regarding the suitability of the displayed contour line. The action candidate data 842 is data regarding the next operation determined according to the input result.

[0129] The storage 850 stores a database, various parameters, or the following data or programs necessary for realizing the present embodiment. The storage 850 stores the action table 701. The action table 701 is a table that manages the relationship between the input result 711 and the action 712 shown in FIG. 7.

[0130] The storage 850 further stores a display control module 851 and an input reception module 852. The display control module 851 is a module that superimposes and displays the extracted contour line on the subject image 110. The input reception module 852 is a module that receives an input regarding the suitability of the superimposed contour line.

[0131] Next, with reference to the flowchart shown in FIG. 9, the processing procedure of the hair property estimation device 600 will be described. This flowchart is executed by the CPU 410 in FIG. 8 using the RAM 840, and realizes each functional configuration of the hair property estimation device 600 in FIG. 6. The program necessary for the realization of this flowchart is, for example, copied from a removable storage medium such as a USB memory or a CD-ROM (not shown) to the storage 850, or received from a server via a communication line and copied to the storage 850, and then stored in the storage 850. By doing so, in the hair property estimation device 600, this flowchart is realized. The same applies to necessary data and tables.

[0132] In step S901 (superimposed display step), the display control unit 601 superimposes and displays the extracted contour line on the subject image 110. In step S903 (suitability input reception step), the input reception unit 602 receives an input regarding the suitability of the superimposed contour line. In step S905, the hair property estimation device 600 determines whether the input result is "suitable" or not. If the input result is "suitable" (YES in step S905), the hair property estimation device 600 proceeds to step S505. If the input result is "no (not suitable)" (NO in step S905), the hair property estimation device 600 returns to step S503 and repeats the subsequent processing. The contour line extraction unit 202 that has received an input result indicating inapplicability acquires the subject image 110 again and extracts the contour line in step S503. Also, in step S503, the contour line extraction unit 202 may perform a plurality of contour line extractions in advance, and allow the suitability to be input for those arranged and displayed, so that a more suitable one can be selected as the contour line.

[0133] According to the present embodiment, since an input regarding the suitability of the extracted contour line is received, the properties of the hair can be estimated based on a more accurate contour line of the hair, and the accuracy of property estimation can be further improved.

[0134] [Third Embodiment] Next, the hair property estimation apparatus according to the third embodiment of the present invention will be described with reference to FIGS. 10 to 13. FIG. 10 is a block diagram for explaining the configuration of the hair property estimation apparatus 1000 according to the present embodiment. The hair property estimation apparatus 1000 according to the present embodiment is different in that it includes a hair care plan generation unit, a presentation unit, and a question presentation / answer reception unit 1003 as compared with the first embodiment or the second embodiment. Since the other configurations and operations are the same as those in the first embodiment or the second embodiment, the same reference numerals are given to the same configurations and operations, and detailed descriptions thereof are omitted.

[0135] The hair property estimation apparatus 1000 includes a hair care plan generation unit 1001, a presentation unit 1002, and a question presentation / answer reception unit 1003. The hair care plan generation unit 1001 generates a hair care plan according to the properties of the hair of the subject 111 based on the estimation result by the hair property estimation unit 203. The hair care plan is, for example, a proposal of a product, a proposal of how to use the product, a recommendation of a hairstyle, a proposal of a treatment menu at a beauty salon, a proposal of a treatment timing at a beauty salon, and the like. In addition to these, the hair property estimation apparatus 1000 includes a subject image acquisition unit 201, a contour line extraction unit 202, a hair property estimation unit 203, a display control unit 601, and an input reception unit 602, which are the same as those in the first embodiment or the second embodiment.

[0136] When the hair characteristics of the target person 111 are, for example, the strong curl types 121 and 122 (when the outline spread and bulge are large), the hair care plan generation unit 1001 generates a hair care plan that makes use of the original strong curl shape, such as a perm. Also, when the hair characteristics of the target person 111 are, for example, the weak curl types 123 and 124 (when the outline spread and bulge are small), the hair care plan generation unit 1001 generates a hair care plan such as straightening the hair, a styling-friendly agent, a hair care plan for a styling method, or a hair care plan for a style that makes use of the original curl shape of each type. In this way, the hair care plan generation unit 1001 generates a hair care plan according to the strength and degree of the hair curl.

[0137] For example, in the case of the target person 111 with strong hair curls, the hair care plan generation unit 1001 does not regard the curly hair as a complex, but generates a hair care plan that makes use of the curls so that the curly hair can be changed into the personality and strength of the target person 111. If the target person 111 who feels complex about the hair curls generates a hair care plan to straighten the curly hair, the complex of having curly hair remains without disappearing. In such a case, the hair care plan generation unit 1001 generates a plan that allows the target person 111 to have confidence in their own hair or a plan that allows them to like their own hair. For example, as a recommended hairstyle, the hair care plan generation unit 1001 generates a hair care plan such as a curl style or a wave style that makes use of the curls and undulations, or recommended hair care products.

[0138] In addition, for a subject 111 with weak hair habits, the hair care plan generation unit 1001 generates a hair care plan to eliminate (suppress) the habits, for example, a plan to make the hair straight and natural just by washing the hair (without using a straight iron or the like), or a plan to make it easier to stretch the habit with a straight iron or the like. Further, the hair care plan generation unit 1001 generates a straight style hairstyle with the habit stretched, recommended hair care products, etc. as the hair care plan.

[0139] The presentation unit 1002 presents the generated hair care plan to the subject 111. The presentation unit 1002, for example, transmits data related to the display of the generated hair care plan to a mobile terminal such as a smartphone or a predetermined monitor held by the subject 111 or the like, and causes it to be displayed on the display of the mobile terminal or the like. In addition, the presentation unit 1002 transmits data related to the display of the hair care plan to a monitor or the like connected to the hair property estimation device 1000, and causes the hair care plan to be displayed. Note that the presentation unit 1002 may transmit the original hair properties and the analysis results of the hair properties to a predetermined display device and cause them to be displayed together with the presentation of the hair care plan. Further, the presentation unit 1002 may transmit only the estimated hair properties to a predetermined display device and cause them to be displayed.

[0140] The question presentation / answer reception unit 1003 presents questions to the subject 111 and receives answers to the questions. The questions are provided to the subject 111 from, for example, the hair property estimation device 1000. The provided questions may be single questions or may be such that the next question changes according to the answer of the subject 111, such as in an interview format (question and answer). The content of the questions is, for example, how one usually feels about one's own hair quality and properties, the part one wants to correct, the hairstyle one wants to try, etc., but is not limited to these. The question presentation / answer reception unit 1003 may present a plurality of answer candidates assumed for the questions to the subject 111 and enable the subject to select an appropriate answer from among them.

[0141] Next, with reference to FIG. 11, an example of the hair care plan table 1101 included in the hair property estimation apparatus 1000 will be described. The hair care plan table 1101 stores response content 1112 and a hair care plan 1113 in association with hair properties 1111. The hair properties 1111 are the properties of the hair of the estimated subject 111, and are classified into a first type 121, a second type 122, a third type 123, and a fourth type 124. The response content 1112 is the response from the subject 111 to the question. The hair care plan 1113 is derived from the combination of the hair properties 1111 of the subject 111 and the response content 1112. What is shown here is an exemplification of the hair care plan table 1101, and it is not necessary for one hair care plan to correspond to the combination of the hair properties and the response content. That is, a plurality of hair care plans may correspond to the combination of the hair properties and the response content. Then, the hair property estimation apparatus 1000 generates a hair care plan with reference to the hair care plan table 1101. Also, it is possible to configure the hair care plan generation unit 1001 to create a hair care plan only from the hair properties estimated by the hair property estimation unit 203 from the contour without receiving the response of the subject to the question, that is, without having the question presentation / response reception unit 1003. In that case, the hair care plan table 1101 is composed only of the hair properties 1111 and the hair care plan 1113.

[0142] With reference to FIG. 12, the hardware configuration of the hair property estimation apparatus 1000 will be described. The RAM 1240 is a random access memory used by the CPU 410 as a work area for temporary storage. A storage area for storing data necessary for the realization of the present embodiment is secured in the RAM 1240. The hair care plan data 1241 is data regarding the generated hair care plan. The question data 1242 is data regarding the questions presented to the subject 111. The response data 1243 is data regarding the answers of the subject 111 to the questions.

[0143] The storage 1250 stores a database, various parameters, or the following data or programs necessary for implementing this embodiment. The storage 1250 stores the hair care plan table 1101. The hair care plan table 1101 is a table that manages the relationship between the hair properties 1111 and the hair care plan 1113 shown in FIG. 11. The data or programs necessary for implementing this embodiment are copied to the storage 1250 from a removable storage medium such as a USB memory or a CD-ROM (not shown), or the data or programs stored in a server or the like are copied to the storage 1250 via a communication line (not shown), and then stored in the storage 850.

[0144] The storage 1250 further stores a hair care plan generation module 1251, a presentation module 1252, and a question presentation / answer reception module 1253. The hair care plan generation module 1251 is a module that generates a hair care plan according to the properties of the hair of the subject 111 based on the estimation result by the hair property estimation unit 203. The presentation module 1252 is a module that presents the generated hair care plan to the subject 111. The question presentation / answer reception module 1253 is a module that presents questions to the subject 111 and receives answers to the questions.

[0145] Next, with reference to the flowchart shown in FIG. 13, the processing procedure of the hair property estimation device 1000 will be described. This flowchart is executed by the CPU 410 in FIG. 12 using the RAM 1240 to realize each functional configuration of the hair property estimation device 1000 in FIG. 10.

[0146] In step S501 (subject image acquisition step), the subject image acquisition unit 201 acquires a subject image 110 of the subject 111, more specifically, including the head of the subject 111 from the top of the head to the tip of the chin. In step S503 (contour line extraction step), the contour line extraction unit 202 extracts the contour line 112 of the hair of the subject 111 in the acquired subject image 110. In step S901 (overlay display step), the display control unit 601 causes the extracted contour line to be overlaid and displayed on the subject image 110. In step S903 (appropriateness input reception step), the input reception unit 602 receives an input regarding the appropriateness of the overlaid contour line. In step S905, the hair property estimation device 600 determines whether the input result is "appropriate" or not. If the input result is "appropriate" (YES in step S905), the hair property estimation device 600 proceeds to step S505. If the input result is "no (not appropriate)" (NO in step S905), the hair property estimation device 600 returns to step S503 and repeats the subsequent processing. In step S505 (hair property estimation step), the hair property estimation unit 203 estimates the properties of the subject 111's hair based on the extracted contour line 112.

[0147] In step S1301 (question presentation / answer reception step), the question presentation / answer reception unit 1003 asks the subject 111 a question and receives an answer to the question. In step S1303 (hair care plan generation step), the hair care plan generation unit 1001 generates a hair care plan 1113 according to the subject 111's hair properties from the combination of the subject 111's hair properties 1111 and the answer content 1112.

[0148] According to this embodiment, a hair care plan according to the properties of the subject's hair can be proposed. In addition, since the hair care plan is generated taking into account the answers to the subject's questions, a highly satisfactory hair care plan according to the subject's preferences and desires can be proposed.

[0149] [Fourth Embodiment] Next, a hair property estimation system according to a fourth embodiment of the present invention will be described with reference to FIGS. 14 to 15. FIG. 14 is a diagram for explaining the configuration of a hair property estimation system 1400 according to the present embodiment. The hair property estimation system 1400 according to the present embodiment includes a mobile terminal 1401 and a hair property estimation device 1402. The mobile terminal 1401 captures an image including the head of the subject 111 and acquires the captured image as a subject image 110. The mobile terminal 1401 that has acquired the subject image 110 extracts the contour line 112 of the hair of the subject 111, superimposes the contour line 112 on the subject image 110, and displays it to generate a contour line image 113. Then, the mobile terminal 1401 transmits the generated contour line image 113 to the hair property estimation device 1402. The hair property estimation device 1402 that has received the contour line image 113 estimates the properties of the hair of the subject 111 based on the contour line 112 (the contour of the hair) represented in the contour line image 113. The estimation of the properties is performed, for example, for which of the four types (the first type 121 to the fourth type 124 in the order of the strength of the curl) it corresponds to.

[0150] Next, with reference to FIG. 15, the configurations of the mobile terminal 1401 and the hair property estimation device 1402 included in the hair property estimation system 1400 will be described. The mobile terminal 1401 is a smartphone, a tablet terminal, or the like, and has a subject image acquisition unit 1501 and a contour line extraction unit 1502. The mobile terminal 1401 and the hair property estimation device 1402 can communicate with each other by wireless communication or wired communication.

[0151] <Mobile terminal 1401> The mobile terminal 1401 includes a camera, a subject image acquisition unit 1501 that acquires a subject image 110 captured by the camera, a contour line extraction unit 1502 that extracts the contour line 112 of the hair of the subject 111 in the acquired subject image, a display control unit 1503 that superimposes and displays the contour line 112 extracted by the contour line extraction unit on the subject image, a display unit (not shown) that displays an appropriateness confirmation image, and a transmission unit that transmits the contour line 112 of the hair extracted by the contour line extraction unit to the hair property estimation device 1402.

[0152] The subject image acquisition unit 1501 of the mobile terminal 1401 acquires an image including the head of the subject 111 as the subject image 110. The subject image 110 may be an image captured by a camera provided in the mobile terminal 1401 or an image captured by an imaging device other than the mobile terminal 1401. Further, the subject image 110 may be an image (selfie image) captured using a front camera or the like provided in the mobile terminal 1401.

[0153] When the subject image 110 is captured by a camera or the like provided in the mobile terminal 1401, the captured subject image 110 is temporarily stored in the storage area of the camera or the like. Then, the subject image acquisition unit 1501 may acquire the stored subject image 110, or the subject image acquisition unit 1501 may acquire the subject image 110 by receiving the subject image 110 transmitted from the camera or the like.

[0154] The contour extraction unit 1502 extracts the contour line 112 of the hair of the subject 111 in the subject image 110. Further, the contour extraction unit 1502 extracts the feature amount of the contour line 112. The feature amount of the contour line 112 is, for example, that the head of the subject 111 is divided into several regions from the top of the head to the chin, and the spread (angle) of the hair for each divided region.

[0155] When transmitting the contour image 113 from the mobile terminal 1401 to the hair property estimation device 1402, only the information of the contour line 112 may be transmitted to the hair property estimation device 1402. That is, it may not be appropriate for the operator or the like operating the hair property estimation device 1402 to hold the image in which the face of the subject 111 is captured from the viewpoint of personal information protection.

[0156] Furthermore, the contour extraction unit 1502 extracts the features of the contour line 112. The contour extraction unit 1502 divides the area from the top of the head to the chin of the subject 111 in the subject image 110 into several regions, and extracts the feature amounts of the contour line 112 for each divided region. The feature amount of the contour line 112 is, for example, the angle of the contour line 112. Then, the contour extraction unit 1502 also transmits the feature amount of this contour line 112 to the hair property estimation device 1402 together with the contour line image 113.

[0157] As described above, by transmitting only the data related to the contour line 112 and the data related to the feature amount of the contour line 112 from the mobile terminal 1401 to the hair property estimation device 1402, personal information such as the face photo of the subject 111 does not reach the operator of the hair property estimation system 1400. Therefore, even for the operator side, there is no need to take measures for personal information protection, and the burden related to system operation can be reduced. Note that even if both the data related to the contour line 112 and the data related to the feature amount of the contour line 112 are not sent to the hair property estimation device 1402, only one of them, that is, only the data representing the contour line 112 or only the data related to the feature amount of the contour line 112 may be configured to be transmitted from the mobile terminal 1401 to the hair property estimation device 1402.

[0158] When sending only the data related to the feature amount of the contour line 112, as the feature amount, instead of the angle of the contour line 112, for example, the length of the contour line 112 in a predetermined region may be sent. Alternatively, both the angle and the length of the contour line 112 can be sent. By sending only the data of the feature amount of the contour line 112, the data amount can be reduced compared to the case of sending the data representing the contour line 112.

[0159] The display control unit 1503 superimposes and displays the extracted contour (contour line 112) on the subject image 110. Thereby, the subject 111 can confirm the extracted contour line 112 on the display of the mobile terminal 1401. As a result of confirming the contour line 112, the subject 111 can perform operations such as taking a face image again as needed.

[0160] Note that in the mobile terminal 1401 of the present embodiment, the display control unit 1503 that superimposes and displays the contour line 112 extracted by the contour line extraction unit on the subject image as described above is not essential. When the mobile terminal 1401 does not have the display control unit 1503, it does not necessarily have to have the function of displaying the appropriateness confirmation image on the display unit. Even in this case, by limiting the data transmitted from the mobile terminal 1401 to the hair property estimation device 1402 to only the data representing the contour line 112, only the data regarding the feature amount of the contour line 112, or both of them, personal information such as the face photo of the subject 111 will not be passed to the operator of the hair property estimation device 1402, and the technical effect of reducing the amount of data to be transmitted can be achieved.

[0161] The hair property estimation system 1400 includes a display control unit 1503 that superimposes the contour line 112 extracted by the contour line extraction unit 1502 on the subject image and displays it as an appropriateness confirmation image, and an input reception unit 1511 that receives an input regarding the appropriateness of the superimposed and displayed contour line 112. The hair property estimation unit 1510 estimates the property of the subject's hair based on the received input.

[0162] The program used in the mobile terminal of the hair trait estimation system 1400 can be configured to cause the mobile terminal to execute a contour line extraction step of extracting the contour line 112 of the hair from the subject image by the contour line extraction unit 1502, and a transmission control step of transmitting the contour line 112 of the hair extracted in the contour line extraction step to the hair trait estimation device 1402. Further, it may be configured to cause the mobile terminal 1401 to execute a display control step of superimposing the contour line 112 extracted by the contour line extraction unit on the subject image and displaying it as an appropriateness confirmation image, an input reception step of receiving an input regarding the appropriateness of the superimposed and displayed contour line 112, and a transmission control step of transmitting the contour line 112 to the hair trait estimation device 1402 according to the input in the input reception step. In this case, a plurality of contour line images obtained by superimposing the subject image and the contour line 112 may be displayed, and in the input step, it may be configured to be able to select the contour line 112 to be transmitted from among the plurality of superimposed images. Note that the program used in the above mobile terminal 1401 is preferably stored in a server accessible from the Internet, and transmitted to the mobile terminal 1401 of the user via a communication line such as the Internet so that it can be downloaded.

[0163] <Hair trait estimation device 1402> The hair trait estimation device 1402 includes a hair trait estimation unit 1510, an input reception unit 1511, a hair care plan generation unit 1512, a presentation unit 1513, and a question presentation / answer reception unit 1514.

[0164] The hair trait estimation unit 1510 estimates the hair traits of the subject 111 using a learned hair trait estimation model generated by having the artificial intelligence learn the relationship between the contour line 112 of the hair and the hair traits. The hair traits are, for example, the degree of hair curl, and the hair traits are estimated based on the spreading condition of the hair of the subject 111.

[0165] The input reception unit 1511 receives an input regarding the suitability of the superimposed contour line 112. The input regarding the suitability is made by the subject 111 while viewing the display of the mobile terminal 1401. The input regarding the suitability may be made, for example, using an input device provided in the mobile terminal 1401 or using a connected input device.

[0166] Also, the input regarding the suitability may be made using an input device connected to the hair property estimation device 1402. The hair property estimation unit 1510 estimates the hair property of the subject 111 based on the received input result. That is, an operator of the system or the like checks whether the extracted contour (contour line 112) can be applied on the display of the hair property estimation device 1402. When an operator of the system or the like determines that it can be applied, subsequent processing is performed in the hair property estimation device 1402.

[0167] More specifically, the subject 111 or the like may make an input regarding the suitability by pressing the "YES" and "NO" buttons displayed on the mobile terminal 1401 while viewing the display (suitability confirmation image) of the contour line 112 displayed on the display of the mobile terminal 1401. Then, the mobile terminal 1401 transmits the input information regarding the suitability to the hair property estimation device 1402. Note that the input information regarding the suitability may be transmitted to the hair property estimation device 1402 together with the contour line image 113 or information only about the contour line 112 (information of the contour line 112). The input reception unit 1511 receives the input regarding the suitability of the contour line 112 transmitted in this way.

[0168] Here, the information only about the contour line 112 is information about the shape and position of the contour line 112, that is, the coordinates 235. For example, instead of directly sending the two-dimensional image data in which the contour line 112 appears, the data amount can be compressed by sending only the position information of the pixels constituting the contour line 112. Alternatively, the data amount can also be compressed by sending only the information regarding the position and shape of the curves and straight lines constituting the contour line 112. As these data amount compression methods, techniques well-known to those skilled in the art can be used without limitation.

[0169] Based on the estimated results of the hair properties, the hair care plan generation unit 1512 generates a hair care plan according to the properties of the hair of the subject 111. The hair care plan generation unit 1512 generates a plan according to the properties (degree of curl) of the hair of the subject 111. When the hair properties are of a strong curl type (the first type 121 and the second type 122), for example, a plan that makes use of the strength of the curl is generated. Also, when the hair properties are of a weak curl type (the third type 123 and the fourth type 124), for example, a plan that stretches the curl or a plan that makes use of the weakness of the curl is generated.

[0170] The presentation unit 1513 presents the generated hair care plan to the subject 111. The presentation unit 1513, for example, transmits data regarding the generated hair care plan to the mobile terminal 1401, and the received mobile terminal 1401 presents it to the subject 111 by displaying the hair care plan on the display.

[0171] The question presentation / answer reception unit 1514 asks the subject 111 questions and receives answers to those questions. The questions are transmitted from the hair property estimation device 1402 to the mobile terminal 1401. The transmitted questions are displayed on the display of the mobile terminal 1401. When the subject 111 answers the questions and transmits data regarding the answers from the mobile terminal 1401 to the hair property estimation device 1402, the question presentation / answer reception unit 1514 receives the data regarding the answers. The content of the questions, for example, includes questions about the hairstyles the subject 111 wants to try, the usual hair care methods, the problems and dissatisfaction regarding the hair properties, etc., but is not limited to these.

[0172] Then, the hair care plan generation unit 1512 generates a hair care plan taking into account the answers received by the question presentation / answer reception unit 1514. The generated hair care plan is a plan that makes use of the hair properties (curl), a plan that stretches the hair properties (curl), etc., and may include styling products, styling methods, etc. necessary to achieve this in the plan.

[0173] According to this embodiment, information related to the personal information of the target person remains on the mobile terminal side, and only information not related to the personal information is transmitted to the hair property estimation device. Therefore, a company or the like operating the hair property estimation device can estimate the hair properties without handling the personal information.

[0174] <Other embodiments> Note that each functional configuration of the mobile terminal 1401 and the hair property estimation device 1402 included in the hair property estimation system 1400 according to this embodiment is not limited to the above-described example. Each of the functional configurations of the mobile terminal 1401 and the hair property estimation device 1402 may be possessed by any terminal or device.

[0175] For example, the contour extraction unit 1502 may be possessed by the hair property estimation device 1402. In this case, the target person image 110 is transmitted from the mobile terminal 1401 to the hair property estimation device 1402, and the target person image 110 to be transmitted is processed so that the face of the target person 111 is not recognizable by the operator of the hair property estimation device 1402. Alternatively, data obtained by removing only the face of the target person 111 from the target person image 110 may be transmitted to the hair property estimation device 1402.

[0176] Further, the input reception unit 1511 may be provided in the mobile terminal 1401. In this case, in the mobile terminal 1401, an input regarding the suitability of the extracted contour is received, and the input result is transmitted from the mobile terminal 1401 to the hair property estimation unit 1510 of the hair property estimation device 1402. The hair property estimation unit 1510 estimates the hair property of the subject 111 based on the received input result. That is, the subject 111 checks whether the extracted contour (contour line 112) can be applied on the display of the mobile terminal 1401. When the subject 111 determines that it can be applied, the mobile terminal 1401 transmits data regarding the extracted contour to the hair property estimation device 1402, and subsequent processing is performed in the hair property estimation device 1402. Note that the input regarding the suitability does not necessarily have to be made. That is, without providing the input reception unit 1511, the hair property estimation device 1402 extracts feature amounts and performs subsequent processing without making a determination of suitability in either the mobile terminal 1401 or the hair state estimation device regarding the information on the contour line extracted in the mobile terminal 1401, or the mobile terminal 1401 extracts the contour line and the feature amounts of the contour line from the captured image without making a determination of suitability, and subsequent processing is performed in the hair property estimation device 1402.

[0177] The question for the subject 111 is not transmitted from the hair property estimation device 1402 to the mobile terminal 1401, but may be stored in the mobile terminal 1401 in advance and presented to the subject 111. Specifically, the program executed in the mobile terminal may be provided with the content of the question and the function of presenting it. That is, the question presentation / answer reception unit 1514 may be provided in the mobile terminal. In this case, the content of the question may be constant regardless of the hair property or may vary depending on the hair property. In the latter case, information regarding the estimated hair property is transmitted from the hair property estimation device 1402 to the mobile terminal 1401.

[0178] [Fifth Embodiment] Next, the hair property estimation system 1600 according to the fifth embodiment of the present invention will be described with reference to FIGS. 16 to 18. FIG. 16 is a block diagram for explaining the configurations of the mobile terminal 1601 and the hair property estimation server 1602 included in the hair property estimation system 1600 according to the present embodiment. The hair property estimation system 1600 according to the present embodiment is different from the fourth embodiment in that the input reception unit 1511 is provided in the mobile terminal 1601 instead of the hair property estimation server 1602. Since the other configurations and operations are the same as those of the fourth embodiment, the same configurations and operations are denoted by the same reference numerals and their detailed descriptions are omitted. In the present embodiment, the amount of information for hair property estimation transmitted from the mobile terminal 1601 to the hair property estimation server 1602 is minimized.

[0179] <Mobile terminal 1601> The mobile terminal 1601 includes a camera (not shown), a subject image acquisition unit 1501, a contour line extraction unit 1502, a display control unit 1503, an input reception unit 1511, and a transmission unit (not shown).

[0180] The subject image acquisition unit 1501 acquires an image including at least the head of the subject 111, more specifically, from the top of the head to the tip of the chin, as the subject image 110. The subject image acquisition unit 1501 acquires the subject image 110 temporarily stored in a storage area such as a camera. The subject image 110 is an image including at least the head of the subject 111, more specifically, from the top of the head to the tip of the chin.

[0181] The contour extraction unit 1502 extracts the outer contour line 112 of the hair of the subject 111 as the contour line. Here, the outer contour line means the contour line on the side that does not touch the face among the contour lines of the hair. Further, the contour extraction unit 1502 extracts the feature amount from the extracted outer contour line. In the present embodiment, in order to reduce the calculation amount, the outer contour line is extracted only from the range from the top of the head to the tip of the chin in the subject image 110. The feature amount extracted by the contour extraction unit 1502 is the feature amount of the outer contour line. The feature amount of the outer contour line is, for example, the head of the subject 111 from the top of the head to the tip of the chin, and is divided into several regions based on the facial parts. The spread (angle) of the hair, that is, the contour line, in the divided region or the length of the contour line 112 in the divided region, or a multi-dimensional quantity (vector quantity) appropriately selected and combined from these angles or lengths. Then, the feature amount between the top of the head and the tip of the chin of the outer contour line extracted by the contour extraction unit 1502 is transmitted to the hair property estimation server 1602 via a transmission unit (not shown).

[0182] Note that the contour extraction unit 1502 may be configured to extract the contour line 112 but not extract the feature amount. In this case, the hair property estimation server 1602 extracts the feature amount. When the contour extraction unit 1502 does not extract the feature amount of the contour line, the contour extraction unit 1502 extracts data related to the contour line 112. Here, the data related to the contour line 112 is at least data representing the shape of the contour line 112. From the viewpoint of being able to correctly define the divided region when the hair property estimation server 1602 extracts the feature amount of the contour line 112, it is preferable that the data related to the contour line 112 further includes data indicating the position of the contour line 112 with respect to the eyes, the tip of the chin, and other feature points of the face in addition to the shape of the contour line 112.

[0183] As described above, by transmitting only the data related to the contour line 112 or the feature amount of the contour line 112 from the mobile terminal 1601 to the hair property estimation server 1602, personal information such as the face photo of the subject 111 will not be passed to the operator of the hair property estimation server 1602. Therefore, even on the operator side, there is no need to take measures for personal information protection, and the burden related to system operation can be reduced. Also, from the point that the amount of data transmitted from the mobile terminal 1601 to the hair property estimation server 1602 can be reduced compared to the case of sending an image in which the face of the subject 111 is shown, it is preferable to transmit only the data related to the contour line 112 or the feature amount of the contour line 112 from the mobile terminal 1601 to the hair property estimation server 1602.

[0184] The display control unit 1503 superimposes and displays the extracted contour (contour line 112) on the subject image 110. Also, the display control unit 1503 displays the information received from the hair property estimation server 1602. The display control unit 1503 displays an image or the like on the display of the mobile terminal 1601. Note that the display control unit 1503 may display an image or the like on a display connected to the mobile terminal 1601 by wire or wirelessly.

[0185] <Hair property estimation server 1602> The hair property estimation server 1602 includes a hair property estimation unit 1510, a question presentation / answer reception unit 1514, a hair care plan generation unit 1512, and a presentation unit 1513.

[0186] The question presentation / answer reception unit 1514 creates a question for the subject 111 based on the hair property estimated by the hair property estimation unit 1510, transmits the created question to the mobile terminal 1601, and receives an answer to the question sent from the mobile terminal 1601.

[0187] The hair care plan generation unit 1512 creates a hair care plan based on the estimated hair property and the answer sent from the mobile terminal 1601.

[0188] The prompting unit 1513 prompts the target person 111 with the hair care plan created by the hair care plan generation unit 1512 by sending the hair care plan to the mobile terminal 1601.

[0189] Next, with reference to the flowcharts shown in FIGS. 17A and 17B, the processing procedures of the mobile terminal 1601 and the hair property estimation server 1602 will be described. This flowchart stores a program stored in a replaceable storage medium such as a USB memory or a CD-ROM in a storage device in the mobile terminal 1601 or the hair property estimation server 1602, and is executed, for example, when the program is activated by the operation of the target person 111. The program may be stored in a storage device in the mobile terminal 1601 or the hair property estimation server 1602 by downloading it from the server where the program is stored via a network.

[0190] <Processing Procedure of Mobile Terminal 1601> First, with reference to the flowchart shown in FIG. 17A, the processing procedure of the mobile terminal 1601 will be described. In step S501, the subject image acquisition unit 1501 acquires a subject image 110 including the head of the subject 111. Specifically, the subject image 110 is captured by the camera of the mobile terminal 1601. In step S503, the contour extraction unit 1502 extracts the contour line 112 of the hair from the acquired subject image 110. In step S901, the display control unit 1503 superimposes the extracted contour line 112 of the hair on the subject image 110 and displays an appropriateness confirmation image on the display of the mobile terminal 1601. In this embodiment, the appropriateness confirmation image is the contour line image 113. In step S901, a display for requesting an input of the appropriateness of the contour line to the operator of the mobile terminal 1601, for example, the subject 111, is also displayed on the display of the mobile terminal 1601. In step S903, the input reception unit 1511 receives an input regarding the appropriateness of the superimposed contour line. In step S905, the mobile terminal 1601 determines whether the input result is "appropriate". If the input result is "appropriate" (YES in step S905), the mobile terminal 1601 proceeds to step S1701. If the input result is "no (not appropriate)" (NO in step S905), the mobile terminal 1601 returns to step S503 and repeats the subsequent processing.

[0191] In step S1701 (feature extraction step), the contour extraction unit 1502 extracts the feature amount of the contour line using the information (contour line data) regarding the contour line 112 of the hair determined to be "appropriate". In step S1703 (feature amount transmission step), the transmission unit transmits the extracted feature amount of the contour line to the hair property estimation server 1602.

[0192] In step S1705 (question reception step), the mobile terminal 1601 receives a question from the hair property estimation server 1602 according to the property of the hair of the subject 111. In step S1707, the display control unit 1503 displays the received question on the display of the mobile terminal 1601. Also in step S1707, the input reception unit 1511 receives an answer to the question input by the subject 111. The question is a question that serves as a reference for what kind of hair care plan to provide to the subject 111, such as lifestyle habits, specifically the frequency and time of hair washing, or hair care products used daily, and can be one or more. When all the answers to the question are received, the mobile terminal 1601 proceeds to step S1709 (answer transmission step).

[0193] In step S1709, the transmission unit transmits the answer received by the input reception unit 1511 to the hair property estimation server 1602. When the transmission of all the received answers is completed, it proceeds to step S1711 (hair care plan reception step).

[0194] In step S1711, the mobile terminal 1601 receives a hair care plan from the hair property estimation server 1602.

[0195] In step S1713 (hair care plan display step), the display control unit 1503 displays the received hair care plan on the display of the mobile terminal 1601.

[0196] <Processing procedure of the hair property estimation server 1602> Next, with reference to the flowchart shown in FIG. 17B, the processing procedure of the hair property estimation server 1602 will be described. In step S1715, the hair property estimation server 1602 receives the feature amount of the hair contour line 112 transmitted from the mobile terminal 1601. After receiving the feature amount, it then proceeds to step S505.

[0197] In step S505, the hair property estimation unit 1510 estimates the hair properties of the subject 111 based on the received feature amounts. When the estimation of the hair properties is completed, the process proceeds to step S1717 (question creation / transmission step).

[0198] In step S1717, the question presentation / answer reception unit 1514 creates a question corresponding to the estimated hair properties. As a creation method, it may be to select from a plurality of previously prepared questions according to the estimated hair properties, the age of the subject, and the like. Also in step S1717, the hair property estimation server 1602 transmits the created question to the mobile terminal 1601 in order to present it to the subject 111. When the question is transmitted, the process proceeds to step S1719 (answer reception step). In step S1719, the question creation / answer reception unit 1514 receives and accepts the answer input in response to the presented question from the mobile terminal 1601. The said answer is input by the subject 111, transmitted from the mobile terminal 1601, and received by the hair property estimation server 1602. When the reception of the answer is completed, the process proceeds to step S1303.

[0199] In step S1303, the hair care plan generation unit 1512 creates a hair care plan based on the hair properties estimated in step S505 and the answer received in step S1719. The creation of the hair care plan is first to select a plurality of candidates for the hair care plan to be presented to the subject 111 based on the estimated hair shape, and then, from the selected plurality of hair care plans, based on the answer of the subject 111, that is, the answer received in step S1719, select a hair care plan suitable for the lifestyle of the subject 111, for example, or exclude the unsuitable hair care plans. Specifically, for example, if there is an answer that one does not usually care for the hair during bathing, selecting a hair care plan such as shampooing that is performed during bathing is excluded. When the creation of the hair care plan is completed in step S1303, subsequently, the process proceeds to step S1721 (hair care plan transmission step).

[0200] In step S1721, the presentation unit 1513 transmits the created hair care plan toward the mobile terminal 1601. The transmitted hair care plan is displayed on the display of the mobile terminal 1601 as described above.

[0201] <Other forms> In this embodiment, the display control unit 1503 controls the creation and display of the appropriateness confirmation image so that the subject 111 can determine whether the acquired image is appropriate. However, it can also be configured so that the subject does not determine the appropriateness of the image. In this case, step S901 (superimposed display step), step S903 (appropriateness input reception step), and step S905 (appropriateness determination step) become unnecessary. Further, without creating an appropriateness confirmation image, the acquired image may be directly displayed and used for the appropriateness determination of whether it can be used for the estimation by the subject 111 without being superimposed on the contour line. In this case, as shown in FIG. 18, after acquiring the subject image 110 in step S501, in step S1801, the display control unit 1503 displays the subject image 110 on the display of the mobile terminal 1601. In step S903, the input reception unit 1511 receives an input regarding the appropriateness of the contour line displayed on the subject image 110 displayed on the display. In step S905, the mobile terminal 1601 determines whether the input result is "appropriate". If the input result is "appropriate" (YES in step S905), the mobile terminal 1601 proceeds to step S503. If the input result is "no (not appropriate)" (NO in step S905), the mobile terminal 1601 returns to step S501 and repeats the subsequent processing.

[0202] As can be easily understood from the above description, when the operation of the mobile terminal 1601 changes in this way, the movement of the hair property estimation server 1602 does not change, so a plurality of types of programs for the mobile terminal 1601 may be prepared.

[0203] Note that the determination of whether the range from the top of the head to the chin is included in the acquired image is merely an application of conventional face recognition technology. Therefore, instead of the subject 111 making the determination of suitability, it can also be performed by a program. In this case, instead of step S903 (Suitability Input Reception Step), a Shooting Range Determination Step (not shown) for determining whether the range from the top of the head to the tip of the chin is included in the acquired image can be configured to be performed by the mobile terminal 1801 without the involvement of the subject 111. Therefore, this system can be used more simply. Also in this case, it is not necessary to change the movement of the hair property estimation server 1602, and thus it is not necessary to change the program used in the hair property estimation server 1602.

[0204] In this specification, "hair" means human hair. "Headgear product" means, for example, a hair wig, a wig, a webbing, a hair extension, a blade hair, a hair accessory, a doll hair, etc. "Fiber for headgear product" means a fiber used for the headgear product excluding hair. In other words, either hair or a fiber for headgear product is used for the headgear product.

[0205] The present invention can be applied in a state where fibers for headgear products are mixed in the hair or a headgear product is used in the hair. However, from the viewpoint of accurately estimating the original properties of the hair, application in a state where only the hair or a headgear product is worn on the hair is preferable, and application in a state where only the hair is more preferable. As the fiber for the headdress product, either natural origin fiber or synthetic fiber may be used, but natural origin fiber is preferred. The natural origin fiber refers to the fiber collected from natural animals and plants (excluding hair), or the fiber artificially manufactured using proteins, polysaccharides, etc. as raw materials. Among these, in addition to animal hair, or keratin, collagen, casein, proteins such as proteins derived from soybeans, peanuts, corn, silk, etc., or fibers artificially manufactured using polysaccharides, etc. as raw materials are preferred, and regenerated protein fibers using keratin, collagen, casein, soy protein, peanut protein, corn protein, silk protein (for example, silk fibroin), etc. as raw materials are more preferred, and regenerated protein fibers such as regenerated collagen fibers using collagen as raw material and regenerated silk fibers using silk fibroin as raw material are more preferred, and regenerated collagen fibers are even more preferred.

[0206] The regenerated collagen fiber can be manufactured by known techniques. The composition of the regenerated collagen fiber does not have to be 100% collagen, and natural polymers, synthetic polymers, additives, etc. for quality improvement may be included. Furthermore, the regenerated collagen fiber may be post-processed or post-treated. As the form of the regenerated collagen fiber, a filament is preferred. The filament is generally taken out from the bobbin-wound or box-packed state. Also, the filament coming out of the drying process in the manufacturing process of the regenerated collagen fiber can be directly used.

[0207] Examples of synthetic fibers include fibers containing a synthetic resin as a main component. From the viewpoints of ease of manufacturing synthetic fibers and obtaining a texture similar to that of hair, the synthetic resin is preferably a thermoplastic resin, more preferably at least one selected from the group consisting of a polyester resin, a polyamide resin, a polyimide resin, a polyamideimide resin, a vinyl chloride resin, a polycarbonate resin, a polyphenylene sulfide resin, and a modacrylic resin (a copolymer of acrylonitrile and vinyl chloride). The "main component" as used herein means a component having a content in the synthetic fiber of preferably 50% by mass or more, more preferably 60% by mass or more, still more preferably 70% by mass or more, even more preferably 80% by mass or more, even more preferably 90% by mass or more, and 100% by mass or less.

[0208] In addition to the above synthetic resin, the synthetic fiber can contain various components such as a flame retardant, a flame retardant aid, a light or heat stabilizer, a fluorescent agent, an antioxidant, an antistatic agent, and an ultraviolet absorber, as long as the effects of the present invention are not inhibited.

[0209] Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above-described embodiments and can be appropriately modified. Those skilled in the art can make various modifications within the scope of the present invention regarding the configuration and details of the present invention. Also, a system or device formed by combining the respective separate features included in each embodiment in any manner is also included in the scope of the present invention.

[0210] Further, the present invention may be applied to a system composed of a plurality of devices or to a single device. Furthermore, the present invention is applicable even when an information processing program for realizing the functions of the embodiments is supplied to a system or device in a form recorded on a replaceable storage medium such as a USB memory or a CD-ROM and executed by a built-in processor. Therefore, in order to realize the functions of the present invention by a computer, a program installed in the computer, a storage medium storing the program, a WWW (World Wide Web) server for downloading the program, a processor for executing the program, and a program product are also included in the technical scope of the present invention. In particular, at least a non-transitory computer readable medium storing a program for causing a computer to execute the processing steps included in the above-described embodiments is included in the technical scope of the present invention.

[0211] Regarding the above-described embodiments of the present invention, the following additional remarks are disclosed. <1> A subject image acquisition unit that acquires a subject image including at least from the top of the subject's head to at least the tip of the chin; A contour line extraction unit that extracts a contour line of the subject's hair in the acquired subject image; A hair property estimation unit that estimates the property of the subject's hair based on the extracted contour line; A hair property estimation device comprising the above. <2> The hair property estimation device according to <1>, wherein the hair property estimation unit estimates the property of the subject's hair based on at least one of the angle and length of the extracted contour line. <3> The contour line extraction unit further extracts a feature amount of the contour line, The hair property estimation device according to <1> or <2>, wherein the hair property estimation unit estimates the property of the subject's hair using the extracted feature amount. <4> The hair property estimation device according to <3>, wherein the feature amount is extracted from a predetermined region on the contour line. <5> The hair property estimation device according to <3> or <4>, wherein the feature amount is at least one of the angle and length of the contour line. <6> The hair property estimation device according to any one of <1> to <5>, wherein the hair property estimation unit estimates the hair property of the subject using a learned hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line and the property. <7> The hair property estimation device according to <6>, wherein the learned hair property estimation model is generated by having an artificial intelligence learn the relationship between the feature amount of the contour line and the property. <8> The hair property estimation device according to any one of <1> to <7>, wherein the property includes the degree of hair curl. <9> The hair property estimation device according to any one of <1> to <8>, wherein the contour line extraction unit extracts the contour line of the subject's hair using a learned contour line extraction model generated by having an artificial intelligence learn the relationship between the subject image and the contour line. <10> A display control unit that superimposes and displays the contour line extracted by the contour line extraction unit on the subject image; An input reception unit that receives an input regarding the suitability of the superimposed and displayed contour line; and further includes: The hair property estimation device according to any one of <1> to <9>, wherein the hair property estimation unit estimates the hair property of the subject based on the received input. <11> A hair care plan generation unit that generates a hair care plan according to the hair property of the subject based on the estimation result by the hair property estimation unit; A presentation unit that presents the generated hair care plan to the subject; The hair property estimation device according to any one of the above <1> to <10>, further comprising <12> further comprising a question presentation / answer reception unit that receives an answer to a question from the subject, The hair property estimation device according to the above <11>, wherein the hair care plan generation unit generates the hair care plan from a combination of the answer and the hair property. <13> The hair property estimation device according to any one of the above <1> to <12>, further comprising a normalization unit that normalizes the subject image based on the length from the eyebrow portion to the chin tip of the subject in the subject image. <14> The subject image includes from the top of the subject's head to the hair tips, The hair property estimation device according to any one of the above <1> to <13>, wherein the hair property estimation unit estimates the appearance of the hair tips as the property. <15> A subject image acquisition step of acquiring a subject image including at least from the top of the subject's head to the chin tip, A contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image, A hair property estimation step of estimating the property of the subject's hair based on the extracted contour line, A hair property estimation method including <16> The hair property estimation method according to the above <15>, wherein in the hair property estimation step, the property of the subject's hair is estimated based on at least one of the angle and length of the extracted contour line. <17> In the contour line extraction step, further extract a feature amount of the contour line, The hair property estimation method according to the above <15> or <16>, wherein in the hair property estimation step, the property of the subject's hair is estimated using the extracted feature amount. <18> The hair property estimation method according to <17>, wherein the feature amount is extracted from a predetermined region on the contour line. <19> The hair property estimation method according to <17> or <18>, wherein the feature amount is at least one of the angle and length of the contour line. <20> The hair property estimation method according to any one of <15> to <19>, wherein in the hair property estimation step, the property of the hair of the subject is estimated using a learned hair property estimation model generated by causing an artificial intelligence to learn the relationship between the contour line and the property. <21> The hair property estimation method according to <20>, wherein the learned hair property estimation model is generated by causing an artificial intelligence to learn the relationship between the feature amount of the contour line and the property. <22> The hair property estimation method according to any one of <15> to <21>, wherein the property includes the degree of hair curl. <23> The hair property estimation method according to any one of <15> to <22>, wherein in the contour line extraction step, the contour line of the hair of the subject is extracted using a learned contour line extraction model generated by causing an artificial intelligence to learn the relationship between the subject image and the contour line. <24> A display control step of superimposing and displaying the contour line extracted in the contour line extraction step on the subject image, An input reception step of receiving an input regarding the suitability of the superimposed and displayed contour line, further comprising The hair property estimation method according to any one of <15> to <23>, wherein in the hair property estimation step, the property of the hair of the subject is estimated based on the received input. <25> A hair care plan generation step of generating a hair care plan according to the hair property of the subject based on the estimation result in the hair property estimation step, A presenting step of presenting the generated hair care plan to the subject; The hair property estimation method according to any one of <15> to <24> above, further comprising this. <26> Further comprising a question presenting / answer receiving step of receiving an answer to a question from the subject; In the hair care plan generation step, the hair property estimation method according to <25> above, which generates the hair care plan from the combination of the answer and the hair property. <27> The hair property estimation device according to any one of <15> to <26> above, further comprising a normalization step of normalizing the subject image based on the length from the eyebrow part to the chin tip of the subject in the subject image. <28> The subject image includes from the top of the subject's head to the hair tip; In the hair property estimation step, the hair property estimation method according to any one of <15> to <27> above, which estimates the appearance of the hair tip as the property. <29> A subject image acquisition step of acquiring a subject image including at least from the top of the subject's head to the chin tip; In the acquired subject image, a contour line extraction step of extracting the contour line of the subject's hair; A hair property estimation step of estimating the property of the subject's hair based on the extracted contour line; A hair property estimation program for causing a computer to execute. <30> In the hair property estimation step, the hair property estimation program according to <29> above, which estimates the property of the subject's hair based on at least one of the angle and length of the extracted contour line. <31> In the contour line extraction step, further extracting a feature amount of the contour line; The hair property estimation program according to <29> or <30> which estimates the property of the hair of the subject using the extracted feature amount in the hair property estimation step. <32> The hair property estimation program according to <31>, wherein the feature amount is extracted from a predetermined region on the contour line. <33> The hair property estimation program according to <31> or <32>, wherein the feature amount is at least one of the angle and length of the contour line. <34> The hair property estimation program according to any one of <29> to <33>, which estimates the property of the hair of the subject using a learned hair property estimation model generated by having an artificial intelligence learn the relationship between the contour line and the property in the hair property estimation step. <35> The hair property estimation program according to <34>, wherein the learned hair property estimation model is generated by having an artificial intelligence learn the relationship between the feature amount of the contour line and the property. <36> The hair property estimation program according to any one of <29> to <35>, wherein the property includes the degree of hair curl. <37> The hair property estimation program according to any one of <29> to <36>, which extracts the contour line of the hair of the subject using a learned contour line extraction model generated by having an artificial intelligence learn the relationship between the subject image and the contour line in the contour line extraction step. <38> A display control step of superimposing and displaying the contour line extracted in the contour line extraction step on the subject image, An input reception step of receiving an input regarding the suitability of the superimposed and displayed contour line, and further causing a computer to execute, In the hair property estimation step, based on the received input, the hair property estimation program according to any one of <29> to <37> for estimating the properties of the subject's hair. <39> A hair care plan generation step of generating a hair care plan according to the hair properties of the subject based on the estimation result in the hair property estimation step, A presentation step of presenting the generated hair care plan to the subject, The hair property estimation program according to any one of <29> to <38> for further causing a computer to execute. <40> Further causing a computer to execute a question presentation / answer reception step of receiving an answer to a question from the subject, In the hair care plan generation step, the hair property estimation program according to <39> for generating the hair care plan from the combination of the answer and the hair properties. <41> The hair property estimation program according to any one of <29> to <40> for further causing a computer to execute a normalization step of normalizing the subject image based on the length from the eyebrow part to the chin tip of the subject in the subject image. <42> The subject image includes from the top of the subject's head to the hair tips, In the hair property estimation step, the hair property estimation program according to any one of <29> to <41> for estimating the appearance of the hair tips as the property. <43> A subject image acquisition step of acquiring a subject image including at least from the top of the subject's head to the chin tip, In the acquired subject image, a contour line extraction step of extracting the contour line of the subject's hair, A hair property estimation step of estimating the properties of the subject's hair based on the extracted contour line, A storage medium storing a hair property estimation program for causing a computer to execute. <44> The storage medium according to <43>, which estimates the hair characteristics of the subject based on at least one of the angle and length of the extracted contour line in the hair characteristic estimation step. <45> In the contour line extraction step, further extract the feature amount of the contour line, The storage medium according to <43> or <44>, which estimates the hair characteristics of the subject using the extracted feature amount in the hair characteristic estimation step. <46> The storage medium according to <45>, wherein the feature amount is extracted from a predetermined region in the contour line. <47> The storage medium according to <45> or <46>, wherein the feature amount is at least one of the angle and length of the contour line. <48> The storage medium according to any one of <43> to <47>, which estimates the hair characteristics of the subject using a learned hair characteristic estimation model generated by causing an artificial intelligence to learn the relationship between the contour line and the characteristics in the hair characteristic estimation step. <49> The storage medium according to <48>, wherein the learned hair characteristic estimation model is generated by causing an artificial intelligence to learn the relationship between the feature amount of the contour line and the characteristics. <50> The storage medium according to any one of <43> to <49>, wherein the characteristics include the degree of hair curl. <51> The storage medium according to any one of <43> to <50>, which extracts the contour line of the subject's hair using a learned contour line extraction model generated by causing an artificial intelligence to learn the relationship between the subject image and the contour line in the contour line extraction step. <52> A display control step of superimposing and displaying the extracted contour line on the subject image in the contour line extraction step, An input reception step of receiving an input regarding the appropriateness of the superimposed and displayed contour line, further cause a computer to execute, In the hair property estimation step, based on the received input, a storage medium according to any one of <43> to <51> that estimates the properties of the subject's hair. <53> A hair care plan generation step of generating a hair care plan according to the hair properties of the subject based on the estimation result in the hair property estimation step; A presentation step of presenting the generated hair care plan to the subject; A storage medium according to any one of <43> to <52> that further causes a computer to execute. <54> Further cause a computer to execute a question presentation / answer reception step of receiving an answer to the subject's question, In the hair care plan generation step, a storage medium according to <53> that generates the hair care plan from the combination of the answer and the hair properties. <55> A storage medium according to any one of <43> to <54> that further causes a computer to execute a normalization step of normalizing the subject image based on the length from the subject's eyebrow portion to the chin tip in the subject image. <56> The subject image includes from the top of the subject's head to the hair tip, In the hair property estimation step, a storage medium according to any one of <43> to <55> that estimates the appearance of the hair tip as the property. <57> A hair property estimation system including a hair property estimation device and a mobile terminal, A subject image acquisition unit that acquires a subject image including at least from the top of the subject's head to the chin tip; In the acquired subject image, a contour line extraction unit that extracts the contour line of the subject's hair; A hair property estimation unit that estimates the properties of the subject's hair based on the extracted contour line; A hair trait estimation system comprising the following. <58> The hair trait estimation system according to <57>, wherein the hair trait estimation unit estimates the hair traits of the subject based on at least one of the angle and length of the extracted contour line. <59> The contour line extraction unit further extracts feature amounts of the contour line, The hair trait estimation system according to <57> or <58>, wherein the hair trait estimation unit estimates the hair traits of the subject using the extracted feature amounts. <60> The hair trait estimation system according to <59>, wherein the feature amounts are extracted from a predetermined region in the contour line. <61> The hair trait estimation system according to <60>, wherein the feature amounts are at least one of the angle and length of the contour line. <62> The hair trait estimation system according to any one of <57> to <61>, wherein the hair trait estimation unit estimates the hair traits of the subject using a learned hair trait estimation model generated by having an artificial intelligence learn the relationship between the contour line and the traits. <63> The hair trait estimation system according to <62>, wherein the learned hair trait estimation model is generated by having an artificial intelligence learn the relationship between the feature amounts of the contour line and the traits. <64> The hair trait estimation method according to any one of <57> to <63>, wherein the traits include the degree of hair curl. <65> The hair trait estimation system according to any one of <57> to <64>, wherein the contour line extraction unit extracts the contour line of the subject's hair using a learned contour line extraction model generated by having an artificial intelligence learn the relationship between the subject image and the contour line. <66> A display control unit that superimposes the contour line extracted by the contour line extraction unit on the subject image and displays it as an appropriateness confirmation image, An input reception unit that receives an input regarding the suitability of the superimposed outline comprising The hair property estimation unit estimates the properties of the subject's hair based on the received input, and the hair property estimation system according to any one of <57> to <65> above <67> A hair care plan generation unit that generates a hair care plan according to the hair properties of the subject based on the estimation result by the hair property estimation unit A presentation unit that presents the generated hair care plan to the subject The hair property estimation system according to any one of <57> to <66> above, further comprising <68> Further comprising a question presentation / answer reception unit that receives the subject's answer to a question The hair care plan generation unit generates the hair care plan from the combination of the answer and the hair properties, and the hair property estimation system according to <67> above <69> Further comprising a normalization unit that normalizes the subject image based on the length from the subject's eyebrow part to the chin tip in the subject image, and the hair property estimation system according to any one of <57> to <68> above <70> The subject image includes from the top of the subject's head to the hair tips The hair property estimation unit estimates the appearance of the hair tips as the property, and the hair property estimation system according to any one of <57> to <69> above <71> A camera A subject image acquisition unit that acquires a subject image taken by the camera An outline extraction unit that extracts the outline of the subject's hair in the acquired subject image A display control unit that superimposes and displays the outline extracted by the outline extraction unit on the subject image A display unit that displays a suitability confirmation image A transmission unit that transmits the outline of the hair extracted by the outline extraction unit to a hair property estimation device; A mobile terminal comprising the same. <72> A program for a mobile terminal used in a hair property estimation system, An outline extraction step of extracting the outline of hair from a subject image; A transmission control step of transmitting the outline of the hair extracted by the outline extraction unit to a hair property estimation device; A mobile terminal program that causes the mobile terminal to execute the above steps. <73> A display control step of superimposing the outline extracted by the outline extraction unit on the subject image and displaying it as an appropriateness confirmation image; An input reception step of receiving an input regarding the appropriateness of the superimposed outline; A transmission control step of transmitting the outline to the hair property estimation device according to the input in the input reception step; The mobile terminal program according to <72> above that causes the mobile terminal to execute the above steps. <74> In the display control step, a plurality of superimposed images can be displayed, In the input reception step, the outline to be transmitted can be selected from a plurality of the superimposed images, The mobile terminal program according to <73> above.

Claims

1. A subject image acquisition unit that acquires a subject image including the head of the subject, a contour line extraction unit that extracts a contour line of the hair of the subject in the acquired subject image, a hair property estimation unit that estimates the property of the hair of the subject based on the extracted contour line, A hair property estimation device comprising:

2. The hair property estimation device according to claim 1, wherein the hair property estimation unit estimates the property of the hair of the subject based on at least one of the angle and length of the extracted contour line.

3. The contour line extraction unit further extracts a feature amount of the contour line, The hair property estimation device according to claim 1 or 2, wherein the hair property estimation unit estimates the property of the hair of the subject using the extracted feature amount.

4. The hair property estimation device according to claim 3, wherein the feature amount is extracted from a predetermined region in the contour line.

5. The hair property estimation device according to claim 3, wherein the feature amount is at least one of the angle and length of the contour line.

6. The hair property estimation device according to claim 1, wherein the hair property estimation unit estimates the property of the hair of the subject using a learned hair property estimation model generated by having the relationship between the contour line and the property learned by artificial intelligence.

7. The hair property estimation device according to claim 6, wherein the learned hair property estimation model is generated by having the relationship between the feature amount of the contour line and the property learned by artificial intelligence.

8. The hair property estimation device according to claim 1 or 2, wherein the property includes the degree of hair curl.

9. The hair property estimation device according to claim 1, wherein the contour line extraction unit extracts the contour line of the hair of the subject using a learned contour line extraction model generated by having the relationship between the subject image and the contour line learned by artificial intelligence.

10. A display control unit that superimposes and displays the contour line extracted by the contour line extraction unit on the subject image, an input reception unit that receives an input regarding the suitability of the superimposed and displayed contour line, further comprising: The hair property estimation device according to claim 1, wherein the hair property estimation unit estimates the property of the hair of the subject based on the received input.

11. A hair care plan generation unit that generates a hair care plan according to the hair property of the subject based on the estimation result by the hair property estimation unit, a presentation unit that presents the generated hair care plan to the subject. The hair property estimation device according to claim 1, further comprising

12. further comprising a question presentation / answer reception unit that receives an answer to a question from the subject, The hair property estimation device according to claim 11, wherein the hair care plan generation unit generates the hair care plan from a combination of the answer and the hair property.

13. The hair property estimation device according to claim 1, further comprising a normalization unit that normalizes the subject image based on the length from the subject's eyebrow line to the chin tip in the subject image.

14. A subject image acquisition step of acquiring a subject image including the subject's head, A contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image, A hair property estimation step of estimating the property of the subject's hair based on the extracted contour line, A hair property estimation method including

15. A subject image acquisition step of acquiring a subject image including the subject's head, A contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image, A hair property estimation step of estimating the property of the subject's hair based on the extracted contour line, A hair property estimation program for causing a computer to execute

16. A subject image acquisition step of acquiring a subject image including the subject's head, A contour line extraction step of extracting a contour line of the subject's hair in the acquired subject image, A hair property estimation step of estimating the property of the subject's hair based on the extracted contour line, A storage medium storing a hair property estimation program for causing a computer to execute

17. A hair property estimation system including a hair property estimation device and a mobile terminal, A subject image acquisition unit that acquires a subject image including the subject's head, A contour line extraction unit that extracts a contour line of the subject's hair in the acquired subject image, A hair property estimation unit that estimates the property of the subject's hair based on the extracted contour line, A hair property estimation system comprising

18. A display control unit that superimposes the contour line extracted by the contour line extraction unit on the subject image and displays it as an appropriateness confirmation image, An input reception unit that receives an input regarding the appropriateness of the superimposed and displayed contour line, Comprising The hair property estimation system according to claim 17, wherein the hair property estimation unit estimates the properties of the hair of the subject based on the received input.

19. A camera, A subject image acquisition unit that acquires a subject image photographed by the camera, A contour line extraction unit that extracts the contour line of the hair of the subject in the acquired subject image, A display control unit that superimposes and displays the contour line extracted by the contour line extraction unit on the subject image, A display unit that displays an appropriateness confirmation image, A transmission unit that transmits the contour line of the hair extracted by the contour line extraction unit to a hair property estimation device, A mobile terminal comprising the same.

20. A program for a mobile terminal used in a hair property estimation system, A contour line extraction step of extracting the contour line of the hair from the subject image, A transmission control step of transmitting the contour line of the hair extracted by the contour line extraction unit to a hair property estimation device, A program for a mobile terminal that causes the mobile terminal to execute the same.

21. A display control step of superimposing the contour line extracted by the contour line extraction unit on the subject image and displaying it as an appropriateness confirmation image, An input reception step of receiving an input regarding the appropriateness of the superimposed contour line, A transmission control step of transmitting the contour line to the hair property estimation device according to the input in the input reception step, The program for a mobile terminal according to claim 20, which causes the mobile terminal to execute the same.

22. In the display control step, a plurality of superimposed images can be displayed, In the input reception step, the contour line to be transmitted can be selected from the plurality of superimposed images, The program for a mobile terminal according to claim 21.

Images