Focus area extraction method, beauty treatment method, focus area extraction system, information processing apparatus, and program

Abstract

The present disclosure provides techniques for extracting regions of particular potential interest or focus by a subject.SOLUTION: A focus area extraction method including (A) a step of acquiring a first image including at least a face of a subject, (B) a step of displaying one or more second images in which random patterns different from each other are superimposed on the first image and causing the subject to select a third image conforming to a first determination criterion a plurality of times, (C) a step of generating a fourth image based on a plurality of selected third images, and (D) a step of extracting a focus area in which the subject is interested or focused on based on a difference between the first image and the fourth image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a focal region extraction method, a method for proposing a beauty treatment method, a focal region extraction system, an information processing device, and a program. [Background technology]

[0002] Non-Patent Document 1 discloses that a self-face classification image (CI) is generated using a self-face photograph as a base image, and the subjective self-face is visualized. Non-Patent Document 1 also discloses that the attractiveness of the generated subjective self-face is compared between self-evaluation and evaluation by others, and an investigation is conducted into the self-face positivity bias. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Kawamura, Suzuka, and three others, "Verification of the positive bias in self-facial attractiveness through subjective visualization of one's own face," Proceedings of the Japanese Society of Cognitive Psychology, Japanese Society of Cognitive Psychology, 2023, p.80 Summary of the Invention [Problem to be solved by the invention]

[0004] It is known that humans have a positivity bias, which causes them to judge their own abilities and characteristics highly and rate their own face as more attractive than the faces of others.

[0005] The present disclosure provides techniques for extracting areas of particular potential interest or interest to a subject. [Means for solving the problem]

[0006] The present disclosure provides a focal area extraction method including the steps of: (A) acquiring a first image including at least the face of a subject; (B) displaying one or more second images in which different random patterns are superimposed on the first image, and having the subject select a third image that meets a first determination criterion; (C) generating a fourth image based on the selected third images; and (D) extracting a focal area that the subject is interested in or paying attention to based on the difference between the first image and the fourth image. [Effects of the Invention]

[0007] The present disclosure provides a technique for extracting areas that are of particular potential interest or interest to a subject. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an overview of a focal region extraction system according to this embodiment. [Figure 2] FIG. 2 is a diagram illustrating the hardware configuration of an information processing device in the focus region extraction system according to this embodiment. [Figure 3] FIG. 3 is a flowchart illustrating the processing in the focus region extraction system according to this embodiment. [Figure 4] FIG. 4 is a diagram illustrating the processing in the focal region extraction system according to this embodiment. [Figure 5] FIG. 5 is a diagram illustrating the processing in the focal region extraction system according to this embodiment. [Figure 6] FIG. 6 is a diagram illustrating the processing in the focal region extraction system according to this embodiment. [Figure 7] FIG. 7 is a diagram illustrating the processing in the focus region extraction system according to this embodiment. [Figure 8] FIG. 8 is a diagram illustrating the processing in the focus region extraction system according to this embodiment. [Figure 9]FIG. 9 is a flowchart illustrating the processing in the beauty suggestion system according to this embodiment. [Figure 10] FIG. 10 is a flowchart illustrating the processing in the beauty suggestion system according to this embodiment. [Figure 11] FIG. 11 is a diagram illustrating makeup (part 1) suggested by the beauty suggestion system according to this embodiment. [Figure 12] FIG. 12 is a diagram illustrating makeup (part 2) suggested by the beauty suggestion system according to this embodiment. [Figure 13] FIG. 13 is a diagram illustrating makeup (part 2) suggested by the beauty suggestion system according to this embodiment. [Figure 14] FIG. 14 is a diagram illustrating makeup (part 3) suggested by the beauty suggestion system according to this embodiment. [Figure 15] FIG. 15 is a diagram illustrating makeup (part 3) suggested by the beauty suggestion system according to this embodiment. [Figure 16] FIG. 16 is a diagram illustrating makeup (part 4) suggested by the beauty suggestion system according to this embodiment. [Figure 17] FIG. 17 is a diagram illustrating makeup (part 4) suggested by the beauty suggestion system according to this embodiment. [Figure 18] FIG. 18 is a diagram illustrating makeup (part 5) suggested by the beauty suggestion system according to this embodiment. [Figure 19] FIG. 19 is a diagram illustrating makeup (part 5) suggested by the beauty suggestion system according to this embodiment. [Figure 20] FIG. 20 is a diagram illustrating makeup (part 6) suggested by the beauty suggestion system according to this embodiment. [Figure 21] FIG. 21 is a diagram illustrating makeup (part 6) suggested by the beauty suggestion system according to this embodiment. [Figure 22] FIG. 22 is a diagram illustrating makeup (No. 7) suggested by the beauty suggestion system according to this embodiment. [Figure 23] FIG. 23 is a diagram illustrating makeup (No. 7) suggested by the beauty suggestion system according to this embodiment. [Figure 24] FIG. 24 is a diagram illustrating makeup (part 8) suggested by the beauty suggestion system according to this embodiment. [Figure 25] FIG. 25 is a diagram illustrating processing in a modified example of the focal region extraction system according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, each embodiment of the present disclosure will be described with reference to the accompanying drawings. Note that, in the description of the specification and drawings relating to each embodiment, components having substantially the same or corresponding functional configurations may be designated by the same reference numerals, and redundant description may be omitted.

[0010] ≪Focal area extraction method≫ A focus region extraction method according to this embodiment will be described. The focus region extraction method according to this embodiment includes the steps of (A) acquiring a first image including at least the face of a subject. The focus region extraction method according to this embodiment also includes the steps of (B) displaying one or more second images in which different random patterns are superimposed on the first image, and having the subject select a third image that meets a first criterion, and repeating this process multiple times. The focus region extraction method according to this embodiment also includes the steps of (C) generating a fourth image based on the selected third images, and (D) extracting a focus region that the subject is interested in or paying attention to based on the difference between the first image and the fourth image.

[0011] The focal area is not particularly limited, but may include facial parts such as the eyes, nose, mouth, and areas surrounding these, as well as the entire face, such as the facial outline.

[0012] Next, a focus area extraction system according to the present embodiment will be described as a system for executing the focus area extraction method according to the present embodiment. The focus area extraction system according to the present embodiment includes an imaging unit, a display unit, an input unit, and an information processing device. The information processing device in the focus area extraction system according to the present embodiment executes the following steps: (a) controlling the imaging unit to capture at least the face of the subject; and (b) acquiring the captured image of the subject's face from the imaging unit as a first image. The information processing device in the focus area extraction system according to the present embodiment also executes the following steps: (c) generating multiple second images by superimposing different random patterns on the first image. The information processing device in the focus area extraction system according to the present embodiment also executes the following steps: (d) controlling the display unit to display one or more second images. The information processing device in the focus area extraction system according to the present embodiment also executes the following steps: (e) acquiring, from the input unit, an input regarding a third image selected by the subject based on a first criterion from the second images displayed on the display unit. The information processing device in the focus area extraction system according to the present embodiment also executes the following steps: (f) repeating (d) and (e) in order multiple times. The information processing device in the focal region extraction system according to this embodiment also executes (g) a procedure for generating a fourth image based on the selected third images, and (h) a procedure for extracting a focal region that the subject is interested in or paying attention to based on the difference between the first image and the fourth image.

[0013] In addition, the focal area extraction system according to this embodiment may acquire, as the first image, an image including at least the face of the subject stored in a storage medium or the like, instead of acquiring the first image from the imaging unit.

[0014] The focus region extraction method according to this embodiment will be described in detail using a focus region extraction system according to this embodiment. Fig. 1 is a diagram showing an overview of a focus region extraction system 1, which is an example of the focus region extraction system according to this embodiment.

[0015] The focal region extraction system 1 uses an image of the subject's own face to extract a facial region that the subject is particularly likely to be interested in or pay attention to. The focal region extraction system 1 includes an imaging unit 10, a display unit 20, an input unit 30, and an information processing device 40.

[0016] The imaging unit 10 is, for example, a still camera or a video camera, and is used to capture at least the face of the subject.

[0017] The display unit 20 is, for example, a display. The display unit 20 displays, for example, an image (second image) in which a random pattern is superimposed on an image (first image) obtained by capturing an image of the subject's face. The display unit 20 also displays, for example, a fourth image generated based on a plurality of third images in which the subject meets the first criterion. The display unit 20 also displays, for example, a difference image between the image (first image) obtained by capturing an image of the subject's face and the fourth image generated based on a plurality of third images that meet the first criterion.

[0018] The input unit 30 is, for example, a keyboard. The input unit 30 inputs, for example, information on a third image selected by the subject from a plurality of second images.

[0019] The information processing device 40 captures at least the face of the subject using the imaging unit 10 and acquires it as a first image. The information processing device 40 also generates multiple second images by superimposing different random patterns on the first image. While displaying the second images, the information processing device 40 generates a fourth image based on multiple third images selected from the multiple second images displayed to the subject. The information processing device 40 then extracts a focal area that the subject is particularly potentially interested in or paying attention to from the difference between the first image and the fourth image.

[0020] The following describes the hardware configuration of the information processing device 40. Fig. 2 is a diagram illustrating the hardware configuration of the information processing device 40, which is an example of a skin condition estimating device according to the first embodiment.

[0021] The information processing device 40 is, for example, a computer. The information processing device 40 includes a CPU (Central Processing Unit) 41, a RAM (Random Access Memory) 42, and a ROM (Read Only Memory) 43. The information processing device 40 also includes a storage I / F (Interface) 44i, a communication I / F 45i, an external I / F 46i, and a display I / F 47i. The CPU 41, the RAM 42, the ROM 43, the storage I / F 44i, the communication I / F 45i, the external I / F 46i, and the display I / F 47i are each connected to a bus B1. For example, a storage medium 44 is connected to the storage I / F 44i. The imaging unit 10 and the input unit 30 are each connected to the external I / F 46i. The display unit 20 is connected to the display I / F 47i.

[0022] The CPU 41 is a computing device that reads a program (application) from a storage device such as the ROM 43 or the storage medium 44 onto the RAM 42 and executes processing.

[0023] The RAM 42 is a volatile semiconductor memory that temporarily stores, for example, programs (applications) and the like.

[0024] The ROM 43 is a non-volatile semiconductor memory that can retain programs (applications) even when the power is turned off. The ROM 43 stores programs such as the BIOS that is executed at startup, and various settings such as OS settings and network settings.

[0025] The storage I / F 44i is an interface with an external storage device such as the storage medium 44. The information processing device 40 reads and writes data from and to the storage medium 44 using the storage I / F 44i.

[0026] The storage medium 44 is, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0027] The communication I / F 45i is an interface that connects to an external network such as the Internet, etc. The communication I / F 45i connects to the external network using, for example, a wired or wireless communication method.

[0028] The external I / F 46i is an interface that connects to an external device of the information processing device 40. The external I / F 46i is, for example, a USB (Universal Serial Bus) or the like.

[0029] The display I / F 47i is an interface for connecting to an external display device in the information processing device 40. The display I / F 47i is, for example, a VGA (Video Graphics Array) or a DVI (Digital Visual Interface). The display I / F 47i may also be an HDMI (registered trademark) (High-Definition Multimedia Interface) or a DisplayPort.

[0030] The information processing device 40 executes various processes, which will be described later, by executing programs (applications) in the above-described hardware configuration.

[0031] The details of the processing of the information processing device 40 will be described. By using the processing of the information processing device 40 as an example, the steps included in the focal region extraction method according to this embodiment will be described. Furthermore, by using the processing of the information processing device 40 as an example, the procedure executed by the program according to this embodiment will be described. Figure 3 is a flow diagram illustrating the processing in the focal region extraction system 1, which is an example of the focal region extraction system according to this embodiment.

[0032] (Step S10) First, the information processing device 40 captures at least the face of the subject and acquires it as a first image. For example, the information processing device 40 controls the imaging unit 10 to capture at least the face of the subject. Then, the information processing device 40 acquires an image including the face of the subject captured by the imaging unit 10 as the first image. Alternatively, the information processing device 40 may acquire, for example, an image including at least the face of the subject from captured images stored in a storage medium or the like as the first image.

[0033] For example, if the first image is a color image, the information processing device 40 may convert the first image into a monochrome image by grayscaling it. Furthermore, the information processing device 40 may perform filtering as preprocessing, for example, to reduce noise or emphasize features.

[0034] An example of an image to be processed in the information processing device 40 will be described. Fig. 4 is a diagram illustrating a first image IMG1, which is an example of the first image acquired in step S10 in processing in the focal region extraction system 1, which is an example of the focal region extraction system according to this embodiment. The information processing device 40 acquires the first image IMG1 shown in Fig. 4.

[0035] (Step S20) Next, the information processing device 40 generates a plurality of second images by superimposing different random patterns on the first image. Fig. 5 is a diagram illustrating the second image IMG2, which is an example of the second image generated in step S20 in the processing in the focal region extraction system 1, which is an example of the focal region extraction system according to this embodiment. The information processing device 40 generates the second image IMG2 shown in Fig. 5 by superimposing the random pattern on the first image IMG1 shown in Fig. 4.

[0036] Then, the information processing device 40 controls the display unit 20 to display the generated one or more second images on the display unit 20. When the one or more second images are displayed on the display unit 20, the subject selects a third image that meets the first criteria from the one or more displayed second images. Then, the information processing device 40 controls the input unit 30 to input information related to the third image selected by the subject.

[0037] For example, when one second image is displayed on the display unit 20, the subject determines whether the displayed second image complies with the first judgment criterion. If the subject determines that the displayed second image complies with the first judgment criterion, the subject selects the displayed second image as a third image that complies with the first judgment criterion. On the other hand, if the subject determines that the displayed second image does not comply with the first judgment criterion, the subject inputs through the input unit 30 that the subject has determined that the displayed second image does not comply with the first judgment criterion. Therefore, if the subject determines that the displayed second image does not comply with the first judgment criterion, the subject does not select the displayed second image as the third image.

[0038] For example, when two or more second images are displayed on the display unit 20, the subject selects a second image that meets the first criterion from among the displayed second images as the third image. The number of second images that meet the first criterion and are selected as the third image may be one, or two or more. Furthermore, if there is no second image that meets the first criterion among the multiple second images, no image may be selected. Furthermore, when there are multiple second images that meet the first criterion, only the second image that best meets the first criterion may be selected.

[0039] Furthermore, for example, when two second images are displayed on the display unit 20, a relative judgment may be made and the one that is more suitable for the first criterion may be selected.

[0040] The first determination criterion may be, for example, a criterion for selecting a second image that the subject determines to be similar to the subject. An image that the subject determines to be similar to the subject is, for example, an image including "a face that is imagined in one's mind, that the subject thinks is similar to the subject," "one's appearance image," "one's body image," "a cognitive self-image," "a mirror image in one's mind," "a self-image face," etc. For example, the subject selects a second image that the subject determines to be similar to the subject from one or more second images. If the criterion for selecting a second image that the subject determines to be similar to the subject is adopted as the first determination criterion, the finally generated fourth image represents a subjective self-face that the subject considers to be similar to the subject's face.

[0041] The first criterion may be, for example, a criterion of selecting a second image that the subject determines to be close to the person they want to be based on the subject's face. An image that the subject determines to be close to the person they want to be may be, for example, an image that includes the subject's "ideal face or desired face," "the person they want to portray," "the way they want others to see them," etc. Furthermore, an image that the subject determines to be close to the person they want to be may be an image that includes a specific image they want to be, such as an impression of "looking healthy" or "looking active." For example, the subject selects, from one or more second images, a second image that the subject determines to be close to the person they want to be based on the subject's face. If the first criterion is to select a second image that the subject determines to be close to the person they want to be based on the subject's face, the finally generated fourth image represents the person they want to be based on the subject's face.

[0042] Furthermore, the information processing device 40 displays one or more second images on the display unit 20, and the subject selects a third image that meets the first criteria from the one or more displayed second images, and repeats the process of inputting information about the third image via the input unit 30 multiple times. The number of repetitions is, for example, a number selected from the range of 10 to 10,000. Alternatively, the number of repetitions may be, for example, a number selected from the range of 400 to 1,500.

[0043] Fig. 6 is a diagram illustrating an example of a third image IMG3 selected in step S20 in processing in a focal region extraction system 1, which is an example of a focal region extraction system according to this embodiment. Note that Fig. 6 shows a portion of the selected third images IMG3. The information processing device 40 stores information relating to the plurality of third images IMG3 shown in Fig. 6.

[0044] (Step S30) Next, the information processing device 40 generates a fourth image based on the plurality of third images selected in step S20. The fourth image is generated, for example, by averaging the plurality of third images.

[0045] FIG. 7 is a diagram illustrating an example of a fourth image IMG4 generated in step S30 in the processing in the focal region extraction system 1, which is an example of the focal region extraction system according to this embodiment.

[0046] The first image IMG1 acquired in step S10 is an image of the subject's face, and therefore represents the subject's objective self-face that is not influenced by the subject's subjectivity. On the other hand, the fourth image IMG4 generated in step S30 is generated from the third image IMG3 that the subject subjectively selected based on the first criterion in step S20, and therefore represents the subject's face that reflects the subjective impressions, etc. In other words, the fourth image IMG4 generated in step S30 is an image that visualizes the subject's self-face that reflects the subjective impressions, etc.

[0047] (Step S40) Next, the information processing device 40 extracts a focal region based on the difference between the first image and the fourth image. For example, the information processing device 40 calculates the difference between the first image IMG1 shown in Fig. 4 and the fourth image IMG4 shown in Fig. 7. Fig. 8 is a diagram illustrating an example of a difference image IMG5 between the first image IMG1 acquired in step S10 and the fourth image IMG4 generated in step S30 in the focal region extraction system 1, which is an example of the focal region extraction system according to this embodiment.

[0048] For example, if the first criterion is to select a second image that the subject judges to be similar to the subject, then the fourth image IMG4 is the subjective self-face of the subject. The subject's subjective self-face shown in the fourth image IMG4 is an image of the subject's face that the subject considers to be similar to the subject, particularly subconsciously. The difference between the first image IMG1, which is the subject's objective self-face, and the fourth image IMG4, which is the subject's subjective self-face, is considered to be an area in which the subject is particularly subconsciously or explicitly interested or paying attention, in relation to the first criterion of being similar to the subject. Therefore, the focus area extraction system according to this embodiment extracts the difference between the first image containing the subject's objective self-face and the fourth image containing the subject's subjective self-face as the area in which the subject is interested or paying attention, as the focus area.

[0049] For example, if the first criterion is to select a second image based on the subject's face that the subject judges to be closest to the self that the subject desires to be, the fourth image IMG4 is an image showing the subject's own face that the subject subjectively considers to be the self that the subject desires to be based on the subject's face. The difference between the first image IMG1, which is the subject's objective self-face, and the fourth image IMG4, which is the subject's self that the subject desires to be based on the subject's face, is considered to be an area of ​​particular interest or attention to the subject in relation to the first criterion of the self that the subject desires to be based on the subject's face. Therefore, the focus area extraction system according to this embodiment extracts the difference between the first image, which is the subject's objective self-face, and the fourth image, which is the subject's self that the subject desires to be based on the subject's face, as the focus area.

[0050] 8, it is determined that the subject is interested in or paying attention to the area around the eyes in relation to the first determination criterion. Therefore, the information processing device 40 extracts the area around the eyes as a focal area.

[0051] According to the focal region extraction method of this embodiment, it is possible to extract a region (focal region) in which a subject is particularly potentially interested or paying attention.

[0052] <How to propose beauty treatments> Next, a method for proposing a beauty treatment using the focal region extraction method according to this embodiment will be described. By using the focal region extraction method according to this embodiment, it is possible to extract an area (focal region) in which a subject is interested or paying attention. By proposing a beauty treatment, for example, makeup, to the area (focal region) in which the subject is interested or paying attention, it is possible to improve the subject's satisfaction.

[0053] Specific processing will be described. Fig. 9 is a flow diagram illustrating processing in the beauty suggestion system according to this embodiment. Note that, in order to explain the processing in the beauty suggestion system according to this embodiment, the information processing device 40 included in the focal region extraction system 1 will be used.

[0054] Steps S10, S20, S30, and S40 are the same processes as those in the focus area extraction method according to this embodiment, and therefore the explanations for these steps are omitted here and reference should be made to the explanations for the focus area extraction method according to this embodiment.

[0055] (Step S50) The information processing device 40 proposes a beauty treatment to the subject in accordance with the focal region extracted in step S40. The proposed beauty treatment is presented to the subject by, for example, displaying it on the display unit 20.

[0056] The proposed beauty method is, for example, makeup. For example, the information processing device 40 may propose makeup that emphasizes the parts corresponding to the focal areas extracted in step S40 so that they stand out. Alternatively, for example, the information processing device 40 may propose makeup that is subdued so that the parts corresponding to the focal areas extracted in step S40 do not stand out. Furthermore, for example, the information processing device 40 may propose makeup that brings the parts corresponding to the focal areas extracted in step S40 closer to the balance of an average face.

[0057] An average face is a face obtained by averaging the faces of multiple people in each category, such as age, region, etc. By applying makeup that brings the balance of the average face closer to that of the average face, it can be evaluated as being highly attractive by many people.

[0058] Next, more specific processing in the method for proposing a beauty treatment method according to this embodiment will be described. Fig. 10 is a flow diagram illustrating processing in the beauty treatment suggestion system according to this embodiment. Fig. 10 shows details of the processing in step S50 shown in Fig. 9. When performing the processing of Fig. 10, the first determination criterion in step S20 is to select a second image that the subject determines to be similar to the subject.

[0059] (Step S51) First, the information processing device 40 prompts the subject to input an ideality level, which indicates whether the fourth image generated in step S40 is close to the subject's ideal, using the input unit 30. Note that the ideality level may be determined by subjective assessment, or a physiological index (a response in the ventral striatum or an N170 component in ERP) or an index in a psychological task (reaction time under binocular rivalry) may be utilized.

[0060] (Step S52) Next, the information processing device 40 determines whether the ideality level input in step S51 is higher than a predetermined threshold. If the ideality level input in step S51 is higher than the predetermined threshold (YES in step S52, the ideality level is high), the information processing device 40 proceeds to step S53. If the ideality level input in step S51 is lower than the predetermined threshold (NO in step S52, the ideality level is low), the information processing device 40 proceeds to step S54.

[0061] (Step S53) If the ideality level input in step S51 is higher than a predetermined threshold (YES in step S52), the subject considers that the part corresponding to the focal area extracted in step S40 is a charming feature that the subject considers to be the subject's own charm point. Therefore, the information processing device 40 suggests makeup that emphasizes the part corresponding to the focal area that is the subject's charm point.

[0062] (Step S54) If the ideality level input in step S51 is lower than a predetermined threshold (YES in step S52), the subject considers that the area corresponding to the focal area extracted in step S40 is an area that the subject feels insecure about. Therefore, the information processing device 40 proposes makeup that corrects the insecurities in the area corresponding to the focal area, which is an area that the subject feels insecure about. For example, the information processing device 40 may propose makeup that brings the area corresponding to the focal area closer to the balance of an average face.

[0063] Note that if a beauty method is proposed and the subject does not like it, another proposal may be made again. For example, in the above example, when the process proceeds to step S53, if the subject does not like the beauty method (makeup) proposed in step S53, a beauty method (makeup) proposed in step S54 may be proposed to the subject. Also, for example, in the above example, when the process proceeds to step S54, if the subject does not like the beauty method (makeup) proposed in step S54, the beauty method (makeup) proposed in step S53 may be proposed to the subject. Note that the beauty method to be re-proposed is not limited to the above example, and a different new beauty method may also be proposed.

[0064] A more specific example will be described below, in which the body part corresponding to the focal area is the "eyes."

[0065] (1) When the "eyes" in the fourth image are smaller than those in the first image ("small eyes") A case will be described where, when comparing the first image and the fourth image, the "eyes" in the fourth image are smaller ("small eyes") than those in the first image.

[0066] (1-1) If small eyes are the target person's charm point If the ideality level input in step S51 is higher than the predetermined threshold (YES in step S52), it is considered that the subject considers small eyes to be a charming feature. Therefore, the information processing device 40 suggests makeup that emphasizes small eyes, which are a charming feature. FIG. 11 is a diagram illustrating an example of makeup suggested by the beauty suggestion method according to this embodiment. In the following explanation, the left eye will be used as an example.

[0067] The information processing device 40 suggests makeup that makes the most of the characteristic of "small eyes." The information processing device 40 presents the suggested makeup to the subject by, for example, displaying images, text, etc. on the display unit 20. Note that the method by which the information processing device 40 presents the makeup is not limited to the above-mentioned method, and may also present the makeup by, for example, audio, etc.

[0068] A specific method for suggesting makeup suggested by the information processing device 40 will be described. The information processing device 40 displays an image DIMG1 shown in FIG. 11 on the display unit 20. The information processing device 40 then suggests to the subject that they apply light-colored eyeshadow to the area indicated by line L11 (above and below the eye) to create a soft, bright look. Next, the information processing device 40 suggests to the subject that they apply glitter color to the area indicated by line L12 (the center of the upper eyelid and the inner corner of the eye) to make their eyes more striking. Next, the information processing device 40 suggests to the subject that they apply a thin brown eyeliner indicated by line L13 to the edge of their eyes, in line with the width of their eyes. The information processing device 40 also suggests to the subject that the eyeliner not extend beyond the outer corner of their eyes.

[0069] (1-2) If the subject feels complex about having small eyes If the ideality level input in step S51 is lower than a predetermined threshold (NO in step S52), it is considered that the subject feels insecure about having small eyes. Therefore, the information processing device 40 suggests makeup to correct the subject's insecure small eyes. Figures 12 and 13 are diagrams illustrating examples of makeup suggested by the beauty suggestion method according to this embodiment.

[0070] The information processing device 40 suggests makeup to make the eyes look bigger so as to correct the "small eyes" point.

[0071] The information processing device 40 displays an image DIMG21 shown in FIG. 12 on the display unit 20. The information processing device 40 then suggests to the subject to apply light-colored eyeshadow in the area indicated by line L21 (above and below the eye). Next, the information processing device 40 suggests to the subject to apply dark-colored eyeshadow in the area indicated by line L22 (from the edge of the upper eyelid to the area visible when the eyes are open) to clearly emphasize the eyes. Next, the information processing device 40 suggests to apply a thick black eyeliner indicated by line L23 along the edge of the eye, extending beyond the outer corner of the eye, to lengthen the width of the eyes.

[0072] 13 on the display unit 20. Then, the information processing device 40 prompts the subject to apply the mascara MSC2 firmly.

[0073] (2) When the eyes in the fourth image are longer than those in the first image ("long, narrow eyes") When comparing the first image with the fourth image, we will explain the case where the "eyes" in the fourth image are longer ("long, narrow eyes (the eyes are narrow vertically and long horizontally)") than in the first image.

[0074] (2-1) If the target person's charm point is their "long, narrow eyes" If the ideality level input in step S51 is higher than a predetermined threshold (YES in step S52), it is considered that the subject considers long, narrow eyes to be a charming feature. Therefore, the information processing device 40 suggests makeup that emphasizes the subject's long, narrow eyes, which are a charming feature. Each of Figs. 14 and 15 is a diagram illustrating an example of makeup suggested by the beauty suggestion method according to this embodiment.

[0075] A specific method for suggesting makeup suggested by the information processing device 40 will be described. The information processing device 40 displays an image DIMG31 shown in FIG. 14 on the display unit 20. The information processing device 40 then suggests to the subject that light-colored eyeshadow be applied to the area indicated by line L31 (upper eyelid). Next, the information processing device 40 suggests to the subject that dark-colored eyeshadow be applied to the area indicated by line L32 (from the edge of the upper eyelid to the area visible when the eyes are open), focusing on the outer corner of the eye. The information processing device 40 also suggests to the subject that dark-colored eyeshadow be applied to the area indicated by line L33 (the outer third of the lower eyelid). Next, the information processing device 40 suggests to the subject that black-colored eyeliner be applied thickly from the edge of the eye, extending beyond the outer corner and flicking up, as shown by line L34.

[0076] 15 on the display unit 20. The information processing device 40 then prompts the subject to apply mascara firmly to accentuate the outer corners of the eyes, as shown as mascara MSC3.

[0077] (2-2) If the subject feels insecure about their "long, narrow eyes" If the ideality level input in step S51 is lower than a predetermined threshold (NO in step S52), it is considered that the subject feels insecure about their long, narrow eyes. Therefore, the information processing device 40 suggests makeup to correct the long, narrow eyes that the subject feels insecure about. Figures 16 and 17 are diagrams illustrating examples of makeup suggested by the beauty suggestion method according to this embodiment.

[0078] The information processing device 40 suggests makeup that softens the narrow eyes so as to correct the "narrow eyes."

[0079] The information processing device 40 displays an image DIMG41 shown in FIG. 16 on the display unit 20. The information processing device 40 then suggests to the subject that they should apply light-colored eyeshadow in the area indicated by line L41 (above and below the eye) to create a soft, bright look. Next, the information processing device 40 suggests to the subject that they should apply dark-colored eyeshadow in the area indicated by line L42 (from the edge of the upper eyelid to the area visible when the eyes are open) with a wider width in the center of the eye. Next, the information processing device 40 suggests to the subject that they should apply dark-colored eyeshadow in the area indicated by line L43 (from the outer corner to the center of the lower eyelid) with a wider width in the outer corner of the eye. Next, the information processing device 40 suggests to the subject that they should apply glitter color in the area indicated by line L44 (the inner third of the lower eyelid). Next, the information processing device 40 suggests to the subject that they should apply brown-colored eyeliner indicated by line L45, drawing it downward or horizontally from the outer corner of the eye, slightly extending beyond the outer corner.

[0080] 17 on the display unit 20. The information processing device 40 then instructs the subject to apply mascara MSC4 in layers so as to emphasize the center.

[0081] (3) When the eyes in the fourth image are farther apart than those in the first image ("wide-set eyes") We will explain the case where, when comparing the first image with the fourth image, the eyes in the fourth image are farther apart than in the first image ("wide-apart eyes" or "centrifugal eyes (far-apart eyes)").

[0082] (3-1) When wide-set eyes are the target person's charm point If the ideality level input in step S51 is higher than a predetermined threshold (YES in step S52), it is considered that the subject considers wide-set eyes to be a charming feature. Therefore, the information processing device 40 suggests makeup that makes the most of the wide-set eyes, which are a charming feature. Figures 18 and 19 are diagrams illustrating examples of makeup suggested by the beauty suggestion method according to this embodiment.

[0083] A specific method for suggesting makeup suggested by the information processing device 40 will be described. The information processing device 40 displays an image DIMG51 shown in FIG. 18 on the display unit 20. The information processing device 40 then suggests to the subject that light-colored eyeshadow be applied to the area indicated by line L51 (upper and lower eyelids). Next, the information processing device 40 suggests to the subject that dark-colored eyeshadow be applied to the area indicated by line L52 (from the edge of the upper eyelid to the area visible when the eyes are open), focusing on the outer corner of the eye. The information processing device 40 also suggests to the subject that glitter color be applied to the area indicated by line L53 (inner corner of the eye). Next, the information processing device 40 suggests to the subject that black-colored eyeliner be drawn, indicated by line L54, from the center of the upper eyelid to the outer corner of the eye, extending beyond the outer corner.

[0084] 19 on the display unit 20. The information processing device 40 then prompts the subject to apply mascara MSC5 so as to emphasize the outer corners of the eyes.

[0085] (3-2) When the subject feels complex about their wide-set eyes If the ideality level input in step S51 is lower than a predetermined threshold (NO in step S52), it is considered that the subject feels insecure about their widely spaced eyes. Therefore, the information processing device 40 suggests makeup to correct the widely spaced eyes that the subject feels insecure about. Figures 20 and 21 are diagrams illustrating examples of makeup suggested by the beauty suggestion method according to this embodiment.

[0086] The information processing device 40 suggests makeup to soften the wide-set eyes so as to correct the "wide-set eyes."

[0087] The information processing device 40 displays an image DIMG61 shown in FIG. 20 on the display unit 20. The information processing device 40 then suggests to the subject that they apply light-colored eyeshadow in the area indicated by line L61 (above and below the eye) to create a soft, bright look. Next, the information processing device 40 suggests to the subject that they apply dark-colored eyeshadow in the area indicated by line L62 (from the edge of the upper eyelid to the area visible when the eyes are open) with a larger width in the center of the eye. Next, the information processing device 40 suggests to the subject that they apply dark-colored eyeshadow in the area indicated by line L63 (the center of the lower eyelid). Next, the information processing device 40 suggests to the subject that they apply a shadow color in the area indicated by line L64 (the area from the inner corner of the eyebrows to the shadow of the bridge of the nose). Next, the information processing device 40 suggests to the subject that they apply brown-colored eyeliner indicated by line L65, with the center slightly thicker and along the edge of the eye so as not to extend beyond the outer corner of the eye.

[0088] 21 on the display unit 20. The information processing device 40 then instructs the subject to apply mascara MSC6 in layers to emphasize the center.

[0089] Next, makeup applied to the face in the fourth image will be described. The following processing will be described in terms of a case where the first criterion in step S20 is to select a second image determined to be similar to the person the subject desires to be based on the subject's face. When the first criterion is to select a second image determined to be similar to the person the subject desires to be based on the subject's face, the fourth image is the subject's own face, which the subject subjectively considers to be the person they desire to be based on the subject's face. For example, even if there is general information that suggests that a certain makeup style tends to create a specific impression, it has been difficult to obtain guidelines from this general information on how to apply makeup to create a face that gives a specific impression when using one's own face as a base. Therefore, the proposed beauty method according to this embodiment extracts the person the subject desires to be based on the subject's face, etc., as the fourth image, thereby providing guidelines on how to use makeup to create various impressions of oneself using one's own face.

[0090] (1) If the eyes in the fourth image are larger than those in the first image The information processing device 40 suggests makeup to make the eyes larger and more prominent so as to draw them closer to the face in the fourth image. Figures 22 and 23 are diagrams illustrating examples of makeup suggested by the beauty suggestion method according to this embodiment.

[0091] The information processing device 40 displays an image DIMG71 shown in FIG. 22 on the display unit 20. The information processing device 40 then suggests to the subject to apply light-colored eyeshadow in the area indicated by line L71 (above and below the eye). Next, the information processing device 40 suggests to the subject to apply dark-colored eyeshadow in the area indicated by line L72 (from the edge of the upper eyelid to the area visible when the eyes are open) so as to clearly highlight the eyes. Next, the information processing device 40 suggests to the subject to apply brown-colored eyeliner indicated by line L73, thick from the edge of the eye and long beyond the outer corner of the eye.

[0092] 23 on the display unit 20. Then, the information processing device 40 prompts the subject to apply the mascara MSC7 firmly.

[0093] (2) When the eyebrows in the fourth image are more pronounced than those in the first image The information processing device 40 proposes makeup to make the eyebrows more prominent so as to bring them closer to the face in the fourth image. Fig. 24 is a diagram illustrating an example of makeup proposed by the beauty suggestion method according to this embodiment. Note that Fig. 24 also describes makeup in the case where the nose in the fourth image is more prominent than in the first image (3), which will be described later.

[0094] The information processing device 40 displays the image DIMG8 shown in Fig. 24 on the display unit 20. The information processing device 40 then instructs the subject to draw the eyebrows firmly from the beginning to the end in the area (eyebrows) indicated by the line L81 using an eyebrow pencil, eyebrow powder, etc. The information processing device 40 also instructs the subject to use eyebrow mascara to comb the eyebrows to create a natural look.

[0095] (3) When the nose is clearer in the fourth image than in the first image The information processing device 40 proposes makeup to make the nose more prominent so as to resemble the face in the fourth image. This will be explained with reference to FIG.

[0096] The information processing device 40 displays an image DIMG8 shown in Fig. 24 on the display unit 20. The information processing device 40 then prompts the subject to apply nose shadow color in the area indicated by line L82 (from under the eyebrows to the bridge of the nose). The information processing device 40 also prompts the subject to apply shadow color in a V-shape in the area indicated by line L83 (the tip of the nose). The information processing device 40 then prompts the subject to apply highlight color in the area indicated by line L84 (between the eyebrows, the bridge of the nose, and the tip of the nose).

[0097] In the above example, makeup was used as an example of the beauty treatment method, but the beauty treatment method may also include suggestions for beauty equipment to be applied, massage methods, lifestyle habits, etc.

[0098] Next, a method for detecting the difference between the first image and the fourth image will be described. As shown in Fig. 8, the first image and the fourth image may be compared pixel by pixel and the result may be output as a difference image.

[0099] Furthermore, the information processing device 40 may assign feature points to the first image and the fourth image by image recognition. The information processing device 40 may then extract differences between the elements constituting the face based on the positional relationship and size of the elements constituting the face, based on the positional relationship of the assigned feature points. FIG. 25 is a diagram showing a state in which feature points have been assigned to a face image. For example, the information processing device 40 may calculate, for example, the size (length and width) of the eyes, the angle of the eyes, the distance between the eyes, etc., based on the positional relationship of the feature points.

[0100] Although the focal region extraction system 1, which is an example of a focal region extraction system according to the present embodiment, includes the imaging unit 10, the display unit 20, the input unit 30, and the information processing device 40 as separate components, the focal region extraction system according to the present embodiment is not limited to the above configuration. For example, an information processing terminal such as a smartphone, tablet terminal, or smart mirror, in which the imaging unit 10, the display unit 20, the input unit 30, and the information processing device 40 are integrated, may be used.

[0101] The present disclosure is not limited to the above examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. [Explanation of symbols]

[0102] 1. Focal area extraction system 10. Imaging unit 20 Display section 30 Input section 40 Information processing equipment

Claims

1. (A) acquiring a first image including at least a face of a subject; (B) displaying one or more second images in which different random patterns are superimposed on the first image, and having the subject select a third image that meets the first judgment criterion, and repeating this process multiple times; (C) generating a fourth image based on the selected third images; (D) extracting a focal area of ​​interest or attention of the subject based on a difference between the first image and the fourth image; Including, Focal region extraction method.

2. In the step (B), if the number of the second images is two or more, the second images are displayed side by side, and the subject is prompted to select the third image from the second images arranged side by side based on the first determination criterion. The method of claim 1 .

3. In the step (B), if there is one second image, it is determined whether the second image satisfies the first judgment criterion, and if it satisfies the first judgment criterion, it is designated as the third image. The method of claim 1 .

4. (A) acquiring a first image including at least a face of a subject; (B) displaying one or more second images in which different random patterns are superimposed on the first image, and having the subject select a third image that meets the first judgment criterion, and repeating this process multiple times; (C) generating a fourth image based on the selected third images; (D) extracting a focal area of ​​interest or attention of the subject based on a difference between the first image and the fourth image; (E) suggesting a cosmetic treatment to the subject corresponding to the focal area; Including, How to suggest beauty treatments.

5. the first determination criterion is a criterion that the subject selects the second image that the subject determines to be similar to the subject from among the arranged second images, In the step (E), makeup that emphasizes the area corresponding to the focal area or makeup that brings the face closer to an average face balance is proposed. The method for proposing a beauty treatment according to claim 4.

6. the first determination criterion is a criterion that the subject selects the second image that the subject determines to be similar to the subject from among the arranged second images, In the step (E), The subject is asked to input a degree of ideality indicating whether the fourth image is close to the ideal that the subject considers; If the ideality level is high, makeup that emphasizes the part corresponding to the focal area is suggested, and if the ideality level is low, makeup that brings the face closer to an average face balance is suggested. The method for proposing a beauty treatment according to claim 4.

7. the first determination criterion is a criterion for selecting, from the arranged plurality of second images, the second image that the subject determines to be closest to the person he or she wants to be, In the step (E), a makeup application is suggested in the focal area to approximate the fourth image. The method for proposing a beauty treatment according to claim 4.

8. An imaging unit; A display unit; an input unit; an information processing device; Equipped with The information processing device includes: (a) controlling the imaging unit to capture at least a face of a subject; (b) acquiring an image of the face of the subject from the imaging unit as a first image; (c) generating a plurality of second images by superimposing different random patterns on the first image; (d) controlling the display unit to display one or more of the second images; (e) acquiring, from the input unit, an input regarding a third image selected by the subject based on a first determination criterion from the second images displayed on the display unit; (f) repeating steps (d) and (e) in order multiple times; (g) generating a fourth image based on the selected third images; (h) extracting a focal area of ​​interest or attention of the subject based on the difference between the first image and the fourth image; To execute Focal region extraction system.

9. A display unit; an input unit; an information processing device; Equipped with The information processing device includes: (a1) acquiring a first image including at least a face of a subject; (c) generating a plurality of second images by superimposing different random patterns on the first image; (d) controlling the display unit to display one or more of the second images; (e) acquiring, from the input unit, an input regarding a third image selected by the subject based on a first determination criterion from the second images displayed on the display unit; (f) repeating steps (d) and (e) in order multiple times; (g) generating a fourth image based on the selected third images; (h) extracting a focal area of ​​interest or attention of the subject based on a difference between the first image and the fourth image; To execute Focal region extraction system.

10. (A1) acquiring a first image including at least a face of a subject; (B1) displaying one or more second images in which different random patterns are superimposed on the first image, and having the subject select a third image that meets a first judgment criterion, and repeating this process multiple times to obtain multiple selected third images; (C1) generating a fourth image based on the selected third images; (D1) extracting a focal area of ​​interest or attention of the subject based on a difference between the first image and the fourth image; To execute Information processing device.

11. On the computer, (A1) acquiring a first image including at least a face of a subject; (B1) displaying one or more second images in which different random patterns are superimposed on the first image, and having the subject select a third image that meets a first judgment criterion, and repeating this process multiple times to obtain multiple selected third images; (C1) generating a fourth image based on the selected third images; (D1) extracting a focal area of ​​interest or attention of the subject based on a difference between the first image and the fourth image; A program to execute.

Images