Image processing device, image processing method and program

Abstract

To appropriately simulate a state of the face to which cosmetics are applied according to types of facial parts.SOLUTION: An image acquisition unit 102 acquires a first facial image and a second facial image. A display unit 104 displays the first facial image and the second facial image. A type discrimination unit 110 discriminates types of facial parts of a user on the basis of at least one of the first facial image and the second facial image. An area determination unit 113 determines a first makeup area on the basis of a makeup action that the user performs when the facial parts are in a first state, and determines a second makeup area on the basis of the makeup action and shapes of the facial parts.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Currently, there are known techniques for simulating the state of a user's face when cosmetics are applied to the user's face. For example, Patent Document 1 describes a technique for superimposing makeup parts representing cosmetics on a user's facial image, and for changing the makeup parts superimposed on the facial image according to the state of the eyes. Specifically, Patent Document 1 describes a technique for transforming makeup parts drawn on an image of closed eyes for an image of open eyes, and superimposing the transformed makeup parts on the image of open eyes.

[0003] In actual makeup, the way in which the appearance of cosmetics applied to the face changes with changes in the state of the eyes varies depending on the type of upper eyelid. For example, even if a person with single eyelids and a person with double eyelids apply eyeshadow to the same area with their eyes closed, the applied area of ​​eyeshadow that is visible when the person opens their eyes differs between the person with single eyelids and the person with double eyelids. In this regard, Patent Document 1 describes that the type of upper eyelid is taken into consideration when changing the makeup parts superimposed on the face image according to the state of the eyes. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 159231 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the method described in Patent Document 1, the way in which the makeup parts are changed does not match the type of upper eyelid, and the state of the face with the cosmetics applied may not be properly simulated. For example, Patent Document 1 does not distinguish between single eyelids and deep double eyelids, but it is considered preferable to distinguish between single eyelids and deep double eyelids. Furthermore, Patent Document 1 describes that when the upper eyelid is single eyelid, the upper eyelid makeup parts drawn on the image with the eyes closed are superimposed on the image with the eyes open without being deformed, but it is considered preferable to deform the upper eyelid makeup parts. For this reason, a technology is desired that can properly simulate the state of the face with the cosmetics applied according to the type of face parts.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an image processing device, an image processing method, and a program that appropriately simulate the state of a face after cosmetics have been applied according to the type of facial features. [Means for solving the problem]

[0007] In order to achieve the above object, the image processing device according to the present invention comprises: Upper eyelid open is a facial image of a user captured when the Opening image An image acquisition means for acquiring an image; The upper eyelid is closed A make-up action performed by the user while in the and the type of upper eyelid; Based on When determining a makeup area on the image of the user with the eyes open, the makeup area is determined so that the makeup area when the user has a single upper eyelid is larger than the makeup area when the user has a deep double eyelid. and a region determining means. Effect of the Invention

[0008] According to the present invention, the state of a face to which cosmetics have been applied can be appropriately simulated according to the type of facial feature. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is an external view of a makeup simulation device according to a first embodiment of the present invention; [Diagram 2]FIG. 1 is a block diagram showing a hardware configuration of an image processing device according to a first embodiment of the present invention. [Diagram 3] FIG. 1 is a block diagram showing a hardware configuration of an instruction device according to a first embodiment of the present invention. [Figure 4] FIG. 1 is a block diagram showing a functional configuration of a makeup simulation device according to a first embodiment of the present invention. [Diagram 5] FIG. 1 is a diagram showing a state in which a user is using an image processing device according to a first embodiment of the present invention; [Figure 6] FIG. 1 is a diagram showing a state in which a user uses a pointing device to operate a touch screen included in an image processing device according to a first embodiment of the present invention. [Figure 7] A diagram showing various lines detected near the eye. [Figure 8] 1 is a diagram showing characteristics of each type of upper eyelid, (A) is a diagram showing characteristics of a single eyelid, (B) is a diagram showing characteristics of a double eyelid, and (C) is a diagram showing characteristics of a deep double eyelid. [Figure 9] A diagram showing a user applying makeup while closing their eyes [Figure 10] 1A is a diagram showing an eye-opening makeup area for a single eyelid, FIG. 1B is a diagram showing an eye-opening makeup area for a double eyelid, and FIG. 1C is a diagram showing an eye-opening makeup area for a deep double eyelid. [Figure 11] 10 is a flowchart showing an image display process executed by the image processing device according to the first embodiment of the present invention; [Figure 12] A flowchart showing the type determination process in FIG. [Figure 13] Flowchart showing the region determination process in FIG. [Figure 14] FIG. 11 is a block diagram showing a functional configuration of an image processing device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0011] (Embodiment 1) First, the configuration of a makeup simulation device 100 according to a first embodiment of the present invention will be described with reference to Fig. 1. The makeup simulation device 100 is a device that simulates on a computer what the user's state will be after applying makeup with cosmetics. By using the makeup simulation device 100, the user can easily practice applying makeup and check in advance what the user's state after applying makeup will be, without using real cosmetics.

[0012] 1, the makeup simulation device 100 includes an image processing device 10 and an instruction device 30. The image processing device 10 is a device that generates and displays an image representing a state after makeup is applied by simulation using a captured image of a user's face. The image processing device 10 has a shape that imitates, for example, a makeup palette.

[0013] Fig. 2 shows a hardware configuration of the image processing device 10. As shown in Fig. 2, the image processing device 10 includes a processor 11, a flash memory 12, a camera 13, a first touch screen 14, a second touch screen 15, and a communication interface 16. These elements included in the image processing device 10 are connected to each other via, for example, a system bus.

[0014] The processor 11 controls the overall operation of the image processing device 10. The processor 11 includes, for example, a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), and a real time clock (RTC). The flash memory 12 stores various types of information.

[0015] The camera 13 captures an image of a subject. The camera 13 includes a lens that collects light emitted from the subject, an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) that receives the collected light and obtains an image of the subject, and an A / D (Analog / Digital) converter that converts imaging data sent from the imaging element as an electrical signal into digital data. As shown in Fig. 1, the camera 13 is disposed near the first touch screen 14 and captures an image of the face of a user who is using the image processing device 10 while looking at the first touch screen 14.

[0016] The first touch screen 14 and the second touch screen 15 are user interfaces that accept various types of information from the user and present various types of information to the user. As shown in FIG. 1, the first touch screen 14 is disposed on the upper surface of the image processing device 10 and displays a captured image obtained by imaging with the camera 13. The second touch screen 15 is disposed on the lower surface of the image processing device 10 and displays a cosmetic image that represents a cosmetic to be virtually applied to the user's face in a makeup simulation. In addition, the second touch screen 15 accepts a user operation that involves contact with the surface of the second touch screen 15.

[0017] The communication interface 16 is a module that communicates with external devices including the instruction device 30 under the control of the processor 11. The communication interface 16 communicates with external devices by wireless communication such as a wireless LAN (Local Area Network) or Bluetooth (registered trademark), or by wired communication such as a USB (Universal Serial Bus) cable.

[0018] The pointing device 30 is a device for selecting cosmetics in a makeup simulation and indicating the position on the face where the selected cosmetics is virtually applied. The pointing device 30 has a stick-like shape and is a component that imitates an application tool used to apply cosmetics to the face, such as a so-called makeup brush. The user holds the pointing device 30 in his / her hand and uses it by pointing the end of the pointing device 30 at the cosmetic image displayed on the image processing device 10 and the user's actual face.

[0019] 3 shows a hardware configuration of the instruction device 30. The instruction device 30 includes a processor 31, a flash memory 32, a pressure sensor 33, and a communication interface 34. These elements included in the instruction device 30 are connected to each other via, for example, a system bus.

[0020] The processor 31 controls the overall operation of the indication device 30. The processor 31 includes, for example, a CPU, a ROM, a RAM, and an RTC. The flash memory 32 stores various information. The pressure sensor 33 detects pressure applied to an end of the indication device 30. That is, the pressure sensor 33 detects contact with the end of the indication device 30. The pressure sensor 33 detects pressure using a method such as a capacitance method or a strain gauge method. The communication interface 34 is a module that communicates with the image processing device 10 under the control of the processor 31. The communication interface 34 communicates with the image processing device 10 by wireless communication such as wireless LAN or Bluetooth (registered trademark), or wired communication such as a USB cable.

[0021] Next, functions of the makeup simulation device 100 will be described with reference to Fig. 4. The image processing device 10 functionally includes an imaging unit 101, an image acquisition unit 102, a display control unit 103, a display unit 104, an operation reception unit 105, a cosmetics selection unit 106, an information receiving unit 107, a contact position identification unit 108, a state determination unit 109, a type determination unit 110, an area determination unit 113, a makeup part generation unit 114, and a makeup part holding unit 115. The instruction device 30 functionally includes a contact detection unit 301 and an information transmission unit 302.

[0022] The image acquisition unit 102 is an example of an image acquisition means of the present invention. The display control unit 103 is an example of a display control means of the present invention. The display unit 104 is an example of a display means of the present invention. The operation reception unit 105 is an example of an operation reception means of the present invention. The contact position identification unit 108 is an example of a contact position identification means of the present invention. The type discrimination unit 110 is an example of a type discrimination means of the present invention. The area determination unit 113 is an example of an area determination means of the present invention.

[0023] The imaging unit 101 captures an image of an area including the user's face, and generates a captured image representing this area. Fig. 5 shows a state in which a user faces a first touch screen 14 provided in the image processing device 10 to simulate makeup using the makeup simulation device 100. The imaging unit 101 captures an image of an area inside a field of view indicated by a dashed line in Fig. 5, specifically, an area including the face of the user facing the first touch screen 14.

[0024] Therefore, the installation position and the direction of the optical axis of the camera 13 for realizing the imaging unit 101 are adjusted so that the face of the user facing the first touch screen 14 is included in the field of view of the camera 13. The imaging unit 101 continuously executes a process of capturing an image of the user's face and generating a captured image while the image processing device 10 is being used by the user. The function of the imaging unit 101 is realized, for example, by the processor 11 and the camera 13 working together.

[0025] The image acquisition unit 102 acquires a captured image generated by the imaging unit 101. This captured image is either a first face image or a second face image. The first face image is a face image of the user captured when the facial parts are in a first state. The second face image is a face image of the user captured when the facial parts are in a second state. This facial part is, for example, a facial part whose state changes significantly among the parts included in the face.

[0026] In this embodiment, the eye to which makeup is applied is the left eye, and the facial feature is the upper eyelid of the left eye. Hereinafter, the upper eyelid of the left eye is simply referred to as the upper eyelid, as appropriate. The second state is a state different from the first state. For example, when the first state is a closed state, the second state is an open state, and when the first state is an open state, the second state is a closed state. In this embodiment, the first state is a closed state, and the second state is an open state. In this embodiment, the first face image is a captured image of the user's face captured when the user has closed his / her left eye, and the second face image is a captured image of the user's face captured when the user has opened his / her left eye. Hereinafter, the first face image is referred to as an eye-closed image, and the second face image is referred to as an eye-open image, as appropriate. The function of the image acquisition unit 102 is realized, for example, by the function of the processor 11.

[0027] The display control unit 103 controls the display of an image by the display unit 104. Specifically, the display control unit 103 displays a captured image acquired by the image acquisition unit 102 in a first display area included in the display unit 104. The first display area is, for example, a display area corresponding to the first touch screen 14. In this way, the display control unit 103 displays the appearance of the user's face in real time. Therefore, the user can simulate makeup application while checking the image of his / her face displayed in the first display area included in the display unit 104 as if it were his / her face reflected in a mirror.

[0028] Furthermore, the display control unit 103 displays a cosmetic image representing the cosmetic in a second display area included in the display unit 104. The second display area is a display area different from the first display area, for example, a display area corresponding to the second touch screen 15. The cosmetic image is an image that visually represents the cosmetic virtually applied to the user's face in the makeup simulation. As shown in FIG. 6, the display control unit 103 displays images representing a plurality of cosmetics as the cosmetic image. Note that FIG. 6 shows a state in which the user operates the second touch screen 15 included in the image processing device 10 using the pointing device 30.

[0029] Here, each of the multiple cosmetics is a cosmetic that differs in at least one of color, gloss, texture, etc. Images representing the multiple cosmetics are displayed in different areas in the second display area. The user can refer to the cosmetic image and select and use the cosmetic for which the user desires to perform a makeup simulation from the multiple cosmetics. The image showing the cosmetic image is stored in the flash memory 12. In this way, the captured image showing the user's face is displayed in the first display area, and the cosmetic image is displayed in the second display area, so that the user can handle the image processing device 10 like a palette of cosmetics. In this embodiment, the cosmetics are eye shadows. The function of the display control unit 103 is realized, for example, by the function of the processor 11.

[0030] Display unit 104 displays images under the control of display control unit 103. Specifically, display unit 104 displays a first face image or a second face image in a first display area, and displays a cosmetic image in a second display area. The function of display unit 104 is realized, for example, by cooperation between processor 11, first touch screen 14, and second touch screen 15.

[0031] The operation receiving unit 105 receives a user operation involving contact with the second display area using the pointing device 30 when the display unit 104 displays a cosmetic image in the second display area. FIG. 6 shows a state in which a user touches the tip of the pointing device 30 held in the hand to the surface of the second touch screen 15. The action of touching the tip of the pointing device 30 to the surface of the second touch screen 15 is an action simulating the action of applying cosmetics stored in a cosmetic palette to the tip of a cosmetic brush. The operation receiving unit 105 transmits, for example, contact position information indicating the contact position on the second display area to the cosmetic selection unit 106. The function of the operation receiving unit 105 is realized, for example, by the processor 11 and the second touch screen 15 working together.

[0032] Cosmetic selection unit 106 selects cosmetics to be used in the makeup simulation based on a user operation received by operation reception unit 105. Specifically, cosmetic selection unit 106 identifies the cosmetics selected by the user based on contact position information received from operation reception unit 105 and arrangement information stored in flash memory 12. The arrangement information is information that indicates the arrangement of each cosmetic product in the second display area, and is information that indicates the coordinates of an area where an image representing the cosmetic product is displayed. The function of cosmetic selection unit 106 is realized, for example, by the function of processor 11.

[0033] The contact detection unit 301 detects whether or not any object is in contact with the tip of the pointing device 30. The contact detection unit 301 detects pressure applied to the tip of the pointing device 30, and determines whether or not there is contact based on the detected pressure. For example, when the contact detection unit 301 detects contact, it transmits contact information indicating the contact time to the information transmission unit 302. The function of the contact detection unit 301 is realized, for example, by the processor 31 and the pressure sensor 33 working together.

[0034] The information transmission unit 302 transmits the contact information generated by the contact detection unit 301 to the image processing device 10. The function of the information transmission unit 302 is realized, for example, by the processor 31 and the communication interface 34 working together.

[0035] The information receiving unit 107 receives contact information from the instruction device 30. The function of the information receiving unit 107 is realized, for example, by the processor 11 and the communication interface 16 working together.

[0036] When the instruction device 30 comes into contact with the user's face while the display unit 104 is displaying the first face image after the operation receiving unit 105 receives a user's operation, the contact position identifying unit 108 identifies a contact position on the user's face where the instruction device 30 has come into contact, based on the first face image. When the instruction device 30 overlaps with the user's face on the first face image while the tip of the instruction device 30 is in contact with some object, the contact position identifying unit 108 determines that the instruction device 30 has come into contact with the user's face.

[0037] The contact position identifying unit 108 can identify a period during which the tip of the pointing device 30 is in contact with some object, based on the contact information supplied from the information receiving unit 107. The contact position identifying unit 108 can also identify the contact position from the position of the pointing device 30 in the first face image acquired during the period during which the pointing device 30 is in contact with the user's face. The function of the contact position identifying unit 108 is realized, for example, by the function of the processor 11.

[0038] The state discrimination unit 109 discriminates the state of a specific facial part of the user's face. Specifically, the state discrimination unit 109 discriminates whether the state of the upper eyelid of the user's left eye is closed or open, in other words, whether the captured image acquired by the image acquisition unit 102 is a first face image or a second face image. The state of the upper eyelid can also be said to be the state of the eyes. Therefore, the state discrimination unit 109 discriminates whether the user's left eye in the captured image is closed or open. The method of discriminating the state of the eyes can be adjusted as appropriate.

[0039] For example, the state discrimination unit 109 identifies the position of a face image in a captured image, identifies the position of a left eye image that is an image in the vicinity of the left eye in the face image, and discriminates whether or not a circular image representing an pupil is present in the left eye image. If the state discrimination unit 109 determines that there is no circular image, it determines that the state of the upper eyelid of the user's left eye is closed. If the state discrimination unit 109 determines that there is a circular image, it determines that the state of the upper eyelid of the user's left eye is open. The function of the state discrimination unit 109 is realized, for example, by the function of the processor 11.

[0040] The type discrimination unit 110 discriminates the type of the user's facial features based on at least one of the first face image and the second face image acquired by the image acquisition unit 102. Specifically, the type discrimination unit 110 discriminates whether the type of the upper eyelid of the user's left eye is single, double, or deep-set eyelid based on at least one of the closed eye image and the open eye image. Note that the type of the upper eyelid being single is synonymous with the upper eyelid being single eyelid, the type of the upper eyelid being double is synonymous with the upper eyelid being double eyelid, and the type of the upper eyelid being deep-set eyelid is synonymous with the upper eyelid being deep-set eyelid. In this embodiment, the type of the upper eyelid is discriminated from the open eye image. The function of the type discrimination unit 110 is realized by, for example, the function of the processor 11. The type discrimination unit 110 includes a line detection unit 111 and a line discrimination unit 112. The line detection unit 111 is an example of the line detection means of the present invention. The line discrimination unit 112 is an example of the line discrimination means of the present invention.

[0041] The line detection unit 111 detects a first line representing the groove of a double eyelid or deep double eyelid from the second face image acquired by the image acquisition unit 102. The line detection unit 111 also detects a second line representing the mucous membrane part of the boundary between the upper eyelid and the eye from the second face image acquired by the image acquisition unit 102. The method of detecting the first and second lines can be adjusted as appropriate. Hereinafter, with reference to FIG. 7, a method of detecting 12 lines from an image around the left eye in an image of eyes open and identifying the first and second lines from the 12 lines will be described.

[0042] As shown in FIG. 7, a total of 12 lines, lines L1-L12, can be detected from the image around the left eye in the image with eyes open. FIG. 7 shows the image around the left eye in the image with eyes open displayed on the first touch screen 14. Line L1 is a line that represents the head of the eyebrow. Line L2 is a line that represents the upper boundary of the eyebrows among the lines that extend from the head of the eyebrow to the brow ridge. Line L3 is a line that represents the upper boundary of the eyebrows among the lines that extend from the brow ridge to the end of the eyebrow. Line L4 is a line that represents the lower boundary of the eyebrows among the lines that extend from the head of the eyebrow to the brow ridge. Line L5 is a line that represents the lower boundary of the eyebrows among the lines that extend from the brow ridge to the end of the eyebrow.

[0043] Line L6 is the line on the mucous membrane side of the frame lines that represent the upper boundary of the eye. Line L6 is the second line. Line L7 is the line on the eyelash side of the frame lines that represent the upper boundary of the eye. Line L8 is the line on the mucous membrane side of the frame lines that represent the lower boundary of the eye. Line L9 is the line on the eyelash side of the frame lines that represent the lower boundary of the eye. Line L10 is a line that represents the groove of a double eyelid or a deep double eyelid. Line L10 is the first line. Line L11 is the upper line of a line called an eyehole that extends from the inner corner of the eye to the outer corner of the eye. Line L12 is the lower line of a line called an eyehole that extends from the inner corner of the eye to the outer corner of the eye.

[0044] It is preferable to determine these 12 lines in order of ease of detection, taking into consideration, for example, the positional relationship with other lines. For example, line L2, which is the line indicating the upper boundary of the eyebrows, and line L4, which is the line indicating the lower boundary of the eyebrows, are considered to be relatively easy to detect. In addition, line L6 and line L7, which are the two lines indicating the upper boundary of the eye, and line L8 and line L9, which are the two lines indicating the lower boundary of the eye, are considered to be relatively easy to detect. Therefore, it is preferable to detect these six lines first.

[0045] Here, the two lines that may be detected in the area between line L4 and line L7 are line L10 and line L11. Therefore, the upper line among the lines present in this area can be detected as line L11, and the lower line among the lines present in this area can be detected as line L10. Note that line L10 is a line that represents a double eyelid or a groove of a deep double eyelid. Therefore, line L10 is not detected from a captured image capturing the face of a user with a single eyelid. The function of the line detection unit 111 is realized by, for example, the function of the processor 11.

[0046] The line discrimination unit 112 discriminates that the user's upper eyelid is a single eyelid when the first line is not detected by the line detection unit 111. Moreover, the line discrimination unit 112 discriminates that the user's upper eyelid is not a single eyelid when the first line is detected by the line detection unit 111. Moreover, the line discrimination unit 112 discriminates whether the user's upper eyelid is a single eyelid, a double eyelid, or a deep double eyelid, based on the first line and the second line detected by the line detection unit 111.

[0047] That is, when the first line is detected, the line discrimination unit 112 discriminates whether the user's upper eyelid is a double eyelid or a deep double eyelid based on the first line and the second line. Specifically, the line discrimination unit 112 discriminates that the user's upper eyelid is a double eyelid when the length in the width direction of the face of the gap area between the first line and the second line is equal to or greater than a threshold. Then, the line discrimination unit 112 discriminates that the user's upper eyelid is a deep double eyelid when this length is less than the threshold.

[0048] Hereinafter, a method for determining the type of upper eyelid based on the gap area will be described with reference to Fig. 8. Fig. 8(A) is a diagram showing the characteristics of a single eyelid. Fig. 8(B) is a diagram showing the characteristics of a double eyelid. Fig. 8(C) is a diagram showing the characteristics of a deep double eyelid.

[0049] As shown in FIG. 8(A), when the upper eyelid of the user's left eye is a single eyelid, the first line, line L6A, is detected, but the second line, line L10, is not detected. As shown in FIG. 8(B), when the upper eyelid of the user's left eye is a double eyelid, the first line, line L6B, and the second line, line L10B, are detected. Also, as shown in FIG. 8(C), when the upper eyelid of the user's left eye is a deep double eyelid, the first line, line L6C, and the second line, line L10C, are detected. Note that lines L6A, L6B, and L6C are collectively referred to as line L6, and lines L10B and L10C are collectively referred to as line L10. Here, the positional relationship between lines L6 and L10 differs between double eyelids and deep double eyelids, and the shape and size of the gap area between lines L6 and L10 are different. Therefore, the line discrimination unit 112 discriminates whether the upper eyelid is a double eyelid or a deep double eyelid based on the shape and size of this gap region.

[0050] In this embodiment, for ease of understanding, the region where the line L6 and the line L10 almost overlap, in other words, the region where the distance between the line L6 and the line L10 is extremely short, is not considered to be a gap region. In other words, strictly speaking, in this embodiment, the gap region is a region between the line L6 and the line L10 where the distance between the line L6 and the line L10 is equal to or greater than a threshold value.

[0051] As shown in FIG. 8B, when the upper eyelid is a double eyelid, the gap region E1 is a crescent-shaped region that is elongated in the width direction of the face. The length of region E1 in the width direction of the face (hereinafter, simply referred to as "length") is W1. Region E1 is a region enclosed when both ends of line L6B and line L10B are connected. Specifically, region E1 is a region enclosed by four lines: a line connecting one end of line L6B and one end of line L10B, line L6B, a line connecting the other end of line L6B and the other end of line L10B, and line L10B.

[0052] On the other hand, as shown in FIG. 8(C), when the upper eyelid is a deep double eyelid, the gap area E2 is an area having a shape like a triangle with one vertex extending in the width direction of the face. The length of the area E2 in the width direction of the face is W2. The area E2 is an area surrounded by the ends of the line L6C and the line L10C. Specifically, the area E2 is an area surrounded by four lines: a line connecting one end of the line L6C and one end of the line L10C, the line L6C, a line connecting the other end of the line L6C and the other end of the line L10C, and the line L10C. It is preferable that the line discrimination unit 112 discriminates whether the upper eyelid is a double eyelid or a deep double eyelid by discriminating whether the detected gap area is similar to the area E1 or the area E2.

[0053] In this embodiment, the fact that the length of the region E1 and the region E2 in the width direction of the face is significantly different is used to determine the upper eyelid. That is, the line determination unit 112 determines that the upper eyelid is a double eyelid when the length of the gap region is equal to or greater than a predetermined threshold, and determines that the upper eyelid is a deep double eyelid when the length of the gap region is less than this threshold. This threshold is, for example, about half the length of the eye. The function of the line determination unit 112 is realized, for example, by the function of the processor 11.

[0054] The region determination unit 113 determines a first makeup region on the first face image based on a makeup action performed by the user while the display unit 104 is displaying the first face image. The first makeup region is a region on which a first makeup part is superimposed. The first makeup part is an image representing a cosmetic product applied to the face in a pseudo manner, and is an image superimposed on the first face image. The makeup action is an action of the user applying the cosmetic product to the face in a pseudo manner using the instruction device 30. The region determination unit 113 determines the first makeup region on the first face image based on the contact position identified by the contact position identification unit 108. For example, the region determination unit 113 determines a region corresponding to a contact position identified due to a makeup action of the user with the eyes closed as a closed-eyes makeup region on which closed-eyes makeup parts are superimposed on the closed-eyes image.

[0055] Furthermore, the area determination unit 113 determines a second makeup area on the second face image based on the makeup action and the type of face part determined by the type determination unit 110. The second makeup area is an area different from the first makeup area, and is an area on which the second makeup part is superimposed. The second makeup part is an image that represents cosmetics applied to the face in a pseudo manner, and is an image that is superimposed on the second face image. For example, the area determination unit 113 determines an area obtained by changing the closed-eyes makeup area according to the type of face part as the open-eyes makeup area.

[0056] Specifically, when type discrimination unit 110 determines that the user's upper eyelid is a single eyelid, region determination unit 113 determines a region having a first size as the second makeup region. When type discrimination unit 110 determines that the user's upper eyelid is not a single eyelid, region determination unit 113 determines a region having a size smaller than the first size as the second makeup region.

[0057] More specifically, when the type discrimination unit 110 discriminates that the user's upper eyelid is a double eyelid, the region determination unit 113 determines a region having a second size smaller than the first size as the second makeup region. When the type discrimination unit 110 discriminates that the user's upper eyelid is a deep double eyelid, the region determination unit 113 determines a region having a third size smaller than the second size as the second makeup region. In other words, even if the size of the makeup region for closed eyes is the same, the size of the makeup region for open eyes decreases in the order of single eyelid, double eyelid, and deep double eyelid.

[0058] Fig. 9 shows how a user applies makeup in a simulated manner with their eyes closed. Fig. 9 shows how area E3, which is a makeup area for closed eyes, is determined from the contact position detected when an image of closed eyes is acquired. Here, area determination unit 113 can determine a makeup area for open eyes based on the makeup area for closed eyes. For example, area determination unit 113 can determine the remaining area after excluding the area corresponding to the eyes from the makeup area for closed eyes as the makeup area for open eyes.

[0059] Here, the region determination unit 113 adjusts the size of the eye-opening makeup region according to the type of upper eyelid. Specifically, the region determination unit 113 adjusts the size of the eye-opening makeup region so that the size of the eye-opening makeup region is largest when the upper eyelid is a single eyelid, is medium when the upper eyelid is a double eyelid, and is smallest when the upper eyelid is a deep double eyelid. For example, the region determination unit 113 reduces the size of the eye-opening makeup region based on the eye-opening makeup region when the upper eyelid is a single eyelid.

[0060] Specifically, the region determination unit 113 determines the remaining region, excluding the region corresponding to the eyes, from region E3, which is the makeup region for closing eyes shown in Fig. 9, as region E4, which is the makeup region for opening eyes for single eyelids shown in Fig. 10(A). Furthermore, the region determination unit 113 determines the region obtained by vertically shrinking region E4, which is the makeup region for opening eyes for single eyelids shown in Fig. 10(A), as region E5, which is the makeup region for opening eyes for double eyelids shown in Fig. 10(B). Then, the region determination unit 113 determines the region obtained by vertically shrinking region E5, which is the makeup region for opening eyes for double eyelids shown in Fig. 10(B), as region E6, which is the makeup region for opening eyes for deep-set double eyelids shown in Fig. 10(C).

[0061] In this way, the region determination unit 113 determines each region so that the region E4 is larger than the region E5, and the region E5 is larger than the region E6. More specifically, the region determination unit 113 determines each region so that the vertical width of the region E4 is larger than the vertical width of the region E5, and the vertical width of the region E5 is larger than the vertical width of the region E6. This is because the overlap of the upper eyelids is shallowest for single eyelids, medium depth for double eyelids, and deep double eyelids is the deepest. In other words, the width of the upper eyelid area hidden when the eyes are opened is narrowest for single eyelids, medium width for double eyelids, and widest for deep double eyelids.

[0062] Furthermore, region determination unit 113 can determine the makeup region for closing the eyes based on the makeup region for opening the eyes, taking into consideration the type of upper eyelid. Specifically, region determination unit 113 determines the makeup region for closing the eyes so that, when the makeup regions for opening the eyes are the same size, the makeup region for closing the eyes for a single eyelid is the smallest, the makeup region for closing the eyes for a double eyelid is of medium size, and the makeup region for closing the eyes for a deep double eyelid is the largest. Specifically, region determination unit 113 can determine the makeup region for closing the eyes by performing the reverse procedure to that for determining the makeup region for opening the eyes.

[0063] For example, the region determination unit 113 determines the region obtained by adding a region corresponding to the eyes to region E4, which is a makeup region for closed eyes for single eyelids, as region E3, which is a makeup region for open eyes. The region determination unit 113 also determines the region obtained by slightly extending region E5, which is a makeup region for closed eyes for double eyelids, in the vertical direction and adding a region corresponding to the eyes to the region, as region E3, which is a makeup region for open eyes. The region determination unit 113 also determines the region obtained by further extending region E6, which is a makeup region for closed eyes for deep double eyelids, in the vertical direction and adding a region corresponding to the eyes to the region, as region E3, which is a makeup region for open eyes. The function of the region determination unit 113 is realized, for example, by the function of the processor 11.

[0064] Makeup part generation unit 114 generates a first makeup part which is a makeup part to be overlaid on the first makeup area, and a second makeup part which is a makeup part to be overlaid on the second makeup area. Makeup part generation unit 114 generates the first makeup part based on the cosmetics selected by cosmetics selection unit 106 and the first makeup area determined by area determination unit 113. Also, makeup part generation unit 114 generates the second makeup part based on the cosmetics selected by cosmetics selection unit 106 and the second makeup area determined by area determination unit 113. The function of makeup part generation unit 114 is realized, for example, by the function of processor 11.

[0065] The makeup parts holding unit 115 holds the first makeup parts and the second makeup parts generated by the makeup parts generation unit 114. The first makeup parts stored in the makeup parts holding unit 115 are superimposed on the first face image by the display control unit 103. The second makeup parts stored in the makeup parts holding unit 115 are superimposed on the second face image by the display control unit 103. A first makeup image is generated by processing the first makeup parts to be superimposed on the first makeup area in the first face image. A second makeup image is generated by processing the second makeup parts to be superimposed on the second makeup area in the second face image. The function of the makeup parts holding unit 115 is realized by, for example, the function of the flash memory 12.

[0066] Next, the image display process executed by the image processing device 10 will be described with reference to the flowchart shown in FIG.

[0067] First, processor 11 included in image processing device 10 displays a cosmetic image (step S101). Specifically, processor 11 displays the cosmetic image on second touch screen 15. Upon completing the process of step S101, processor 11 starts capturing images (step S102). Specifically, processor 11 causes camera 13 to start capturing images, and thereafter, continuously acquires captured images from camera 13.

[0068] When the processor 11 completes the process of step S102, it displays the captured image (step S103). Specifically, the processor 11 causes the captured image acquired by the camera 13 to be displayed on the first touch screen 14. When the processor 11 completes the process of step S103, it executes a type determination process (step S104). The type determination process will be described in detail with reference to FIG. 12.

[0069] First, the processor 11 determines whether or not the captured image is an image of eyes open (step S201). If the processor 11 determines that the captured image is not an image of eyes open (step S201: NO), the type determination process is completed. If the processor 11 determines that the captured image is an image of eyes open (step S201: YES), the processor 11 detects the coordinates of the eyes (step S202). For example, the processor 11 detects a circular image included in the captured image, and detects the position of the detected circular image as the coordinates of the eye.

[0070] When the processor 11 completes the process of step S202, it detects lines around the eyes (step S203). For example, the processor 11 detects the 12 lines shown in FIG. 7. When the processor 11 completes the process of step S203, it determines whether or not the line L10 is detected (step S204). When the processor 11 determines that the line L10 is not detected (step S204: NO), it determines that the upper eyelid is a single eyelid (step S205).

[0071] When the processor 11 determines that the line L10 has been detected (step S204: YES), it identifies a gap region (step S206). When the processor 11 completes the process of step S206, it determines whether the length of the gap region is equal to or greater than a threshold (step S207). When the processor 11 determines that the length of the gap region is equal to or greater than a threshold (step S207: YES), it determines that the upper eyelid is a double eyelid (step S208). When the processor 11 determines that the length of the gap region is not equal to or greater than the threshold (step S207: NO), it determines that the upper eyelid is a deep double eyelid (step S209).

[0072] When the processor 11 completes the processes of steps S205, S208, and S209, it stores the type information (step S210). The type information is information indicating whether the upper eyelid is a single eyelid, a double eyelid, or a deep double eyelid, and is stored in the flash memory 12. When the processor 11 completes the process of step S210, it completes the type determination process.

[0073] When the processor 11 completes the process of step S104, the processor 11 determines whether or not the pointing device 30 is in contact with the second touch screen 15 (step S105). When the processor 11 determines that the pointing device 30 is in contact with the second touch screen 15 (step S105: YES), the processor 11 selects a cosmetic product (step S106). When the processor 11 determines that the pointing device 30 is not in contact with the second touch screen 15 (step S105: NO) or when the processor 11 completes the process of step S106, the processor 11 determines whether or not the pointing device 30 is in contact with a face in the captured image (step S107).

[0074] When the processor 11 determines that the indicator device 30 has contacted the face in the captured image (step S107: YES), the processor 11 determines whether or not contact information has been received (step S108). When the processor 11 determines that contact information has been received (step S108: YES), the processor 11 executes a region determination process (step S109). The region determination process will be described in detail with reference to FIG. 13.

[0075] First, processor 11 determines a basic makeup area (step S301). Specifically, processor 11 determines the basic makeup area from the contact position. When processor 11 completes the process of step S301, processor 11 determines whether the captured image is an image with eyes closed (step S302). When processor 11 determines that the captured image is an image with eyes closed (step S302: YES), processor 11 determines the basic makeup area to be an image with closed eyes makeup (step S303).

[0076] When the processor 11 completes the process of step S303, it determines whether the user's upper eyelids are single eyelids (step S304). If the processor 11 determines that the user's upper eyelids are single eyelids (step S304: YES), it determines a larger makeup area as an eye-opening makeup area (step S305). If the processor 11 determines that the user's upper eyelids are not single eyelids (step S304: NO), it determines whether the user's upper eyelids are double eyelids (step S306).

[0077] When the processor 11 determines that the user's upper eyelid is a double eyelid (step S306: YES), it determines a medium makeup area as the eye-opening makeup area (step S307).When the processor 11 determines that the user's upper eyelid is not a double eyelid (step S306: NO), it determines a small makeup area as the eye-opening makeup area (step S308).

[0078] When the processor 11 determines that the captured image is not an image of closed eyes (step S302: NO), it determines the basic makeup area to be an image of open-eye makeup (step S309). When the processor 11 completes the process of step S309, it determines whether the user's upper eyelids are single eyelids (step S310). When the processor 11 determines that the user's upper eyelids are single eyelids (step S310: YES), it determines a smaller makeup area as the closed-eye makeup area (step S311). When the processor 11 determines that the user's upper eyelids are not single eyelids (step S310: NO), it determines whether the user's upper eyelids are double eyelids (step S312).

[0079] When the processor 11 determines that the user's upper eyelids are double eyelids (step S312: YES), it determines a medium makeup area as the closed-eye makeup area (step S313).When the processor 11 determines that the user's upper eyelids are not double eyelids (step S312: NO), it determines a large makeup area as the closed-eye makeup area (step S314).

[0080] When the processor 11 completes the processes of steps S305, S307, S308, S311, S313, and S314, it stores the region information (step S315). When the processor 11 completes the process of step S315, it completes the region determination process.

[0081] When the processor 11 completes the process of step S109, it generates makeup parts for closing eyes and makeup parts for opening eyes (step S110). When the processor 11 determines that the instruction device 30 is not in contact with the face in the captured image (step S107: NO), when it determines that it has not received contact information (step S108: NO), or when it completes the process of step S110, it determines whether the captured image is an image with the eyes closed (step S111).

[0082] When the processor 11 determines that the captured image is an image of closed eyes (step S111: YES), it displays an image of closed eyes makeup in which closed eyes makeup parts are superimposed on the closed eyes makeup area (step S112). When the processor 11 determines that the captured image is not an image of closed eyes (step S111: NO), it displays an image of open eyes makeup in which open eyes makeup parts are superimposed on the open eyes makeup area (step S113). When the processor 11 completes the process of step S112 or step S113, it returns the process to step S103.

[0083] According to this embodiment, the state of the face with cosmetics applied can be appropriately simulated according to the type of facial part. In this embodiment, even if the makeup area is the same size when applying makeup, if the state of the eyes changes, the size of the makeup area is set appropriately according to the characteristics of the eyes (type of upper eyelid), so that a more realistic makeup simulation can be realized. That is, in this embodiment, the makeup area for closed eyes and the makeup area for open eyes are set to appropriate sizes according to whether the upper eyelid is single eyelid or not, so that even if the state of the user's eyes changes during the makeup simulation, a natural makeup image can be displayed. Furthermore, according to this embodiment, it is possible to easily detect whether the upper eyelid is single eyelid or not by detecting the first line.

[0084] According to the present embodiment, the makeup area for closing the eyes and the makeup area for opening the eyes are set to appropriate sizes according to whether the upper eyelid is a single eyelid, a double eyelid, or a deep double eyelid, so that even if the state of the user's eyes changes during the makeup simulation, a natural makeup image can be displayed. According to the present embodiment, by detecting the first line and the second line, it is possible to easily detect whether the upper eyelid is a single eyelid, a double eyelid, or a deep double eyelid. According to the present embodiment, by determining the gap area between the first line and the second line, it is possible to easily detect whether the upper eyelid is a single eyelid, a double eyelid, or a deep double eyelid. According to the present embodiment, it is possible to easily detect whether the upper eyelid is a single eyelid, a double eyelid, or a deep double eyelid. According to the present embodiment, it is possible to perform a makeup simulation using an indicator.

[0085] (Embodiment 2) In the first embodiment, a makeup simulation device 100 has been described that includes the image processing device 10 and the instruction device 30. The image processing device 10 can also function as a makeup simulation device by itself.

[0086] Hereinafter, functions of the image processing device 10 according to this embodiment will be described with reference to Fig. 14. Functionally, the image processing device 10 according to this embodiment does not include an information receiving unit 107 and a contact position identifying unit 108, but includes an application operation receiving unit 116.

[0087] The application operation receiving unit 116 receives a simulated application operation of a cosmetic product by a user. This application operation is not an actual contact operation on the user's face, but an operation on the captured image displayed on the first touch screen 14. The function of the application operation receiving unit 116 is realized, for example, by the processor 11 and the first touch screen 14 working together.

[0088] Region determination section 113 determines a first makeup region and a second makeup region based on the application operation accepted by application operation accepting section 116. Specifically, region determination section 113 determines the first makeup region and the second makeup region based on information indicating coordinates in the captured image specified by the application operation.

[0089] According to this embodiment, the image processing device 10 alone can appropriately simulate the state of a face to which cosmetics have been applied in accordance with the shapes of facial features.

[0090] (Modification) Although the embodiment of the present invention has been described above, various modifications and applications are possible in carrying out the present invention.

[0091] In the present invention, it is arbitrary which parts of the configurations, functions, and operations described in the above embodiments are adopted. In addition to the above-mentioned configurations, functions, and operations, further configurations, functions, and operations may be adopted in the present invention. The above-mentioned embodiments can be freely combined as appropriate. The number of components described in the above-mentioned embodiments can be adjusted as appropriate. In addition, it is needless to say that the materials, sizes, electrical characteristics, etc. that can be adopted in the present invention are not limited to those shown in the above-mentioned embodiments.

[0092] In the above embodiment, an example has been described in which the processor 11 executes various processes by the CPU executing a program stored in the ROM. In the present invention, instead of the processor 11, dedicated hardware such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), various control circuits, etc. may be provided, and the dedicated hardware may execute various processes. In this case, the functions of each unit may be realized by individual hardware, or the functions of each unit may be realized together by a single piece of hardware. Also, some of the functions of each unit may be realized by dedicated hardware, and the other parts may be realized by software or firmware.

[0093] In addition to being able to provide an image processing device that is equipped with the configuration for realizing the functions according to the present invention, an existing information processing device or the like can be made to function as the image processing device according to the present invention by applying a program. That is, by applying a program for realizing each functional configuration of the image processing device 10 exemplified in the above embodiment so that it can be executed by a CPU or the like that controls an existing information processing device, it can be made to function as the image processing device according to the present invention.

[0094] Moreover, the method of applying such a program is arbitrary. For example, the program can be applied by storing it in a computer-readable storage medium such as a flexible disk, a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disc)-ROM, or a memory card. Furthermore, the program can be superimposed on a carrier wave and applied via a communication medium such as the Internet. For example, the program can be posted and distributed on a bulletin board (BBS: Bulletin Board System) on a communication network. Then, the program can be started and executed under the control of an OS (Operating System) in the same way as other application programs, thereby enabling the above processing to be performed.

[0095] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the specific embodiment, and the present invention includes the inventions described in the claims and their equivalents. The inventions described in the original claims of this application are listed below. (Appendix 1) an image acquisition means for acquiring a first face image, which is a face image of a user captured when a face part is in a first state, and a second face image, which is a face image of the user captured when the face part is in a second state different from the first state; a display means for displaying the face image of the user acquired by the image acquisition means; a type discrimination means for discriminating a type of the facial feature of the user based on at least one of the first face image and the second face image acquired by the image acquisition means; a region determining means for determining a first makeup area on the first face image based on a makeup operation performed by the user when the facial feature is in the first state, and for determining a second makeup area, the second makeup area having a size different from that of the first makeup area, on the second face image based on the makeup operation and the type of the facial feature determined by the type determining means. Image processing device. (Appendix 2) a display control means for causing the display means to display a first makeup image in which the first makeup area on the first facial image has been processed when the image acquisition means acquires the first facial image after the makeup application is performed, and for causing the display means to display a second makeup image in which the second makeup area on the second facial image has been processed when the image acquisition means acquires the second facial image after the makeup application is performed, 2. An image processing device as described in claim 1. (Appendix 3) the first makeup image is an image in which makeup parts expressing a state in which the cosmetic product designated by the makeup action is applied to the face of the user are superimposed on the first makeup area on the first face image, the second makeup image is an image in which the makeup parts are superimposed on the second makeup area on the second face image; 3. An image processing device as described in claim 2. (Appendix 4) The facial feature is an upper eyelid, the first state being a closed state and the second state being an open state; The type determination means determines whether the upper eyelid of the user is a single eyelid, The area determination means determines an area having a first size as the second makeup area when the type determination means determines that the upper eyelid of the user is a single eyelid, and determines an area having a size smaller than the first size as the second makeup area when the type determination means determines that the upper eyelid of the user is not a single eyelid. 4. The image processing device according to claim 1 , (Appendix 5) The type determination means is a line detection means for detecting a first line representing a groove of a double eyelid or a deep double eyelid from the second face image acquired by the image acquisition means; and a line discrimination means for discriminating that the upper eyelid of the user is a single eyelid when the first line is not detected by the line detection means, and for discriminating that the upper eyelid of the user is not a single eyelid when the first line is detected by the line detection means. 5. An image processing device as described in claim 4. (Appendix 6) The type discrimination means discriminates whether the upper eyelid of the user is a single eyelid, a double eyelid, or a deep double eyelid, The area determination means determines an area having the first size as the second makeup area when the type determination means determines that the upper eyelid of the user is a single eyelid, determines an area having a second size smaller than the first size as the second makeup area when the type determination means determines that the upper eyelid of the user is a double eyelid, and determines an area having a third size smaller than the second size as the second makeup area when the type determination means determines that the upper eyelid of the user is a deep double eyelid. 5. An image processing device as described in claim 4. (Appendix 7) The type determination means is a line detection means for detecting a first line representing a sulcus of a double eyelid or a deep double eyelid and a second line representing a mucous membrane portion of a boundary between an upper eyelid and an eye from the second face image acquired by the image acquisition means; and a line discrimination means for discriminating whether the upper eyelid of the user is a single eyelid, a double eyelid, or a deep double eyelid based on the first line and the second line detected by the line detection means. 7. The image processing device according to claim 6. (Appendix 8) The line discrimination means discriminates that the upper eyelid of the user is a single eyelid when the first line is not detected by the line detection means, discriminates that the upper eyelid of the user is a double eyelid when the length in the width direction of the face of the gap area between the first line and the second line is equal to or greater than a threshold, and discriminates that the upper eyelid of the user is a deep double eyelid when the length is less than the threshold. 8. The image processing device according to claim 7. (Appendix 9) the display control means causes the display means to display the first face image or the second face image in a first display area, and to display a cosmetic image representing a cosmetic product in a second display area different from the first display area; an operation receiving means for receiving a user operation involving contact with the second display area using a pointer while the cosmetic image is being displayed in the second display area by the display means; a contact position specifying means for specifying, when the indicator comes into contact with the face of the user while the display means is displaying the first face image after the operation accepting means has accepted the user operation, a contact position on the user's face where the indicator has come into contact based on the first face image, the area determining means determines an area on the first face image corresponding to the contact position identified by the contact position identifying means as the first makeup area, and determines an area on the second face image corresponding to the contact position and a shape of the facial feature as the second makeup area; the first makeup image is an image in which makeup parts expressing a state in which the cosmetic product is applied to the face of the user are superimposed on the first makeup area on the first face image, the second makeup image is an image in which the makeup parts are superimposed on the second makeup area on the second face image; 3. An image processing device as described in claim 2. (Appendix 10) acquiring a first face image, which is a face image of a user captured when a face part is in a first state, and a second face image, which is a face image of the user captured when the face part is in a second state different from the first state; Displaying a facial image of the user; determining a type of the facial feature of the user based on at least one of the first face image and the second face image; determining a first makeup area on the first face image based on a makeup operation performed by the user when the facial feature is in the first state, and determining a second makeup area on the second face image, the second makeup area having a size different from that of the first makeup area, based on the makeup operation and a shape of the facial feature; Image processing methods. (Appendix 11) Computer, an image acquisition means for acquiring a first face image, which is a face image of a user captured when a face part is in a first state, and a second face image, which is a face image of the user captured when the face part is in a second state different from the first state; a display control means for displaying the face image of the user acquired by the image acquisition means on a display device; a type discrimination means for discriminating a type of the facial feature of the user based on at least one of the first face image and the second face image acquired by the image acquisition means; a region determining means for determining a first makeup area on the first face image based on a makeup application performed by the user when the facial feature is in the first state, and for determining a second makeup area on the second face image, the second makeup area having a size different from that of the first makeup area, based on the makeup application and the type of the facial feature determined by the type determining means; program. [Explanation of symbols]

[0096] 10...image processing device, 11...processor, 12...flash memory, 13...camera, 14...first touch screen, 15...second touch screen, 16...communication interface, 30...instruction device, 31...processor, 32...flash memory, 33...pressure sensor, 34...communication interface, 100...makeup simulation device, 101...imaging unit, 102...image acquisition unit, 103...display control unit, 104...display unit, 105...operation reception unit, 106...cosmetic selection unit, 107...information information receiving unit, 108...contact position identifying unit, 109...state determining unit, 110...type determining unit, 111...line detecting unit, 112...line determining unit, 113...area determining unit, 114...make-up part generating unit, 115...make-up part holding unit, 116...application operation receiving unit, 301...contact detecting unit, 302...information transmitting unit, L1, L2, L3, L4, L5, L6, L6A, L6B, L6C, L7, L8, L9, L10, L10B, L10C, L11, L12...lines, E1, E2, E3, E4, E5, E6...areas

Claims

1. an image acquisition means for acquiring an open-eye image, which is a facial image of a user captured when the upper eyelid is open; and an area determining means for determining a makeup area on the eye-open image of the user based on a makeup action performed by the user when the upper eyelid is closed and a type of the upper eyelid, so that the makeup area when the upper eyelid of the user is a single eyelid is larger than the makeup area when the upper eyelid of the user is a deep double eyelid. Image processing device.

2. A type discrimination means for discriminating a type of the upper eyelid of the user based on the eye-open image acquired by the image acquisition means, The area determining means determines the makeup area based on the makeup operation and the type of the upper eyelid determined by the type determining means. The image processing device according to claim 1 .

3. An open-eye image is obtained, which is a facial image of the user captured when the upper eyelid is open; When determining a makeup area on the eye-open image of the user based on the makeup action performed by the user when the upper eyelid is closed and the type of the upper eyelid, the makeup area is determined so that a makeup area when the upper eyelid of the user is a single eyelid is larger than a makeup area when the upper eyelid of the user is a deep double eyelid. Image processing methods.

4. Computer, An image acquisition means for acquiring an open-eye image, which is a facial image of the user captured when the upper eyelid is open; and functioning as an area determining means for determining, when determining a makeup area on the eye-open image of the user based on the makeup action performed by the user when the upper eyelid is closed and the type of the upper eyelid, a makeup area when the upper eyelid of the user is a single eyelid is larger than a makeup area when the upper eyelid of the user is a deep double eyelid. program.

Images