Cosmetics proposal device, cosmetics proposal method, cosmetics proposal program, and virtual cosmetics image provision method

Abstract

To provide a cosmetics proposal device capable of proposing makeup bases and foundations which enable skin color desired by a user.SOLUTION: A photographic image acquisition part 2 acquires a photographic image obtained by photographing a skin to which makeup is applied for a user. A makeup base color selection part selects a color to be recommended to a user 30 from a product of a makeup base of m colors held by a product information / color information holding part 4 on the basis of a result of a questionnaire to the user 30. A foundation color proposal part 7 proposes a color to be recommended to the user from foundation products of n colors held by the product information / color information holding part 4 on the basis of the photographic image acquired by the photographic image acquisition part 2. A foundation color selection part 8 selects the color proposed by the foundation color proposal part 7. A communication part 1 provides, as a product information provision part, product information of the makeup base of the color selected by the makeup base color selection part 6 and product information of the foundation of the color selected by the foundation color selection part 8 to the user 30.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a cosmetics suggestion device, a cosmetics suggestion method, a cosmetics suggestion program, and a virtual makeup image providing method. [Background technology]

[0002] Patent Document 1 describes recommending a foundation suitable for the skin color of a user, while Patent Document 2 describes simulating the state when foundation is applied to the user's face. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-149051 A [Patent Document 2] Patent Publication No. 2021-77117 Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, a makeup base is applied to the face before applying foundation, so even if only foundation suitable for the user's skin color is recommended to the user and a simulation is performed of the user's face with only foundation applied, it is not necessarily possible to achieve the skin color desired by the user.

[0005] One or more embodiments aim to provide a cosmetics suggestion device, a cosmetics suggestion method, a cosmetics suggestion program, and a virtual makeup image providing method that can suggest makeup bases and foundations that will achieve the user's desired skin color by applying the makeup bases and foundations to the skin. [Means for solving the problem]

[0006] A first aspect of one or more embodiments includes a product information and color information storage unit that stores product information and color information of m colors of makeup base products and product information and color information of n colors of foundation products, where m and n are integers of 2 or more; a captured image acquisition unit that acquires a captured image of a user's skin on which makeup is applied; a makeup base color selection unit that selects a color to be recommended to the user from the m colors of makeup base products stored in the product information and color information storage unit based on the results of a questionnaire regarding the user's skin administered to the user; and a makeup base color selection unit that selects the product information and color information of the n colors of foundation products stored in the product information and color information storage unit based on the captured image acquired by the captured image acquisition unit. a foundation color suggestion unit that suggests a color recommended to the user from among the n colors of foundation products stored in the product information and color information storage unit if the user accepts the foundation of the suggested color, and a product information provision unit that provides the user with product information of the makeup base of the color selected by the makeup base color selection unit and product information of the foundation of the color selected by the foundation color selection unit based on the product information of the makeup base and the product information of the foundation stored in the product information and color information storage unit.

[0007] A second aspect of one or more embodiments provides a cosmetics suggestion method, comprising: storing product information and color information of makeup base products of m colors, and product information and color information of foundation products of n colors, where m and n are integers of 2 or more, in a product information and color information storage unit; acquiring a captured image of a user's skin on which makeup is applied; selecting a color to be recommended to the user from the m colors of makeup base products stored in the product information and color information storage unit based on a result of a questionnaire regarding the user's skin administered to the user; proposing a color to be recommended to the user from the n colors of foundation products stored in the product information and color information storage unit based on the captured image; and, if the user accepts the proposed color of foundation, selecting a proposed color from the n colors of foundation products stored in the product information and color information storage unit; and providing the user with product information of the makeup base of the selected color and product information of the foundation of the selected color based on the product information of the makeup base and the product information of the foundation stored in the product information and color information storage unit.

[0008] A third aspect of one or more embodiments provides a cosmetics suggestion program that causes a computer to execute the steps of acquiring a captured image of a user's skin on which makeup is to be applied, selecting a color to recommend to the user from among m colors of makeup base products stored in a product information and color information storage unit, where m is an integer of 2 or more, based on the results of a questionnaire regarding the user's skin administered to the user, proposing a color to recommend to the user from among n colors of foundation products stored in the product information and color information storage unit, where n is an integer of 2 or more, based on the captured image, and if the user accepts the proposed color of foundation, selecting the proposed color from among the n colors of foundation products stored in the product information and color information storage unit, and providing to the user product information of the makeup base of the selected color and product information of the foundation of the selected color based on the product information of the makeup base and the product information of the foundation stored in the product information and color information storage unit.

[0009] A fourth aspect of one or more embodiments includes a product information storage unit that stores product information of a makeup base product of m colors and product information of a foundation product of n colors, where m and n are integers of 2 or more; a combined color information storage unit that stores color information indicating a combination color of the makeup base and the foundation of (m×n) colors, each of which is obtained by superimposing each color of the n colors of foundation products on each color of the m colors of makeup base products; a photographed image acquisition unit that acquires a photographed image of a user's skin on which makeup is applied; and a combination color information storage unit that acquires color information indicating a combination color of the makeup base and the foundation of (m×n) colors stored in the combined color information storage unit based on the photographed image acquired by the photographed image acquisition unit. a combination color suggestion unit that suggests one of the combination colors with a foundation as a recommended combination color to the user; a combination color selection unit that, if the user accepts the suggested combination color, selects the combination color suggested by the combination color suggestion unit from the combination color stored in the combination color information storage unit; and a product information provision unit that provides the user with product information on the makeup base and the foundation that correspond to the combination color selected by the combination color selection unit, based on the product information on the makeup base and the product information on the foundation stored in the product information storage unit.

[0010] A fifth aspect of one or more embodiments provides a cosmetics suggestion method, comprising: storing, in a product information storage unit, product information on a makeup base product of m colors and product information on a foundation product of n colors, where m and n are integers of 2 or more; storing, in a combination color information storage unit, color information indicating combinations of the makeup base and the foundation of (m×n) colors, each of which is obtained by superimposing each of the n colors of the foundation products on each of the m colors of the makeup base products; acquiring a photographed image of a user's skin on which makeup is applied; proposing, based on the acquired photographed image, one of the (m×n) color combinations of the makeup base and the foundation stored in the combination color information storage unit as a recommended color combination to the user; and, if the user accepts the proposed color combination, selecting the proposed color combination from the combination color information stored in the combination color information storage unit; and providing, to the user, the product information on the makeup base and the product information of the foundation corresponding to the selected color combination based on the product information on the makeup base and the product information on the foundation stored in the product information storage unit.

[0011] A sixth aspect of one or more embodiments provides a cosmetics suggestion program that causes a computer to execute the steps of acquiring a captured image of a user's skin on which makeup is to be applied, suggesting to the user, based on the acquired captured image, one of (m×n) color combinations of makeup base and foundation stored in a color combination information storage unit, where m and n are integers of 2 or greater, as a recommended color combination to the user, and if the user accepts the suggested color combination, selecting the suggested color combination from the color combinations stored in the color combination information storage unit, and providing to the user product information of the makeup base and product information of the foundation corresponding to the selected color combination based on product information of m colors of makeup base and product information of n colors of foundation stored in a product information storage unit.

[0012] A seventh aspect of one or more embodiments provides a virtual makeup image providing method which acquires a captured image of a user's skin to which makeup is to be applied, generates a color correction value for correcting the color of the skin when a selected makeup base and a selected foundation are applied to the skin in that order, corrects the color of the skin in the captured image with the color correction value to generate a virtual makeup image, and provides the virtual makeup image to the user. Effect of the Invention

[0013] According to one or more embodiments of the cosmetics suggestion device, cosmetics suggestion method, cosmetics suggestion program, and virtual makeup image providing method, it is possible to suggest a makeup base and foundation that can be applied to the skin to achieve the user's desired skin color. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing a cosmetics suggestion device according to a first embodiment. [Diagram 2] FIG. 2 is a conceptual diagram showing in three dimensions a number of foundation colors with different hues, values, and saturations. [Diagram 3] FIG. 3 is a conceptual diagram showing, in a plan view, a number of foundation colors with different hues, values, and saturations. [Figure 4] FIG. 4 is a diagram showing examples of suggested makeup bases suggested by a makeup base color selection unit when a questionnaire conducted by the questionnaire conducting unit of the cosmetics suggestion device in the first embodiment is a questionnaire inquiring about the user's skin concerns. [Diagram 5] FIG. 5 is a diagram showing examples of suggested makeup bases suggested by a makeup base color selection unit when a questionnaire conducted by the questionnaire conducting unit of the cosmetics suggestion device in the first embodiment is a questionnaire inquiring about the user's ideal skin type. [Figure 6A] FIG. 6A is a partial flowchart showing processing executed by the cosmetics suggestion device according to the first embodiment when a cosmetic product suggestion is requested by a user terminal. [Figure 6B] FIG. 6B is a partial flowchart following FIG. 6A, showing the processing executed by the cosmetics suggestion device according to the first embodiment when a cosmetic product suggestion is requested by a user terminal. [Figure 7] FIG. 7 is a diagram showing an example of an image including questionnaire text and check boxes displayed on the display unit of a user terminal. [Figure 8] FIG. 8 is a diagram showing an example of an image displayed on the display unit of the user terminal, instructing the user to photograph the face of the user. [Figure 9] FIG. 9 is a diagram showing an example of a proposed makeup base and foundation displayed on the display unit of a user terminal. [Figure 10] FIG. 10 is a diagram showing an example of a virtual makeup image displayed on a display unit of a user terminal. [Figure 11] FIG. 11 is a block diagram showing a cosmetics suggestion device according to the second embodiment. [Figure 12] FIG. 12 is a flowchart showing the process executed by the cosmetics suggestion device according to the second embodiment when a cosmetic product suggestion is requested by a user terminal. [Figure 13] FIG. 13 is a diagram showing an example of a proposed combination of a makeup base and a foundation displayed on the display unit of a user terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, a cosmetics suggestion device, a cosmetics suggestion method, a cosmetics suggestion program, and a virtual makeup image providing method according to each embodiment will be described with reference to the accompanying drawings.

[0016] First Embodiment 1, a product proposal server 101, a user terminal 20, and a product sales and delivery management server 40 are connected via a network 50. The product proposal server 101 constitutes a cosmetics proposal device according to the first embodiment, and executes the cosmetics proposal method according to the first embodiment to propose cosmetics to a user 30. Typically, the network 50 is the Internet.

[0017] The product proposal server 101 and the product sales and delivery management server 40 may be configured with computer devices. The product proposal server 101 may be configured with a non-transitory storage medium that stores the cosmetics proposal program according to the first embodiment, and a central processing unit (hereinafter, referred to as CPU) that executes the cosmetics proposal program. The user terminal 20 may be configured with computer devices such as a smartphone, a tablet terminal, or a personal computer.

[0018] The product proposal server 101 includes a communication unit 1, a photographed image acquisition unit 2, a numerical conversion unit 3, a product information / color information storage unit 4, a questionnaire implementation unit 5, a makeup base color selection unit 6, a foundation color proposal unit 7, a foundation color selection unit 8, a combination color setting unit 9, a color correction value generation unit 10, and a makeup image generation unit 11. The user terminal 20 includes a central processing unit 21 (hereinafter, CPU 21), a non-transitory storage medium 22, a camera 23, a display unit 24, an operation unit 25, and a communication unit 26. The communication units 1 and 26 communicate with each other via a network 50 using the Internet Protocol. The non-transitory storage medium 22 is any storage medium such as a semiconductor memory. The display unit 24 is any display panel such as a liquid crystal panel or an organic EL panel. The operation unit 25 is an operation button or a touch pad provided integrally with the display unit 24.

[0019] The operation of each component in the product proposal server 101 will be described. The product information and color information storage unit 4 stores product information and color information for m-color makeup base products and product information and color information for n-color foundation products, where m and n are integers of 2 or more. m is, for example, 10, and n is, for example, 27. The product information includes at least the product number or product name, and may also include the price of the product. The color information includes the color name and a numerical value indicating the color. The numerical value indicating the color is, for example, L * a * b * Color space L * Value, a * Value, b * These values ​​are based on Lab values ​​(hereinafter referred to as Lab values).

[0020] A makeup base is a cosmetic that is applied before applying foundation to prepare the skin. A makeup base plays a role in improving the adhesion of foundation, correcting uneven skin tone, making pores less noticeable, making skin blemishes and freckles less noticeable, making dull skin such as yellowing less noticeable, suppressing excess sebum, and protecting the skin from ultraviolet rays and dryness. Foundations can be liquid, powdery, cushion, or cream. Foundations play a role in preparing the surface of the skin to make it look beautiful, making skin blemishes and freckles less noticeable, and protecting the skin from ultraviolet rays and dryness.

[0021] As shown in Figures 2 and 3, if the foundation is made in 27 colors, it is preferable that the 27 colors have different hues, brightness, and saturation. In Figures 2 and 3, F conceptually indicates the hue, brightness, and saturation of each foundation color. It is preferable that the 27 colors are set so that the hue, brightness, and saturation are equally spaced. As shown in Figure 3, the foundation may be made in 21 colors enclosed by a dashed line. Since colors suitable for people's skin vary from country to country, it is preferable to select a color suitable for the skin of people from the 27 colors. The 21 colors enclosed by a dashed line are colors suitable for Japanese skin.

[0022] When a user 30 requests a cosmetic product proposal from the product proposal server 101, the user 30 transmits an image of himself / herself taken by the camera 23 to the product proposal server 101. The user 30 may take an image of the skin on which makeup is to be applied, such as the face. Here, the skin on which makeup is to be applied is the face. The communication unit 1 receives the captured image transmitted from the user terminal 20, and the captured image acquisition unit 2 acquires and stores the captured image. The numerical value conversion unit 3 converts the captured image into numerical values ​​and supplies them to the foundation color proposal unit 7. The numerical value conversion unit 3 may convert the RGB values ​​of the skin area in the captured image into Lab values ​​and generate a numerical value based on the Lab values. The skin area may be a specific area, such as the cheek.

[0023] The questionnaire implementation unit 5 conducts a questionnaire regarding the user 30's skin. The questionnaire regarding the user 30's skin is preferably a questionnaire inquiring about skin concerns of the user 30, or a questionnaire inquiring about the type of skin that the user 30 desires as an ideal skin. The questionnaire implementation unit 5 acquires the results of the questionnaire in which the user 30 answers the questionnaire from the user terminal 20. The makeup base color selection unit 6 selects and proposes a color recommended to the user 30 from among the m colors of makeup base products stored in the product information / color information storage unit 4 based on the results of the questionnaire. The makeup base color selection unit 6 transmits the product information and color name of the makeup base of the selected color to the user terminal 20 via the communication unit 1.

[0024] FIG. 4 shows examples of suggested makeup bases suggested by the makeup base color selection unit 6 when the questionnaire conducted by the questionnaire implementation unit 5 is a questionnaire inquiring about the skin troubles of the user 30. The suggested makeup base colors for the user 30 who answered "I am concerned about age spots and freckles" are pink beige, yellow, and orange as suggested makeup bases #1 to #3. The suggested makeup base colors for the user 30 who answered "I have redness" are blue, yellow, and green as suggested makeup bases #1 to #3. The suggested makeup base colors for the user 30 who answered "I have a yellowish dullness" are purple and blue as suggested makeup bases #1 and #2. The suggested makeup base colors for the user 30 who answered "I have a bad complexion" are pink and orange as suggested makeup bases #1 and #2. The suggested makeup base color for the user 30 who answered "My pores are noticeable" is light beige as suggested makeup base #1.

[0025] In the example shown in Fig. 4, three or two makeup base colors are suggested as the makeup base colors depending on the options in the questionnaire, but one makeup base color may be suggested for all options. Also, multiple makeup base colors may be suggested for all options. When there are multiple colors of suggested makeup base, the makeup base color selection unit 6 should select the makeup base color that is most suitable for the skin of the user 30 based on the numerical values ​​of the captured image supplied by the numerical value conversion unit 3.

[0026] 5 shows examples of makeup bases suggested by the makeup base color selection unit 6 when the questionnaire conducted by the questionnaire implementation unit 5 is a questionnaire inquiring about the ideal skin desired by the user 30. The suggested makeup base colors for the user 30 who answered "I want to have healthy skin," "I want to have a rosy complexion," and "I want to have bright skin" are orange, pink, and light beige, respectively.

[0027] The foundation color proposal unit 7 proposes a color to be recommended to the user 30 from among the n colors of foundation products stored in the product information / color information storage unit 4 based on the numerical values ​​of the captured image acquired by the captured image acquisition unit 2. The foundation color proposal unit 7 may select the color of foundation having a numerical value closest to the numerical value supplied by the numerical value conversion unit 3. The foundation color selection unit 8 selects the color proposed by the foundation color proposal unit 7 as an initial value. If the user 30 accepts the proposed foundation color, the foundation color selection unit 8 selects the product information and color name of the foundation color proposed by the foundation color proposal unit 7 and transmits them to the user terminal 20 via the communication unit 1.

[0028] The user 30 may be configured to be able to select a color other than the color of foundation proposed by the foundation color proposal unit 7. The user 30 can instruct the foundation color selection unit 8 to select a color other than the color of foundation proposed by the foundation color proposal unit 7 through the user terminal 20. When instructed to select a different color, the foundation color selection unit 8 selects a foundation of the color specified by the user 30 from among the n colors of foundation products stored in the product information / color information storage unit 4. In this way, if the user 30 does not accept the proposed color of foundation and changes it to another color, the foundation color selection unit 8 selects the product information and color name of the foundation of the color changed by the user 30 instead of the color proposed by the foundation color proposal unit 7, and transmits them to the user terminal 20 via the communication unit 1.

[0029] The combination color setting unit 9 sets a combination color of the makeup base color selected by the makeup base color selection unit 6 and the foundation color selected by the foundation color selection unit 8. The color correction value generation unit 10 generates a color correction value corresponding to the combination color set in the combination color setting unit 9. The color correction value is a correction value that corrects the skin color of the captured image stored in the captured image acquisition unit 2. Specifically, the color correction value is a correction value that increases or decreases each RGB value of the captured image.

[0030] Assume that a makeup base of a color selected by the makeup base color selection unit 6 and a foundation of a color selected by the foundation color selection unit 8 are applied in this order to the face of the user 30. The color correction value corresponds to the difference between the color of the face of the user 30 before applying the makeup base and foundation and the color of the face of the user 30 after applying the makeup base and foundation. The makeup image generation unit 11 generates a photographed image in which the skin color of the photographed image stored in the photographed image acquisition unit 2 is corrected by the color correction value supplied by the color correction value generation unit 10, and transmits the photographed image to the user terminal 20 via the communication unit 1. The photographed image generated by the makeup image generation unit 11 in which the skin color of the photographed image is corrected by the color correction value is a virtual makeup image in which the makeup base and foundation are applied to the face of the user 30.

[0031] The makeup image generating unit 11 may generate a virtual makeup image in which the skin color of only a specific area of ​​the face of the user 30, such as the cheeks, is corrected with the color correction value supplied from the color correction value generating unit 10.

[0032] The communication unit 1 functions as a product information providing unit that provides the user 30 with product information on the makeup base of the color selected by the makeup base color selecting unit 6 and product information on the foundation of the color selected by the foundation color selecting unit 8. The makeup base color selecting unit 6 and the foundation color selecting unit 8 provide the user 30 with product information on the selected color based on the product information on the makeup base and the product information on the foundation stored in the product information / color information storage unit 4, respectively. The communication unit 1 also functions as a makeup image providing unit that provides the user 30 with a virtual makeup image.

[0033] When the user 30 completes the procedure to purchase the makeup base and foundation proposed by the product proposal server 101, the user terminal 20 submits an application for purchase of the makeup base and foundation to the product sales and delivery management server 40 via the network 50. The product sales and delivery management server 40 settles the payment for the proposed makeup base and foundation, and arranges for the makeup base and foundation to be delivered to the user 30.

[0034] The product proposal server 101 executes the following virtual makeup image providing method. The product proposal server 101 (captured image acquisition unit 2) acquires a captured image of the skin of the user 30 to which makeup is to be applied. The product proposal server 101 (color correction value generation unit 10) generates a color correction value for correcting the skin color when a selected makeup base and a selected foundation are applied, in that order, to the skin of the user 30. The product proposal server 101 (makeup image generation unit 11) corrects the skin color in the captured image with the color correction value to generate a virtual makeup image. The product proposal server 101 (communication unit 1) provides the virtual makeup image to the user 30.

[0035] The processing executed by product proposal server 101 when a request for a cosmetic product suggestion is made by user terminal 20 will be further described using the flowcharts shown in Figures 6A and 6B and Figures 7 to 10. Figures 7 to 10 show examples of images displayed on display unit 24 when CPU 21 executes an application program stored in non-transitory storage medium 22 in response to the processing of cosmetic product suggestion by product proposal server 101. Figures 6A and 6B also show a cosmetic product suggestion method executed by product proposal server 101, and processing executed by the CPU when product proposal server 101 is configured with a CPU that executes a cosmetic product suggestion program.

[0036] 6A, in step S1, product proposal server 101 determines whether or not a request for cosmetic product suggestions has been made by user terminal 20. If a request for cosmetic product suggestions has not been made (NO), product proposal server 101 repeats the process of step S1. If a request for cosmetic product suggestions has been made (YES), questionnaire implementation unit 5 conducts a questionnaire for user 30 in step S2.

[0037] Taking the questionnaire inquiring about the skin concerns of user 30 shown in Fig. 4 as an example, CPU 21 displays image Im7 including questionnaire text 71 and check box 72 supplied from questionnaire implementation unit 5 on display unit 24 as shown in Fig. 7. When user 30 checks check box 72 that reads, for example, "I am concerned about blemishes and freckles," and operates send button 73, user terminal 20 transmits the questionnaire results to product proposal server 101. In step S3, questionnaire implementation unit 5 acquires and stores the questionnaire results.

[0038] 8, when the CPU 21 transmits the survey result to the product proposal server 101, it displays an image Im8 on the display unit 24 instructing to photograph the face of the user 30. When the user 30 operates the camera start button 81, the camera 23 is started to photograph the user 30, and the photographed image can be transmitted to the product proposal server 101. In step S4, the photographed image acquisition unit 2 acquires and holds the photographed image of the face of the user 30. The order of steps S2 and S3 and step S4 may be reversed.

[0039] In step S5, the makeup base color selection unit selects a color of makeup base recommended to the user 30 based on the questionnaire results, and provides the product information to the user 30. In step S6, the foundation color proposal unit 7 and the foundation color selection unit 8 select a color of foundation recommended to the user 30 based on the captured image, and provide the product information to the user 30. The order of steps S5 and S6 may be reversed, or steps S5 and S6 may be performed simultaneously. When the product proposal server 101 executes the processes of steps S5 and S6, the CPU 21 displays an image Im9 showing the proposed makeup base and foundation on the display unit 24, as shown in FIG. 9.

[0040] The image Im9 includes a makeup base display field 91 showing the product number and color of the proposed makeup base, a foundation display field 92 showing the product number and color of the proposed foundation, a confirm button 93, a foundation change button 94, and a virtual makeup display button 95 for transitioning to an image display of virtual makeup. The virtual makeup is a virtual makeup image in which the makeup base and foundation are applied to the face of the user 30. The makeup base display field 91 and the foundation display field 92 may include the product names of the makeup base and foundation.

[0041] In step S7, the product proposal server 101 determines whether the user has decided on the combination of makeup base and foundation. When the user 30 operates the decision button 93 of the image Im9, the product proposal server 101 determines that the user has decided on the combination of makeup base and foundation (YES), and ends the process. When the user 30 has decided on the combination of makeup base and foundation, the user 30 can apply to the product sales and delivery management server 40 to purchase the makeup base and foundation.

[0042] If the user has not decided on a combination of makeup base and foundation (NO), the foundation color selection unit 8 determines in step S8 whether or not the user 30 has changed the foundation. If the user 30 operates the foundation change button 94 in image Im9 to change the foundation (YES), the foundation color selection unit 8 changes the foundation in step S9 and returns the process to step S7.

[0043] If the user 30 does not change the foundation in step S8 (NO), the makeup image generation unit 11 determines whether or not virtual makeup has been instructed in step S10 shown in Fig. 6B. If virtual makeup has been instructed (YES), the makeup image generation unit 11 provides a virtual makeup image to the user 30 (user terminal 20) in step S11. As shown in Fig. 10, the CPU 21 displays an image Im10 including a virtual makeup image 1001 and a back button 1002 on the display unit 24.

[0044] In step S12, the makeup image generation unit 11 determines whether or not the back button 1002 is operated to end the provision of the virtual makeup image. If the provision of the virtual makeup image is not to be ended (NO), the makeup image generation unit 11 repeats the processes of steps S11 and S12. If the provision of the virtual makeup image is to be ended (YES), the process returns to step S7.

[0045] If virtual makeup is not instructed in step S10 (NO), the product proposal server 101 determines whether or not to end the processing of the cosmetic product proposal in step S13. If the processing of the cosmetic product proposal is not to end (NO), the processing returns to step S7. If the processing of the cosmetic product proposal is to end, the product proposal server 101 ends the processing.

[0046] As described above, user 30 answers the questionnaire administered by product proposal server 101 using user terminal 20, and transmits a captured image of the skin on which makeup is to be applied to product proposal server 101. In this way, the cosmetics proposal device, cosmetics proposal method, cosmetics proposal program, and virtual makeup image providing method according to the first embodiment can propose a makeup base and foundation that will achieve the skin color desired by user 30 by applying the makeup base and foundation to the skin.

[0047] <Second embodiment> In the second embodiment shown in Fig. 11, the same parts as those in the first embodiment shown in Fig. 1 are given the same reference numerals, and their description may be omitted. In Fig. 11, a product suggestion server 102, a user terminal 20, and a product sales and shipping management server 40 are connected via a network 50. The product suggestion server 102 constitutes a cosmetics suggestion device according to the second embodiment, and executes the cosmetics suggestion method according to the second embodiment to suggest cosmetics to a user 30.

[0048] The product suggestion server 102 may be configured with a computer device. The product suggestion server 102 may be configured to include a non-transitory storage medium that stores the cosmetic product suggestion program according to the second embodiment, and a CPU that executes the cosmetic product suggestion program.

[0049] The product proposal server 102 includes a communication unit 1, a captured image acquisition unit 2, a numerical conversion unit 3, a color correction value generation unit 10, a makeup image generation unit 11, a combination color proposal unit 12, a combination color selection unit 13, a combination color information storage unit 14, and a product information storage unit 15.

[0050] The operation of each component in the product proposal server 102 will be described. The combination color information storage unit 14 stores color information indicating combination colors of (m×n) colors of makeup base and foundation, which are obtained by superimposing each color of n colors of foundation products on each color of m colors of makeup base products. A combination color is a color formed by the color of the makeup base and the color of the foundation when viewed from the foundation side when the foundation is applied on top of the makeup base. The product information storage unit 15 stores product information of m colors of makeup base products and product information of n colors of foundation products.

[0051] The combination color suggestion unit 12 suggests one of the (m×n) combination colors of makeup base and foundation stored in the combination color information storage unit 14 as a combination color to be recommended to the user, based on the numerical values ​​of the photographed image acquired by the photographed image acquisition unit 2. The combination color suggestion unit 12 may select a combination color having a numerical value closest to the numerical value supplied by the numerical value conversion unit 3. The combination color selection unit 13 selects the combination color suggested by the combination color suggestion unit 12 as an initial value, and if the user 30 accepts the proposed combination color, it selects the combination color suggested by the combination color suggestion unit 12 as is.

[0052] The communication unit 1 transmits to the user terminal 20, from among the product information of makeup bases and foundations stored in the product information storage unit 15, the product information of makeup bases and foundations corresponding to the combination color selected by the combination color selection unit 13. The communication unit 1 functions as a product information providing unit that provides the user 30 with the product information of makeup bases and foundations corresponding to the combination color selected by the combination color selection unit 13.

[0053] The user 30 can use the user terminal 20 to instruct the color combination selection unit 13 to select a color combination different from the color combination proposed by the color combination proposal unit 12. When instructed to select a different color combination, the color combination selection unit 13 selects the color combination instructed by the user 30. In this way, if the user 30 does not accept the proposed color combination and changes it to another color combination, the color combination selection unit 13 selects the color combination changed by the user 30 instead of the color combination proposed by the color combination proposal unit 12.

[0054] The color correction value generation unit 10 generates a color correction value for correcting the skin color in the captured image acquired by the captured image acquisition unit 2 to a color corresponding to the combination color selected by the combination color selection unit 13, which is obtained by applying a makeup base and foundation of the combination color selected by the combination color selection unit 13 in that order to the skin of the user 30. The makeup image generation unit 11 corrects the skin color in the captured image acquired by the captured image acquisition unit 2 with the color correction value to generate a virtual makeup image. The communication unit 1 transmits the virtual makeup image to the user terminal 20. The communication unit 1 also functions as a makeup image provision unit that provides the virtual makeup image to the user 30.

[0055] The processing executed by product proposal server 102 when a cosmetic product suggestion is requested by user terminal 20 will be further described using the flowchart shown in Fig. 12 and Figs. 8, 10, and 13. Figs. 8, 10, and 13 show examples of images displayed on display unit 24 when CPU 21 executes an application program stored in non-transitory storage medium 22 in response to the processing of cosmetic product suggestions by product proposal server 102. Fig. 12 also shows a cosmetic product suggestion method executed by product proposal server 102, and processing executed by the CPU when product proposal server 102 is configured with a CPU that executes a cosmetic product suggestion program.

[0056] 12, in step S21, the product proposal server 102 determines whether or not a request for a cosmetics proposal has been made from the user terminal 20. If a request for a cosmetics proposal has not been made (NO), the product proposal server 102 repeats the process of step S21. When a request for a cosmetics proposal has been made, the CPU 21 displays an image Im8 on the display unit 24 instructing the user 30 to take a picture of the face of the user 30, as shown in FIG. 8. When the user 30 operates the camera start button 81, the camera 23 is started to take a picture of the user 30, and the taken image can be transmitted to the product proposal server 102. If the taken image has been transmitted to the product proposal server 102 and a request for a cosmetics proposal has been made (YES), the captured image acquisition unit 2 acquires and holds the captured image of the face of the user 30 in step S22.

[0057] In step S23, the color combination suggestion unit 12 suggests a combination color of the makeup base and foundation based on the captured image, and provides the product information to the user 30. When the product suggestion server 102 executes the process of step S23, the CPU 21 displays an image Im13 indicating the proposed combination of makeup base and foundation on the display unit 24, as shown in Fig. 13. The image Im13 includes a combination display field 131 indicating the proposed combination of the product number and color of the makeup base and the product number and color of the foundation, a decision button 132, a combination change button 133, and a virtual makeup display button 134. The combination display field 131 may include the product names of the makeup base and foundation.

[0058] In step S24, the product proposal server 102 determines whether the user has decided on the combination of makeup base and foundation. When the user 30 operates the decision button 132 of the image Im13, the product proposal server 102 determines that the user has decided on the combination of makeup base and foundation (YES), and ends the process. When the user 30 decides on the combination of makeup base and foundation, the user 30 can apply to the product sales and delivery management server 40 to purchase the makeup base and foundation.

[0059] If the user has not decided on the combination of the makeup base and foundation (NO), the combination color selection unit 13 determines in step S25 whether or not the user 30 has changed the combination. If the user 30 operates the combination change button 133 of the image Im13 to change the combination (YES), the combination color selection unit 13 changes the combination in step S26 and returns the process to step S24.

[0060] If the user 30 does not change the combination in step S25 (NO), the makeup image generation unit 11 determines whether or not virtual makeup has been instructed in step S27. If virtual makeup has been instructed (YES), the makeup image generation unit 11 provides a virtual makeup image to the user 30 (user terminal 20) in step S28. As shown in Fig. 10, the CPU 21 displays an image Im10 including a virtual makeup image 1001 and a back button 1002 on the display unit 24.

[0061] In step S29, the makeup image generation unit 11 determines whether or not the back button 1002 is operated to end the provision of the virtual makeup image. If the provision of the virtual makeup image is not to be ended (NO), the makeup image generation unit 11 repeats the processes of steps S28 and S29. If the provision of the virtual makeup image is to be ended (YES), the process returns to step S24.

[0062] If virtual makeup is not instructed in step S27 (NO), the product proposal server 102 determines whether or not to end the processing of the cosmetic product proposal in step S30. If the processing of the cosmetic product proposal is not to end (NO), the processing returns to step S24. If the processing of the cosmetic product proposal is to end, the product proposal server 102 ends the processing.

[0063] The product proposal server 102 executes a virtual makeup image providing method similar to the virtual makeup image providing method executed by the product proposal server 101. The product proposal server 102 (captured image acquisition unit 2) acquires a captured image of the skin of the user 30 to which makeup is to be applied. The product proposal server 102 (color correction value generation unit 10) generates a color correction value for correcting the skin color when a selected makeup base and a selected foundation are applied to the skin of the user 30 in this order. The product proposal server 102 (makeup image generation unit 11) corrects the skin color in the captured image with the color correction value to generate a virtual makeup image. The product proposal server 102 (communication unit 1) provides the virtual makeup image to the user 30.

[0064] As described above, user 30 transmits a captured image of skin to which makeup is to be applied to product proposal server 102. In this way, the cosmetics proposal device, cosmetics proposal method, cosmetics proposal program, and virtual makeup image providing method according to the second embodiment can propose a makeup base and foundation that will achieve the skin color desired by user 30 by applying the makeup base and foundation to the skin.

[0065] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the gist of the present invention. [Explanation of symbols]

[0066] 1. Communications Department 2. Image Acquisition Section 3. Numerical conversion section 4 Product information / color information storage section 5. Survey Implementation Department 6. Makeup base color selection section 7 Foundation Color Suggestion Department 8 Foundation color selection section 9. Color combination setting section 10 Color correction value generator 11 Makeup image generation unit 12 Color Combination Suggestion Department 13 Color combination selection section 14 Combined color information storage unit 15 Product information holding department 20 User terminal 30 users 40 Product sales and shipping management server 50 Network 101,102 Product proposal server

Claims

1. a product information and color information storage unit that stores product information and color information of makeup base products of m colors and product information and color information of foundation products of n colors, where m and n are integers of 2 or more; a captured image acquisition unit that acquires a captured image of a user's skin to be made up; a makeup base color selection unit that selects a color to be recommended to the user from among the m colors of makeup base products stored in the product information / color information storage unit based on the results of a questionnaire about the user's skin conducted for the user; a foundation color suggestion unit that suggests a color to be recommended to the user from among the n colors of foundation products stored in the product information and color information storage unit based on the captured image acquired by the captured image acquisition unit; a foundation color selection unit that selects the color proposed by the foundation color suggestion unit from among the n colors of foundation products stored in the product information / color information storage unit if the user accepts the proposed color of foundation; a product information providing unit that provides, to the user, the product information of the makeup base of a color selected by the makeup base color selecting unit and the product information of the foundation of a color selected by the foundation color selecting unit, based on the product information of the makeup base and the product information of the foundation stored in the product information and color information storage unit; A cosmetic suggestion device comprising:

2. 2. The cosmetic suggestion device according to claim 1, wherein, if the user does not accept the proposed color of foundation and changes it to another color, the foundation color selection unit selects the color changed by the user instead of the color proposed by the foundation color suggestion unit.

3. 3. The cosmetic suggestion device according to claim 1, wherein when there are multiple colors recommended to the user, the makeup base color selection unit selects the most suitable color from the multiple colors based on the captured image acquired by the captured image acquisition unit.

4. a combination color setting unit that sets the color selected by the makeup base color selection unit and the color selected by the foundation color selection unit as a combination color; a color correction value generating unit that generates a color correction value for correcting the skin color in the captured image acquired by the captured image acquiring unit to a color corresponding to a combination color set by the combination color setting unit, the color being obtained by applying the makeup base of a color selected by the makeup base color selecting unit and the color being selected by the foundation color selecting unit to the skin of the user in this order; a makeup image generating unit that corrects the skin color in the captured image acquired by the captured image acquiring unit using the color correction value to generate a virtual makeup image; a makeup image providing unit for providing the virtual makeup image to the user; The cosmetic suggestion device according to claim 1 or 2, further comprising:

5. a product information and color information storage unit stores product information and color information of m-color makeup base products and product information and color information of n-color foundation products, where m and n are integers of 2 or more; Acquire a photographed image of the user's skin to be made up; selecting a color to be recommended to the user from among the m colors of makeup base products stored in the product information / color information storage unit based on a result of a questionnaire about the user's skin conducted on the user; propose a color to be recommended to the user from among the n colors of foundation products stored in the product information / color information storage unit based on the captured image; If the user accepts the proposed color of foundation, the proposed color is selected from the n colors of foundation products stored in the product information / color information storage unit; Based on the product information of the makeup base and the product information of the foundation stored in the product information / color information storage unit, the product information of the makeup base of the selected color and the product information of the foundation of the selected color are provided to the user. How to propose cosmetics.

6. On the computer, acquiring a photographed image of a user's skin to be made up; selecting a color to be recommended to the user from among m colors of makeup base products stored in a product information / color information storage unit, where m is an integer of 2 or more, based on the results of a questionnaire about the user's skin conducted on the user; a step of proposing a recommended color to the user from among n colors of foundation products stored in the product information and color information storage unit based on the captured image, where n is an integer of 2 or more; If the user accepts the proposed color of foundation, selecting the proposed color from among the n colors of foundation products stored in the product information / color information storage unit; providing the user with product information on the makeup base of a selected color and product information on the foundation of a selected color based on the product information on the makeup base and the product information on the foundation stored in the product information and color information storage unit; A cosmetics suggestion program that allows you to:

7. a product information storage unit that stores product information on makeup base products of m colors and product information on foundation products of n colors, where m and n are integers of 2 or more; a combination color information storage unit that stores color information indicating combination colors of the makeup base and the foundation of (m×n) colors obtained by superimposing each color of the foundation product of the n colors on each color of the makeup base product of the m colors; a captured image acquisition unit that acquires a captured image of a user's skin to be made up; a color combination suggesting unit that suggests, based on the captured image acquired by the captured image acquiring unit, any one of the (m×n) color combinations of the makeup base and the foundation stored in the color combination information storage unit as a color combination recommended to the user; a color combination selection unit that selects the color combination proposed by the color combination proposal unit from among the color combinations stored in the color combination information storage unit if the user accepts the proposed color combination; a product information providing unit that provides, to the user, the product information of the makeup base and the product information of the foundation corresponding to the combination color selected by the combination color selecting unit, based on the product information of the makeup base and the product information of the foundation stored in the product information storage unit; A cosmetic suggestion device comprising:

8. 8. The cosmetic suggestion device according to claim 7, wherein, if the user does not accept the proposed color combination and changes it to another color combination, the color combination selection unit selects the color combination changed by the user instead of the color combination proposed by the color combination suggestion unit.

9. a color correction value generation unit that generates a color correction value for correcting the skin color in the captured image acquired by the captured image acquisition unit to a color corresponding to a combination color selected by the combination color selection unit, the color being obtained by applying the makeup base and the foundation in this order to the skin of the user; a makeup image generating unit that corrects the skin color in the captured image acquired by the captured image acquiring unit using the color correction value to generate a virtual makeup image; a makeup image providing unit for providing the virtual makeup image to the user; The cosmetic product suggestion device according to claim 7 or 8, further comprising:

10. a product information storage unit stores product information for a makeup base product in m colors and product information for a foundation product in n colors, where m and n are integers of 2 or more; a combination color information storage unit stores color information indicating combination colors of the makeup base and the foundation of (m×n) colors obtained by superimposing each color of the foundation product of the n colors on each color of the makeup base product of the m colors; Acquire a photographed image of the user's skin to which makeup is to be applied; propose, based on the acquired photographed image, one of the (m×n) color combinations of the makeup base and the foundation stored in the combination color information storage unit as a color combination recommended to the user; If the user accepts the proposed color combination, the proposed color combination is selected from the color combinations stored in the color combination information storage unit; Based on the product information of the makeup base and the product information of the foundation stored in the product information storage unit, the product information of the makeup base and the product information of the foundation corresponding to the selected color combination are provided to the user. How to propose cosmetics.

11. On the computer, acquiring a photographed image of a user's skin to be made up; a step of proposing, based on the acquired photographed image, one of (m×n) color combinations of makeup base and foundation stored in a color combination information storage unit as a color combination recommended to the user, where m and n are integers of 2 or more; If the user accepts the proposed color combination, selecting the proposed color combination from among the color combinations stored in the color combination information storage unit; providing the user with product information on the makeup base product and the foundation product corresponding to the selected color combination based on product information on the makeup base product of m colors and product information on the foundation product of n colors stored in a product information storage unit; A cosmetics suggestion program that allows you to:

12. Acquire a photographed image of the user's skin to be made up; generating a color correction value for correcting the color of the skin when the selected makeup base and the selected foundation are applied to the skin in this order; correcting the skin color in the captured image with the color correction value to generate a virtual makeup image; The virtual makeup image is provided to the user. A method for providing virtual makeup images.

Images