MAKEUP SIMULATION SYSTEM, MAKEUP SIMULATION METHOD, MAKEUP SIMULATION PROGRAM, AND MAKEUP SIMULATION DEVICE

Abstract

To provide a makeup simulation system, a makeup simulation method, a makeup simulation program, and a makeup simulation device capable of increasing reproduction accuracy of makeup simulation.SOLUTION: A makeup simulation device includes: a first spectral reflectivity calculation unit 23b that calculates first spectral reflectivity of an application target before makeup from a first RGB value of the application target before makeup; a first color signal calculation unit 23c that calculates a first color signal of the application target before makeup using the first spectral reflectivity; a second color signal calculation unit 23e that calculates a second color signal of cosmetics using second spectral reflectivity of the cosmetics; a third color signal calculation unit 23f that calculates a third color signal of a makeup state in which the cosmetics are applied to the application target from the first color signal and the second color signal; a second RGB value calculation unit 23g that calculates a second RGB value of the makeup state from the third color signal; and an object generation unit 23a that generates a three-dimensional object in a makeup state from the second RGB value.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a makeup simulation system, a makeup simulation method, a makeup simulation program, and a makeup simulation device for reproducing a state in which a cosmetic material is applied to an application target object. [Background technology]

[0002] There is known a makeup simulation system that uses a facial image of a user to display a simulation image of the face after applying cosmetics. For example, Patent Document 1 describes a makeup simulation method that generates a simulation image of the face with cosmetics applied by superimposing a pattern for each cosmetic on an image of the user's face without makeup. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-257194 A Summary of the Invention [Problem to be solved by the invention]

[0004] When cosmetics are applied to the face, the area where the cosmetics are applied has a color that is a combination of the skin color and the color of the cosmetics. In other words, when users have different skin colors, the areas where the cosmetics are applied will have different colors even if the same cosmetics are applied. However, in the above-mentioned method of simply superimposing a cosmetic pattern on the user's face image, the skin color is not reflected in the simulation image, and there is a large discrepancy between the simulation image and the appearance when the user actually applies makeup. Therefore, the above-mentioned method makes it difficult to perform a makeup simulation with high reproduction accuracy. [Means for solving the problem]

[0005] A makeup simulation system for solving the above problem includes a first spectral reflectance calculation unit that calculates a first spectral reflectance of the application target before makeup from a first RGB value of the application target to which a cosmetic is applied before makeup, a first color signal calculation unit that calculates a first color signal of the application target before makeup using a first three-dimensional reflection characteristic model expressed by the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristic of the application target before makeup, and the spectral distribution and geometric conditions of illumination light, and a second spectral reflectance of the cosmetic and a second reflection characteristic parameter that defines the reflection characteristic of the cosmetic. a second color signal calculation unit that calculates a second color signal of the cosmetic using a second three-dimensional reflection characteristic model expressed by the first data, the spectral distribution of the illumination light, and the geometric conditions; a third color signal calculation unit that calculates a third color signal of the cosmetic application target in a made-up state where the cosmetic is applied to the cosmetic application target before makeup from the first color signal and the second color signal; a second RGB value calculation unit that calculates a second RGB value of the cosmetic application target in the made-up state from the third color signal; and an object generation unit that generates a three-dimensional object including the cosmetic application target in the made-up state from the second RGB value.

[0006] A makeup simulation method for solving the above problem includes the steps of: calculating a first spectral reflectance of the object to be applied with makeup before makeup from a first RGB value of the object to be applied with a cosmetic before makeup; calculating a first color signal of the object to be applied with makeup before makeup using the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristics of the object to be applied with makeup before makeup, and a first three-dimensional reflection characteristic model expressed by the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristics of the object to be applied with makeup before makeup, and the spectral distribution and geometric conditions of the illumination light; calculating a second color signal of the cosmetic using a second spectral reflectance of the cosmetic, a second reflection characteristic parameter that defines the reflection characteristics of the cosmetic, and the spectral distribution and geometric conditions of the illumination light; calculating a third color signal of the object to be applied with makeup after applying the cosmetic to the object to be applied with makeup before makeup from the first color signal and the second color signal; calculating a second RGB value of the object to be applied with makeup from the third color signal; and generating a three-dimensional object including the object to be applied with makeup from the second RGB value.

[0007] A makeup simulation program that solves the above-mentioned problem causes a computer to execute the following processes: calculating a first spectral reflectance of the object to be applied with a cosmetic before makeup from a first RGB value of the object to be applied with a cosmetic before makeup; calculating a first color signal of the object to be applied with a cosmetic before makeup using the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristics of the object to be applied with a cosmetic before makeup, and a first three-dimensional reflection characteristic model expressed by the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristics of the object to be applied with a cosmetic before makeup, and the spectral distribution and geometric conditions of the illumination light; calculating a second color signal of the cosmetic using a second spectral reflectance of the cosmetic, a second reflection characteristic parameter that defines the reflection characteristics of the cosmetic, and the spectral distribution and geometric conditions of the illumination light; calculating a third color signal of the object to be applied with a cosmetic applied to the object to be applied with makeup from the first color signal and the second color signal; calculating a second RGB value of the object to be applied with a cosmetic in the makeup state from the third color signal; and generating a three-dimensional object including the object to be applied with a cosmetic in the makeup state from the second RGB value.

[0008] A makeup simulation device that solves the above problem includes a first spectral reflectance calculation unit that calculates a first spectral reflectance of the application target before makeup from a first RGB value of the application target to which a cosmetic is applied before makeup, a first color signal calculation unit that calculates a first color signal of the application target before makeup using a first three-dimensional reflection characteristic model expressed by the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristic of the application target before makeup, and the spectral distribution and geometric conditions of illumination light, and a second spectral reflectance of the cosmetic and a second reflection characteristic parameter that defines the reflection characteristic of the cosmetic. a second color signal calculation unit that calculates a second color signal of the cosmetic using a second three-dimensional reflection characteristic model expressed by the first color signal and the spectral distribution of the illumination light and the geometric conditions; a third color signal calculation unit that calculates a third color signal of the cosmetic application target in a made-up state where the cosmetic is applied to the cosmetic application target before makeup from the first color signal and the second color signal; a second RGB value calculation unit that calculates a second RGB value of the cosmetic application target in the made-up state from the third color signal; and an object generation unit that generates a three-dimensional object including the cosmetic application target in the made-up state from the second RGB value.

[0009] According to the above configurations, the first color signal is calculated using the first three-dimensional reflection characteristic model, and the second color signal is calculated using the second three-dimensional reflection characteristic model, so that a third color signal in a state in which the color of the application target is reflected in the cosmetic can be calculated as a color signal of the application target in a made-up state. Then, a three-dimensional object of the application target in a made-up state is generated by calculating a second RGB value from the third color signal. As a result, a three-dimensional object of the application target in a made-up state in which the color of the application target and the shading and gloss caused by the illumination light can be obtained as a simulation result. Therefore, a makeup simulation with higher reproduction accuracy can be performed than when the color of the cosmetic is simply superimposed on the application target. In addition, since the shading and gloss caused by the illumination light in a made-up state can be reproduced, the texture of the state in which the cosmetic is applied to the application target can be reproduced. Then, for example, when counseling a user, the above makeup simulation can be used to suggest the most suitable cosmetic to the user.

[0010] The above-mentioned makeup simulation system may further include a spectral conversion database that stores a first conversion function for converting the first RGB value into the first spectral reflectance, the first conversion function corresponding to the color of the object to be applied before the makeup is applied, and the first spectral reflectance calculation unit may calculate the first spectral reflectance using the first conversion function corresponding to the color of the object to be applied before the makeup is applied. According to the above-mentioned configuration, by calculating the first spectral reflectance based on the first conversion function corresponding to the color of the object to be applied, it is possible to calculate a spectral reflectance closer to that of the actual object to be applied as the first spectral reflectance. Therefore, it is possible to perform a makeup simulation with higher reproduction accuracy.

[0011] The above-mentioned makeup simulation system may further include a cosmetic database storing a second conversion function for calculating the second spectral reflectance from the first spectral reflectance, the second conversion function corresponding to the color of the application target before the makeup is applied, and a second spectral reflectance calculation unit calculating the second spectral reflectance using the first spectral reflectance and the second conversion function corresponding to the color of the application target before the makeup is applied, and the second color signal calculation unit may calculate the second color signal using the second three-dimensional reflection characteristic model including the second spectral reflectance calculated by the second spectral reflectance calculation unit. According to the above-mentioned configuration, by calculating the second spectral reflectance based on the first spectral reflectance and the second conversion function corresponding to the color of the application target, it is possible to calculate a second spectral reflectance closer to the spectral reflectance of the cosmetic when the cosmetic is actually applied to the application target and blended with the application target. Therefore, it is possible to perform a makeup simulation with higher reproduction accuracy.

[0012] In the above-mentioned makeup simulation system, the cosmetic database may store the second conversion function and the second reflection characteristic parameter for each cosmetic product, the second spectral reflectance calculation unit may calculate the second spectral reflectance using the second conversion function for each cosmetic product, and the second color signal calculation unit may calculate the second color signal using the second three-dimensional reflection characteristic model including the second spectral reflectance calculated by the second spectral reflectance calculation unit and the second reflection characteristic parameter for each cosmetic product. According to the above-mentioned configuration, by calculating the second color signal using the second spectral reflectance and the second reflection characteristic parameter for each cosmetic product, a three-dimensional object reflecting the color and reflection characteristic of each cosmetic product can be generated. Therefore, a makeup simulation with higher reproduction accuracy can be performed.

[0013] In the above-mentioned makeup simulation system, the cosmetic database stores the spectral reflectance of a shining pigment contained in the cosmetic, a reflection characteristic parameter of the shining pigment that defines the reflection characteristics of the shining pigment, shape information that defines the size of the shining pigment, and dispersion information that defines the dispersion state of the shining pigment, and the second color signal calculation unit calculates the second color signal of the cosmetic using the second three-dimensional reflection characteristic model, and further calculates a color signal of the shining pigment using a third three-dimensional reflection characteristic model that is expressed by the spectral reflectance of the shining pigment, the reflection characteristic parameter of the shining pigment, the spectral distribution of the illumination light, and the geometric conditions in an area where the shining pigment is scattered that is estimated from the shape information and the dispersion information of the shining pigment, and the third color signal calculation unit may calculate the third color signal of the application target in the made-up state where the cosmetic containing the shining pigment has been applied to the application target before makeup, from the first color signal, the second color signal, and the color signal of the shining pigment. According to the above configuration, the color signal of the shining agent can be calculated using a third three-dimensional reflection characteristic model including the spectral reflectance of the shining agent contained in the cosmetic. Then, by calculating a second RGB value from a third color signal calculated based on the first color signal, the second color signal, and the color signal of the shining agent, a three-dimensional object in which the gloss of the shining agent in a made-up state is reproduced can be generated. In addition, by calculating the color signal of the shining agent in an area where the shining agent is scattered, which is estimated from the shape information and the dispersion information, the particle shape of the shining agent contained in the actual cosmetic and the dispersion state of the shining agent can be reproduced. Therefore, a makeup simulation with higher reproduction accuracy can be performed.

[0014] In the above-mentioned makeup simulation system, at least one of the second spectral reflectance and the second reflection characteristic parameter may be an actual measurement value measured for the cosmetic. According to the above-mentioned configuration, by using an actual measurement value for the cosmetic as at least one of the second spectral reflectance and the second reflection characteristic parameter, which are parameters constituting the second three-dimensional reflection characteristic model, it is possible to easily perform simulations of various cosmetics. For example, when a new cosmetic is developed, a simulation can be performed by using this system by actually measuring the second spectral reflectance and the second reflection characteristic parameter of the new cosmetic.

[0015] The above-mentioned makeup simulation system may further include a lighting environment database that stores, for each lighting environment, the spectral distribution of the illumination light and a spatial distribution of an illumination light source that generates the illumination light, the spatial distribution in all directions centered on the three-dimensional object, and the first color signal calculation unit may calculate the first color signal using the first three-dimensional reflection characteristic model including the spectral distribution of the illumination light for each lighting environment, and the second color signal calculation unit may calculate the second color signal using the second three-dimensional reflection characteristic model including the spectral distribution of the illumination light for each lighting environment. According to the above-mentioned configuration, by calculating the first color signal and the second color signal based on the illumination spectral distribution for each lighting environment, a three-dimensional object can be generated in which lighting effects such as shadows and gloss caused by the illumination light for each lighting environment are reflected. Therefore, a makeup simulation with higher reproduction accuracy can be performed.

[0016] The above-mentioned makeup simulation system may further include a GPU that functions as at least one of the first spectral reflectance calculation unit, the first color signal calculation unit, the second color signal calculation unit, the third color signal calculation unit, the second RGB value calculation unit, and the object generation unit. According to the above-mentioned configuration, by executing the processing in the makeup simulation system using a GPU, real-time image processing becomes possible, and the makeup simulation can be performed more preferably. Effect of the Invention

[0017] According to the above-described configuration, by reflecting the color of the object to be applied to the cosmetic, the reproduction accuracy of the makeup simulation can be improved compared to a case where the color of the cosmetic is simply superimposed on the object to be applied. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a schematic diagram showing the overall configuration of a makeup simulation system. [Diagram 2] FIG. 2 is a block diagram showing the configuration of a user terminal. [Diagram 3] FIG. 2 is a block diagram showing the configuration of a server. [Figure 4] Functional block diagram of the server's GPU. [Diagram 5] FIG. 11 is a schematic diagram illustrating an app image in which a first three-dimensional object is displayed. [Figure 6] 11 is a schematic diagram showing an application image on which a cosmetic selection area is displayed. [Figure 7] 4 is a flowchart showing a processing procedure in the makeup simulation system. [Figure 8] FIG. 2 is a schematic diagram showing a geometric model of reflection in the makeup simulation system. [Figure 9] 9 is a schematic diagram showing a reflected light distribution in the geometric model of reflection shown in FIG. 8 . [Figure 10] Schematic diagram showing interference light generated in a multilayer film of a metallic coloring matter contained in a cosmetic. [Figure 11] FIG. 11 is a schematic diagram illustrating an app image in which a second three-dimensional object is displayed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] [First embodiment] Hereinafter, a first embodiment of a makeup simulation system, a makeup simulation method, a makeup simulation program, and a makeup simulation device will be described with reference to FIGS.

[0020] [Overall structure] 1, the makeup simulation system 1 includes a user terminal 10 and a server 20. The user terminal 10 is an information processing device owned by a user 2, and is, for example, a smartphone. Note that the user terminal 10 may also be an information processing device such as a tablet, a laptop computer, or a desktop computer.

[0021] The server 20 is a server computer that executes various processes in the makeup simulation system 1, and is an example of a makeup simulation device for performing a makeup simulation. Specifically, the server 20 generates a first three-dimensional object 30a, which is a three-dimensional object 30 of a face in an unmade-up state, which is a state before makeup is applied. The server 20 also generates a second three-dimensional object 30b, which is a three-dimensional object 30 of a face in a made-up state, which is a state after makeup is applied.

[0022] The user terminal 10 and the server 20 are connected via a network 3, which is a communication line. The network 3 is, for example, a mobile communication system such as 4G or 5G for mobile phones, or a wireless LAN communication system such as Wi-Fi (registered trademark).

[0023] A three-dimensional object display program is installed as an application program in the user terminal 10. The three-dimensional object display program is a program that causes the user terminal 10 to execute a process of displaying the first three-dimensional object 30a and the second three-dimensional object 30b. The three-dimensional object display program is downloaded from an application distribution device via a network and installed in the user terminal 10.

[0024] The user terminal 10 includes a touch panel 13. On the touch panel 13, a first three-dimensional object 30a, which is a three-dimensional object without makeup, and a second three-dimensional object 30b, which is a three-dimensional object 30 with makeup, as a simulation result, are displayed.

[0025] [User terminal] 2, the user terminal 10 includes a control unit 11, a memory 12, a touch panel 13, an imaging unit 14, and a communication unit 15. The control unit 11 is, for example, a CPU, and controls the overall operation of the user terminal 10.

[0026] The memory 12 includes a main memory and a data storage unit. The main memory included in the memory 12 is, for example, a RAM, and temporarily stores data and programs of the user terminal 10. The data storage unit included in the memory 12 is, for example, a non-volatile memory, for example, a flash memory. The data storage unit included in the memory 12 stores programs for executing various processes in the user terminal 10, such as a three-dimensional object display program.

[0027] The touch panel 13 is a display that combines a display device such as a liquid crystal panel or an organic EL panel with a position input device, and allows touch operation input. In other words, the touch panel 13 is a display unit on which the three-dimensional object 30 is displayed in the user terminal 10, and is also an operation unit that performs operation input to the server 20.

[0028] The imaging unit 14 is, for example, an RGB camera equipped with an imaging element such as a CCD or a CMOS. The imaging unit 14 detects light such as visible light, and outputs image data composed of RGB values ​​of red, green, and blue. In the following, the RGB values ​​of the image captured by the imaging unit 14 are referred to as first RGB values.

[0029] The imaging unit 14 acquires a facial image of the user 2 in a makeup-free state, the facial image being configured with first RGB values. The first three-dimensional object 30a is generated based on the first RGB values ​​constituting the facial image of the user 2 in a makeup-free state acquired by the imaging unit 14.

[0030] The communication unit 15 communicates with the server 20 via the network 3. Specifically, the communication unit 15 transmits a signal from the user terminal 10 to the server 20, and also receives data such as the first three-dimensional object 30a and the second three-dimensional object 30b generated by the server 20.

[0031] 〔server〕 3, the server 20 includes a control unit 21, a memory 24, and a communication unit 25. The control unit 21 includes, for example, a CPU 22 and a GPU 23. The CPU 22 controls the overall operation of the server 20 other than the process of generating the three-dimensional object 30. The GPU 23 executes the process of generating the three-dimensional object 30 in the server 20.

[0032] The memory 24 includes a main memory and a data storage unit. The main memory included in the memory 24 is, for example, a RAM, and temporarily stores data and programs of the server 20. The data storage unit included in the memory 24 is, for example, a non-volatile memory, and for example, is a flash memory such as an HDD or SSD.

[0033] The data storage unit included in the memory 24 stores programs for executing various processes in the server 20, such as a makeup simulation program. The makeup simulation program is a program that causes the server 20 to execute a process for generating a first three-dimensional object 30a and a second three-dimensional object 30b. The process of the makeup simulation program is controlled by the GPU 23 included in the control unit 21.

[0034] The memory 24 includes a spectral conversion database 24a as a data storage unit, a lighting environment database 24b, and a cosmetic database 24c. The spectral conversion database 24a stores a first conversion function M for converting a first RGB value constituting an application target, which is a target to which a cosmetic is applied, in the first three-dimensional object 30a into a first spectral reflectance S1(λ), ​​which is the spectral reflectance of the application target in an unmade-up state. In this embodiment, the spectral conversion database 24a stores a first conversion function M for converting the first RGB value into a first spectral reflectance S1(λ) for each of bare skin and lips on a face, which are an example of an application target.

[0035] The first conversion function M is a conversion matrix obtained by actually measuring the RGB values ​​and spectral reflectance of the objects to be coated for a large number of people and statistically analyzing the correspondence between the RGB values ​​and the spectral reflectance of the objects to be coated for each color of the objects to be coated for the measured subjects. As an example, the first conversion function M is a 61×3 conversion matrix that divides the visible wavelength range of 400 nm to 700 nm into 5 nm intervals.

[0036] As a specific example, the spectral conversion database 24a classifies skin into three types, a first type, a second type, and a third type, according to the amount of melanin contained in skin, which is an example of an application target, and stores a first conversion function M according to the skin color. The first type of skin is skin in a first range in which the amount of melanin is less than the second and third types, and has a high relative reflectance at each wavelength when light is applied to the skin. The second type of skin is skin in a second range in which the amount of melanin is greater than the first range, and has a medium relative reflectance at each wavelength when light is applied to the skin. The third type of skin is skin in a third range in which the amount of melanin is greater than the second range, and has a low relative reflectance at each wavelength when light is applied to the skin. The spectral conversion database 24a may, for example, classify the RGB values ​​of skin into predetermined ranges and store the first conversion function M for each classification of the RGB values ​​of skin.

[0037] The lighting environment database 24b stores, for each lighting environment, light source information that is a combination of an illumination spectral distribution E(λ), which is the spectral distribution of the illumination light irradiated onto the three-dimensional object 30, and a spatial distribution of the illumination light source that generates the illumination light. The illumination light source here may be, for example, a direct light source that directly irradiates the three-dimensional object 30 with illumination light, or a reflected light source in which an object present in the lighting environment reflects the illumination light from the direct light source toward the three-dimensional object 30, but these are not particularly distinguished in this embodiment. Therefore, the lighting environment database 24b stores the spatial distribution of the illumination light source, including the direct light source and the reflected light source, and the illumination spectral distribution E(λ) of the illumination light irradiated from the illumination light source, for all directions centered on the three-dimensional object 30. The spatial distribution of the illumination light source is used as information for defining geometric conditions such as the angle of incidence of the illumination light on the three-dimensional object 30.

[0038] The lighting environment here is a combination of the lighting spectral distribution E(λ) and the spatial distribution of the lighting light source for reproducing the lighting effect in a specific scene for the three-dimensional object 30. Specific examples of lighting environments include an indoor environment where an artificial direct light source is applied and an outdoor environment where a natural direct light source is applied, and the outdoor environment is divided into different time periods such as morning, afternoon, evening, and night. Examples of artificial direct light sources include incandescent light bulbs, fluorescent lights, and LED lights, and examples of natural direct light sources include the sun and the moon.

[0039] The illumination spectral distribution E(λ) and the spatial distribution of the illumination light source can be obtained, for example, by acquiring an omnidirectional spectral image in an actual illumination environment and sampling the spectral distribution every 5 nm in the visible wavelength range (for example, 400 to 700 nm) in all directions. In this case, the illumination environment database 24b stores the illumination spectral distribution E(λ) of each pixel of the spectral image. In detail, the position of each pixel in the spectral image is information specifying the spatial distribution of the illumination light source distributed in the space in all directions centered on the three-dimensional object 30. The illumination spectral distribution E(λ) assigned to each pixel is the illumination spectral distribution E(λ) of the illumination light irradiated to the three-dimensional object 30. In other words, by regarding each pixel in the spectral image as an illumination light source and assigning an illumination spectral distribution E(λ) to each pixel, the spatial distribution of the illumination light source in all directions and the illumination spectral distribution E(λ) of the illumination light irradiated from each illumination light source can be obtained. According to this method, in an actual lighting environment, the lighting source of the entire scene can be processed collectively without distinguishing between direct light sources and reflected light sources, and therefore, even if there are multiple lighting sources whose boundaries cannot be distinguished, the lighting environment can be reproduced.

[0040] The cosmetic database 24c stores the second conversion function T(λ) for each cosmetic product. The second conversion function T(λ) is a conversion matrix for calculating the second spectral reflectance S2(λ), which is the spectral reflectance of the cosmetic, based on the first spectral reflectance S1(λ). The cosmetic here refers to a cosmetic that is applied to an application target, such as foundation, blush, eye color, lipstick, etc. In other words, the cosmetic database 24c stores the second conversion function T(λ) for each type of cosmetic, such as foundation or blush, and for each variation in color, purpose, etc. of each cosmetic.

[0041] More specifically, the cosmetic database 24c stores, for each cosmetic product, the second conversion function T(λ) corresponding to the color of the object to be applied. For example, in the case of a cosmetic product to be applied to the skin, the cosmetic database 24c stores, for each cosmetic product, the second conversion function T(λ) for each of a first skin type, a second skin type, and a third skin type.

[0042] The second conversion function T(λ) is, for example, obtained by calculating the spectral reflectance S a and the spectral reflectance S b The spectral reflectance S a and spectral reflectance S b The amount of change is statistically calculated for each color of the object to be applied to the subject.

[0043] Furthermore, the cosmetic database 24c stores reflection characteristic parameters of the cosmetic for each cosmetic product. As will be described in detail later, the reflection characteristic parameters are parameters that depend on the surface properties of the object, such as the roughness of the object surface, and are parameters that define reflection characteristics such as the intensity of reflected light and the spread of gloss with respect to the angle of incidence of the illumination light on the object surface. The reflection characteristic parameters of the cosmetic can be obtained, for example, from actual measurements of the actual cosmetic using a measuring device such as a gloss meter.

[0044] Additionally, the cosmetic database 24c may store the spectral reflectance and reflection characteristic parameters of shining agents such as pearls and glitter contained in cosmetics such as eye colors. The spectral reflectance of the shining agent is calculated using reflection components independent of the spectral reflectance of the skin itself, since it is not easily affected by the color of the object to which the cosmetic is applied even when the cosmetic is applied to the skin. Therefore, no calculation process is performed using the second conversion function T(λ). Therefore, the spectral reflectance of the shining agent is stored as an actual measurement value measured by a measuring device.

[0045] The communication unit 25 communicates with the user terminal 10 via the network 3. Specifically, the communication unit 25 receives a signal from the user terminal 10, and transmits data such as the first three-dimensional object 30a and the second three-dimensional object 30b generated by the server 20 to the user terminal 10.

[0046] [GPU] As shown in FIG. 4, the GPU 23 included in the control unit 21 functions as an object generation unit 23a, a first spectral reflectance calculation unit 23b, a first color signal calculation unit 23c, a second spectral reflectance calculation unit 23d, a second color signal calculation unit 23e, a third color signal calculation unit 23f, and a second RGB value calculation unit 23g.

[0047] The object generating unit 23a generates a first three-dimensional object 30a, which is a three-dimensional object 30 of a face in an unmade-up state, based on a first RGB value of the face image in an unmade-up state. The object generating unit 23a also generates a second three-dimensional object 30b, which is a three-dimensional object 30 of a face in a made-up state, based on a second RGB value calculated by a second RGB value calculating unit 23g, which will be described later.

[0048] The first spectral reflectance calculation unit 23b calculates a first spectral reflectance S1(λ) based on the first RGB value and the first conversion function M. The first color signal calculation unit 23c calculates a first color signal C1(λ), ​​which is a color signal when the illumination light is reflected by the first three-dimensional object 30a, based on a first three-dimensional reflection characteristic model, which is a three-dimensional reflection characteristic model of the application target before makeup is applied.

[0049] The three-dimensional reflection characteristic model here is a mathematical model that describes the process of light reflection on an object, and the color and texture of the object are determined by the spectral reflectance and reflection characteristic parameters of the object that constitute the three-dimensional reflection characteristic model. Then, by providing the three-dimensional reflection characteristic model with geometric conditions such as the spectral distribution of the illumination light and the incident angle of the illumination light, a color signal is calculated when the illumination light is reflected on the object. In other words, the first three-dimensional reflection characteristic model is a mathematical model that describes the process of light reflection on an application target in an uncoated state.

[0050] The color signal here refers to the spectral distribution (energy spectrum distribution) of electromagnetic waves that are reflected by an object and enter the visual system of the human eye or a camera. The color signal is converted into color information such as RGB values ​​that are actually perceived by humans using RGB color matching functions. The first color signal C1(λ) corresponds to the color signal when the illumination light is reflected by an object that is not yet applied with makeup.

[0051] The second spectral reflectance calculation unit 23d calculates the second spectral reflectance S2(λ) based on the first spectral reflectance S1(λ) and the second conversion function T(λ). The second color signal calculation unit 23e calculates the second color signal C2(λ), which is a color signal when the illumination light is reflected from the cosmetic, based on a second three-dimensional reflection characteristic model, which is a three-dimensional reflection characteristic model of the cosmetic. The second three-dimensional reflection characteristic model is a mathematical model that describes the process of light reflection in the cosmetic. In the following, the reflection characteristic parameter for the application target object in an unmade-up state is referred to as the first reflection characteristic parameter, and the reflection characteristic parameter for the cosmetic is referred to as the second reflection characteristic parameter.

[0052] The third color signal calculation unit 23f calculates a third color signal C3(λ), which is a color signal of the second three-dimensional object 30b, based on the first color signal C1(λ) and the second color signal C2(λ). The third color signal C3(λ) corresponds to a color signal when the illumination light is reflected on the application target in a made-up state, that is, on the application target and the cosmetic applied to the application target. The second RGB value calculation unit 23g calculates a second RGB value, which is an RGB value of the second three-dimensional object 30b, based on the third color signal C3(λ) calculated by the third color signal calculation unit 23f. The second RGB value corresponds to the RGB value of the application target in a made-up state.

[0053] [App image] 5, an application image 40, which is an application image displayed by a three-dimensional object display program, is displayed on the touch panel 13 of the user terminal 10. The application image 40 includes a three-dimensional object display area 41 and an operation area 42. The three-dimensional object display area 41 is an area where the first three-dimensional object 30a or the second three-dimensional object 30b is displayed.

[0054] The operation area 42 includes, for example, a switching object 42a, a makeup area selection object 42b, a cosmetic selection object 42c, a lighting selection object 42d, and an operation object 42e. The switching object 42a is an object that switches the simulation result for each cosmetic between ON and OFF.

[0055] The makeup area selection object 42b is an object for manually specifying a makeup area 31, which is an area in the three-dimensional object 30 where cosmetics are applied. As an example of a method for specifying the makeup area 31, an area to which cosmetics are to be applied is specified by tapping input in the first three-dimensional object 30a displayed in the three-dimensional object display area 41. For example, the entire facial skin is specified as the makeup area 31 for foundation or blush, the skin around the eyes is specified for eye color, and the lips are specified for lipstick. Note that the makeup area 31 may be configured to be automatically set using an image recognition function or the like.

[0056] The cosmetic selection object 42c is an object for selecting a cosmetic product to be simulated. As shown in Fig. 6, by tapping on the cosmetic selection object 42c, a cosmetic selection area 43 is displayed in the three-dimensional object display area 41. In the cosmetic selection area 43, cosmetics registered in the cosmetic database 24c are displayed for each product as product objects 43a. By tapping on the product object 43a, a cosmetic product to be applied in the makeup simulation is selected.

[0057] As a specific example, in the case of foundation, the product object 43a of the product to be simulated is selected from the product objects 43a displayed for each type of foundation, such as liquid foundation, powder foundation, etc., and for each color.

[0058] The lighting selection object 42d is an object for selecting a lighting environment in the makeup simulation. By tapping the cosmetic selection object 42c, a lighting selection area (not shown) is displayed in the three-dimensional object display area 41. In the lighting selection area, various lighting environments registered in the lighting environment database 24b are displayed as lighting objects for each lighting environment. By tapping the lighting object, the lighting environment to be applied in the makeup simulation is selected.

[0059] The operation object 42e is an object for moving, rotating, enlarging, or reducing the first three-dimensional object 30a or the second three-dimensional object 30b displayed in the three-dimensional object display area 41.

[0060] [Operation of the first embodiment] Next, the operation of the makeup simulation system 1 will be described. 7, in step S1, control unit 11 of user terminal 10 causes imaging unit 14 to execute a process of acquiring a face image of user 2 without makeup. Then, control unit 11 executes a process of generating face image data of user 2 without makeup, which data is composed of first RGB values, and transmitting the data to server 20.

[0061] The facial image acquired here may be, for example, a single image, multiple images taken at different angles, or an image acquired from a video, as long as it is capable of generating the first three-dimensional object 30a.

[0062] Furthermore, when a face image is photographed in step S1, it is affected by various factors in the photographing environment, such as lighting, etc. From the viewpoint of obtaining a face image more suitable for a makeup simulation, for example, the brightness, saturation, etc. of the photographed face image may be adjusted by an image processing program stored in memory 12 as a correction for reducing the influence of the photographing environment.

[0063] In step S2, the object generation unit 23a of the server 20 executes a process of generating a first three-dimensional object 30a based on the first RGB values ​​constituting the face image of the user 2 in an unmade-up state transmitted from the user terminal 10. Furthermore, the object generation unit 23a executes a process of transmitting data of the generated first three-dimensional object 30a to the user terminal 10 and displaying it on the touch panel 13. As a result, the first three-dimensional object 30a is displayed in the three-dimensional object display area 41 of the application image 40 on the touch panel 13.

[0064] Next, the user 2 performs a tap operation input on an object in the operation area 42 displayed on the touch panel 13, sets the makeup area 31 to which the cosmetic is to be applied, and further selects the cosmetic product and lighting environment to be simulated. The control unit 11 of the user terminal 10 transmits to the server 20 a signal that identifies the makeup area 31 set by the user 2 and the cosmetic product and lighting environment selected by the user 2. Note that a plurality of different types of cosmetics may be selected, such as blush and foundation, for example.

[0065] In step S3, first spectral reflectance calculation unit 23b executes a process of calculating a first spectral reflectance S1(λ) in makeup area 31 based on the first RGB values ​​constituting first three-dimensional object 30a. Specifically, first spectral reflectance calculation unit 23b reads out a first conversion function M corresponding to the color of the application target in makeup area 31 from spectral conversion database 24a, and executes a process of calculating first spectral reflectance S1(λ) in makeup area 31.

[0066] The first conversion function M read out here may be determined automatically, for example, in accordance with the first RGB value constituting the makeup area 31 of the first three-dimensional object 30a, or may be determined by the user 2 selecting a color close to the object to be applied from a color sample such as a color code.

[0067] The first spectral reflectance S1(λ) can be approximated as a discrete 61-dimensional spectral reflectance vector s obtained by dividing the visible wavelength range of 400 nm to 700 nm at 5 nm intervals using the first conversion function M and the values ​​of each element in the first RGB value, R1, G1, and B1. The first spectral reflectance S1(λ) can be expressed by the following formulas (1) and (2).

[0068]

number

[0069] By calculating the first spectral reflectance S1(λ) based on the first conversion function M according to the color of the coating object, it is possible to calculate the first spectral reflectance S1(λ) closer to the actual spectral reflectance of the coating object. In addition, for example, when the coating object is facial skin, it is generally difficult to directly measure the spectral reflectance of the entire facial skin because the measurement range of a measuring device such as a spectrophotometer is small. In contrast, by calculating the first spectral reflectance S1(λ) from the first RGB value of a face image without makeup as in this embodiment, it is possible to suitably obtain the spectral reflectance of the coating object without makeup.

[0070] In this embodiment, the process of calculating the first spectral reflectance S1(λ) from the first RGB value is not executed in areas where cosmetics are not applied, such as hair or accessories such as earrings, i.e., areas other than the makeup area 31. This simplifies the calculation process for calculating the first spectral reflectance S1(λ) from the first RGB value.

[0071] In step S4, the first color signal calculation unit 23c executes a process of calculating a first color signal C1(λ) based on the first spectral reflectance S1(λ), ​​the first reflection characteristic parameter, the illumination spectral distribution E(λ) of the illumination light, and a first three-dimensional reflection characteristic model expressed by the geometric conditions of the illumination light.

[0072] Specifically, the first color signal calculation unit 23c reads out, from the lighting environment database 24b, the lighting spectral distribution E(λ) and the spatial distribution of the lighting light source in the lighting environment selected by the user 2. Then, the first color signal calculation unit 23c executes a process of calculating the first color signal C1(λ) using the first three-dimensional reflection characteristic model.

[0073] The first three-dimensional reflectance characteristic model is a first spectral reflectance S1(λ), ​​a lighting spectral distribution E(λ), and a first diffuse reflectance function D a , and the first specular reflection function G a Using the first diffuse reflectance function D a is a function that defines the diffuse reflection on an object to be applied without makeup. The first specular reflection function G a is a function that defines the specular reflection on a makeup-free application target object. In the first three-dimensional reflection characteristic model, the first term on the right side represents the diffuse reflection component, and the second term on the right side represents the specular reflection component.

[0074]

number

[0075] Here, a geometric model of reflection in this embodiment will be described with reference to Fig. 8 and Fig. 9. As shown in Fig. 8, a virtual space 50 includes a virtual point 51, a virtual light source 52, and a virtual viewpoint 53. The virtual point 51 is a point on the surface S of the first three-dimensional object 30a that receives light from the virtual light source 52. The virtual light source 52 irradiates illumination light onto the virtual point 51. The virtual viewpoint 53 is a viewpoint from which the virtual point 51 is observed.

[0076] Furthermore, lighting direction vector L is a vector directed from virtual point 51 to virtual light source 52. Normal vector N is a vector in the normal direction at virtual point 51 on surface S. Line-of-sight vector V is a vector directed from virtual point 51 to virtual viewpoint 53. Lighting direction vector L, normal vector N, and line-of-sight vector V are parameters for defining the geometric conditions of the illumination light in the three-dimensional reflection characteristic model. The angle between lighting direction vector L and normal vector N is called incident angle θ i The angle between the normal vector N and the line of sight vector V is the light receiving angle θ r Let us assume that.

[0077] In general, when an object is exposed to illumination light, the color signal C0(λ) generated on the object surface is expressed as C0(λ) = E0(λ)S0(λ) using the object's spectral reflectance S0(λ) and the illumination light's spectral distribution E0(λ).

[0078] In this embodiment, rather than simply calculating the color signal of an object from the spectral reflectance of the object and the spectral distribution of the illumination light, the gloss and texture of the object surface are reproduced more accurately by using a three-dimensional reflection characteristic model that takes into account the diffuse reflection and specular reflection on the object surface.

[0079] Specifically, when a virtual point 51 receives light from a virtual light source 52, the diffuse reflected light occurring in all directions at the virtual point 51 and the incident angle θ i and acceptance angle θ r The reflected light is composed of specular reflected light, which occurs when the light intensity is close to the reflected light intensity.

[0080] 9, the intensity distribution of the reflected light occurring at the imaginary point 51 is expressed by a reflected light distribution RD, which is an intensity distribution obtained by adding up the diffuse reflected light intensity α and the specular reflected light intensity I. The intensity of the reflected light occurring at the imaginary point 51 varies with the angle of reception θ r Specifically, the distance from the imaginary point 51 to the reflected light distribution RD in Fig. 9 represents the intensity of the reflected light at the light-receiving angle θr.

[0081] The diffuse reflection light intensity α is a reflection characteristic parameter for defining the intensity of the diffuse reflection light. The diffuse reflection light intensity α is determined by the angle of incidence θ i With respect to the acceptance angle θ r The specular reflected light intensity I is a reflection characteristic parameter that specifies the relative intensity of the specular reflected light with respect to the diffuse reflected light intensity α. The specular reflected light intensity I varies with the angle of incidence θ i With respect to the acceptance angle θ r The closer the value of θ approaches, the greater the incidence angle. i and acceptance angle θ r The maximum value is reached when these values ​​match.

[0082] For example, the acceptance angle θ r is the incidence angle θ i The acceptance angle θ is significantly different from r1 In this case, the intensity of the reflected light observed at the virtual viewpoint 53a is the diffuse reflected light intensity α. r is the incidence angle θ i The acceptance angle θ is approximated as r2 In this case, the intensity of the reflected light observed at the virtual viewpoint 53b is the sum of the diffuse reflected light intensity α and the specular reflected light intensity I.

[0083] First diffuse reflectance function D a can be expressed by the following formula (4) using the first diffuse reflected light intensity α1 and the illumination direction vector L and normal vector N as the geometric conditions of the illumination light. The first diffuse reflected light intensity α1 is a first reflection characteristic parameter for defining the intensity of the diffuse reflected light on an application target in an uncoated state. Here, the first diffuse reflected light intensity α1 is treated as α1=1. Note that the geometric conditions of the illumination light used in the following calculations are calculated from the spatial distribution of the illumination light source, the shape of the three-dimensional object 30, the viewpoint position at which the three-dimensional object 30 is observed, and the like.

[0084]

number

[0085] In addition, the range in which specular reflected light occurs in the reflected light distribution RD is expressed by the specular reflection parameter m. The specular reflection parameter m is a parameter that depends on the surface properties of an object, such as the roughness of the object surface, and is a reflection characteristic parameter that specifies the extent of gloss. The larger the specular reflection parameter m, the larger the range in which specular reflected light occurs.

[0086] First specular reflection function G a is the first specular reflection intensity I 1、 Using the first specular reflection parameter m1 and the illumination direction vector L, normal vector N, and line of sight direction vector V as geometric conditions of the illumination light, it can be expressed by the following formulas (5) to (7). Note that formula (6) is a Beckmann function. The first specular reflection light intensity I1 is a first reflection characteristic parameter for defining the intensity of diffuse reflected light on the application target in an unmade-up state. The first specular reflection parameter m1 is a first reflection characteristic parameter for defining the spread of gloss on the application target in an unmade-up state. The first specular reflection light intensity I1 and the first specular reflection parameter m1 are set to arbitrary values ​​in advance.

[0087]

number

[0088] The first specular reflection light intensity I1 and the first specular reflection parameter m1 can also be obtained from a plurality of images in which the angle of incidence of illumination light is changed with respect to the face of user 2 without makeup. For example, in step S1, when acquiring facial images of user 2 without makeup, the first specular reflection light intensity I1 and the first specular reflection parameter m1 can also be obtained by acquiring a plurality of images in which the angle of incidence of illumination light with respect to the face of user 2 is changed.

[0089] In this way, by calculating the first color signal C1(λ) using the first three-dimensional reflection characteristic model, the color of the object to be treated in an unmade-up state, as well as the shadows and gloss caused by the illumination light, can be reproduced as the first color signal C1(λ).

[0090] In step S5, second spectral reflectance calculation unit 23d executes a process to calculate a second spectral reflectance S2(λ) based on the first spectral reflectance S1(λ) and the second conversion function T(λ) in the makeup area 31. Specifically, second spectral reflectance calculation unit 23d reads out the second conversion function T(λ) corresponding to the color of the application target in the makeup area 31 from the cosmetic database 24c, and executes a process to calculate the second spectral reflectance S2(λ) in the makeup area 31.

[0091] The second conversion function T(λ) read out here may be determined, for example, in step S3 together with the first conversion function M, or may be determined by user 2 selecting a color close to the color of the object to be coated from a color sample such as a color code. The second spectral reflectance S2(λ) can be expressed by the following formula (8) using the first spectral reflectance S1(λ) and the second conversion function T(λ).

[0092]

number

[0093] By calculating the second spectral reflectance S2(λ) based on the first spectral reflectance S1(λ) and the second conversion function T(λ) corresponding to the color of the object to be applied, it is possible to calculate the second spectral reflectance S2(λ) that is closer to the spectral reflectance of the cosmetic when it is actually applied to the object to be applied and blended.

[0094] In step S6, the second color signal calculation unit 23e executes a process of calculating a second color signal C2(λ) based on the second spectral reflectance S2(λ), the second reflection characteristic parameter, the illumination spectral distribution E(λ) of the illumination light, and a second three-dimensional reflection characteristic model expressed by the geometric conditions of the illumination light.

[0095] Specifically, second color signal calculation unit 23e reads out, from cosmetic database 24c, second reflection characteristic parameters of the cosmetic selected by user 2. Furthermore, second color signal calculation unit 23e reads out, from lighting environment database 24b, the lighting spectral distribution E(λ) and the spatial distribution of the lighting light source in the lighting environment selected by user 2. Then, second color signal calculation unit 23e executes a process of calculating second color signal C2(λ) using the second three-dimensional reflection characteristic model.

[0096] The second three-dimensional reflectance characteristic model is the second spectral reflectance S2(λ), the illumination spectral distribution E(λ), and the second diffuse reflectance function D b , and the second specular reflection function G b Using the second diffuse reflectance function D, the following equations (9) to (11) can be expressed. b is a function that defines the diffuse reflection in the cosmetic material. The second specular reflection function G b is a function that defines the specular reflection in cosmetics. Also, B(N, V, L, m2) included in formula (11) is the Beckmann function. In formula (9), the first term on the right side represents the diffuse reflection component, and the second term on the right side represents the specular reflection component.

[0097]

number

[0098] In the second three-dimensional reflection characteristic model, the second diffuse reflected light intensity α2 is a parameter set for each cosmetic product, and is a second reflection characteristic parameter for defining the intensity of the diffuse reflected light of the cosmetic. Here, the second diffuse reflected light intensity α2 is treated as α2=1. The second specular reflected light intensity I2 is a parameter set for each cosmetic product, and is a second reflection characteristic parameter for defining the intensity of the specular reflected light of the cosmetic. The second specular reflection parameter m2 is a parameter set for each cosmetic product, and is a second reflection characteristic parameter for defining the spread of gloss of the cosmetic.

[0099] Furthermore, if multiple cosmetics, such as a foundation and a blush, are selected as the cosmetics to be simulated, in step S6, the second color signal calculation unit 23e executes a process of calculating the second color signal C2(λ) of each cosmetic.

[0100] In this way, by calculating the second color signal C2(λ) using the second three-dimensional reflection characteristic model, the color of the cosmetic material and the shading and luster caused by the illumination light can be reproduced as the second color signal C2(λ).

[0101] In step S6, the second color signal calculation unit 23e calculates the color signal C of the shining agent, such as pearl, glitter, etc., contained in the cosmetic such as the eye color, by using a third three-dimensional reflection characteristic model, which is a three-dimensional reflection characteristic model of the shining agent. G In this case, a second color signal C2(λ) for the base portion of the cosmetic containing the metallic lustre, i.e., the portion of the cosmetic other than the metallic lustre, is calculated by the second three-dimensional reflection characteristic model, and a color signal C for the metallic lustre is calculated by the second three-dimensional reflection characteristic model. G (λ) is calculated using the third three-dimensional reflectance model.

[0102] As shown in Fig. 10, the metallic lustre 60 contained in the cosmetic has a multi-layered film structure in which multiple thin films 61 are laminated. When light from a virtual light source 52 is irradiated onto the metallic lustre 60, interference light that depends on the thickness d of each thin film 61 is observed at a virtual viewpoint 53. Therefore, for the interference light caused by such a multi-layered film, the color signal C of the metallic lustre G By calculating (λ), the rainbow-like interference colors of the metallic color can be reproduced. Information on the multilayer structure of the metallic color 60, such as the number and thickness d of the thin films 61, is stored in the cosmetic database 24c.

[0103] The third three-dimensional reflection characteristic model is the spectral reflectance S G Using the reflectance characteristic parameters of the metallic tinsel, the illumination spectral distribution E(λ), and the geometric conditions of the illumination light, the following equations (12) and (13) can be used. In the third three-dimensional reflectance characteristic model, the color signal CG (λ) is calculated, ignoring the diffuse reflection component. In addition, the glitter has a spectral reflectance S G The B(N, V, L, m G ) is the Beckmann function.

[0104]

number

[0105] Specular reflectance function G of glittering agent G is a function that defines the specular reflection in the glitter. Specular reflection intensity I G is a reflection characteristic parameter of the metallic pigment, and is a parameter that specifies the intensity of the specular reflected light for each metallic pigment. The specular reflection parameter m G is a reflection characteristic parameter of the metallic pigment, and is a parameter that defines the spread of luster for each metallic pigment.

[0106] Also, the color signal C of the glittering agent G The process of calculating (λ) is performed in the area where the shining pigment is scattered within the makeup area 31. Specifically, the area where the shining pigment is scattered within the makeup area 31 can be estimated based on shape information that defines the size of the shining pigment particle shape, and dispersion information that defines the dispersion state of the shining pigment in the area where the cosmetic is actually applied. The area where the shining pigment is scattered within the makeup area 31 is estimated, and the color signal C of the shining pigment in that area is calculated. G By calculating (λ), the particle shape of the shining agent contained in the actual cosmetic and the dispersion state of the shining agent can be reproduced. Note that the shape information and dispersion information of the shining agent are stored in advance in the cosmetic database 24c.

[0107] In addition, the color signal C of the glittering agent G When calculating (λ), the color signal C of the glittering agent G A correction may be made to increase the spectral intensity at a specific wavelength in (λ). The third three-dimensional reflection characteristic model may include parameters such as an orientation parameter that defines the orientation of the metallic pigment, and a thickness d of each thin film 61.

[0108] In step S7, the third color signal calculation unit 23f executes a process of calculating a third color signal C3(λ), which is a color signal of the second 3D object 30b, based on the first color signal C1(λ) and the second color signal C2(λ). Note that in an area of ​​the makeup area 31 where multiple cosmetics are superimposed, the third color signal C3(λ) is calculated by adding up the second color signals C2(λ) of the respective cosmetics.

[0109] As a specific example, the third color signal calculation unit 23f calculates the first color signal C1(λ) and the second color signal C 2f (λ),C 2c The third color signal C3(λ) is calculated based on the following formula (14) using the second color signal C(λ) and weighting coefficients w1 to w3. 2f (λ) is the color signal of a foundation, which is an example of a cosmetic product. 2c (λ) is a color signal for cheek, which is an example of a cosmetic product. Weighting coefficients w1 to w3 are coefficients for weighting each color signal, and are determined appropriately depending on the thickness of the cosmetic product in the simulation, the order in which the cosmetic products are applied in layers, and the like.

[0110]

number

[0111] In this way, by calculating the third color signal C3(λ) based on the first color signal C1(λ) and the second color signal C2(λ), it is possible to calculate the third color signal C3(λ) in a state in which the color of the object to be applied is reflected in the cosmetic as the color signal of the object to be applied in a made-up state.

[0112] Even when simulating a state in which multiple cosmetics are superimposed, by calculating the third color signal C3(λ) based on the first color signal C1(λ) and the second color signal C2(λ) of each cosmetic, it is possible to calculate a color signal in which the color of the object to be applied and the color of the underlying cosmetic are reflected in the outermost cosmetic.

[0113] In step S7, the third color signal calculation unit 23f calculates the first color signal C1(λ), ​​the second color signal C2(λ), and the color signal C G It is also possible to execute a process of calculating a third color signal C3(λ) based on (λ).

[0114] As a specific example, the third color signal calculation unit 23f executes a process of calculating the third color signal C3(λ) based on the following formula (15) in an area of ​​the makeup area 31 where the foundation and the eye color containing pearl and glitter as a radiant are overlapped. 2e (λ) is the color signal of the eye color excluding the glitter. Color signal C Gp (λ) is the color signal of pearl, which is an example of a glittering agent contained in eye color. Color signal C Gl (λ) is the color signal of glitter, which is an example of a glittering agent contained in eye color.

[0115]

number

[0116] In this way, the first color signal C1(λ), ​​the second color signal C2(λ), and the color signal C G By calculating a color signal of the makeup state based on (λ), a third color signal C3(λ) that reflects the color and luster of the shining agent contained in the cosmetic can be calculated.

[0117] In step S8, the second RGB value calculation unit 23g executes a process of calculating a second RGB value, which is an RGB value of the second 3D object 30b, based on the third color signal C3(λ) calculated by the third color signal calculation unit 23f. Specifically, the second RGB value calculation unit 23g executes a process of calculating the second RGB value based on the following formula (16) using the third color signal C3(λ) and an RGB color matching function. Note that R2, G2, and B2 on the left side of formula (16) are the values ​​of each element in the second RGB value.

[0118]

number

[0119] In step S9, the object generating unit 23a executes a process of generating a second three-dimensional object 30b based on the second RGB value calculated by the second RGB value calculating unit 23g. Furthermore, the object generating unit 23a executes a process of transmitting data of the generated second three-dimensional object 30b to the user terminal 10 and displaying it on the touch panel 13.

[0120] As shown in Fig. 11, the second three-dimensional object 30b is displayed in the three-dimensional object display area 41 of the application image 40 on the touch panel 13. In the makeup area 31 of the second three-dimensional object 30b, the skin color, which is an example of an application target, the color of the base cosmetic, and the skin in a made-up state, reflecting the shadows caused by the illumination light, are displayed. In addition, in the makeup area 31, the light receiving angle θ r is the incidence angle θ i In the glossy area 32, which is an area similar to the above, the glossiness due to specular reflection is reproduced and displayed. Through the above processing, the makeup simulation is completed.

[0121] It should be noted that, by selecting a different cosmetic while the second three-dimensional object 30b is displayed, the processing of steps S3 to S8 can be executed again to generate and display the second three-dimensional object 30b to which the different cosmetic has been applied. In this case, for example, the processing of steps S3 and S4 may be omitted by storing the first color signal C1(λ) once calculated in the main memory of the memory 24. In this way, by repeatedly simulating various cosmetics, it is possible to find a cosmetic that suits one's desires.

[0122] By calculating the first color signal C1(λ) and the second color signal C2(λ) based on the illumination spectral distribution E(λ) for each illumination environment and the spatial distribution of the illumination light source, the illumination environment is reflected in the second three-dimensional object 30b as a simulation result. Therefore, it is possible to perform a makeup simulation that reflects the lighting effects of various actual illumination environments. In addition, by using the illumination spectral distribution E(λ) calculated from the actual measurement value of the spectral distribution in the actual illumination environment, various illumination environments can be easily reproduced.

[0123] By having the GPU 23 of the server 20 execute the processes of steps S2 to S8, the processes of steps S2 to S8 can be performed at high speed, enabling a real-time makeup simulation.

[0124] In addition, for example, by using the makeup simulation system 1 to perform simulations of various cosmetics for the user 2, counseling can be performed to suggest the most suitable cosmetics in accordance with the needs of the user 2.

[0125] [Effects of the first embodiment] According to the first embodiment, the following effects can be obtained. (1-1) By calculating the first color signal C1(λ) using the first three-dimensional reflection characteristic model and calculating the second color signal C2(λ) using the second three-dimensional reflection characteristic model, a third color signal C3(λ) in which the color of the application target is reflected in the cosmetic can be calculated as a color signal of the application target in a made-up state. Then, by calculating a second RGB value from the third color signal C3(λ), a second three-dimensional object 30b including the application target in a made-up state can be generated. As a result, a second three-dimensional object 30b in a made-up state in which the color of the application target is reflected in the cosmetic can be obtained as a simulation result. Therefore, a makeup simulation with higher reproduction accuracy can be performed than when the color of the cosmetic is simply superimposed on the area of ​​the application target. In addition, since the gloss caused by the illumination light can be reproduced, the texture of the state in which the cosmetic is applied to the application target can be reproduced in the second three-dimensional object 30b.

[0126] (1-2) By calculating the first spectral reflectance S1(λ) based on the first conversion function M corresponding to the color of the skin or lips as the object to be applied, it is possible to calculate the first spectral reflectance S1(λ) that is closer to the spectral reflectance of actual skin or lips.

[0127] (1-3) By calculating the second spectral reflectance S2(λ) based on the first spectral reflectance S1(λ) and the second conversion function T(λ) corresponding to the color of the object to be coated, it is possible to calculate the second spectral reflectance S2(λ) that is closer to the spectral reflectance of the cosmetic when it is actually applied to the object to be coated and blended with the object to be coated.

[0128] (1-4) By calculating a second color signal C2(λ) using the second spectral reflectance S2(λ) and the second reflection characteristic parameters for each cosmetic product, a second three-dimensional object 30b that reflects the color and reflection characteristics of each cosmetic product can be generated as a simulation result.

[0129] (1-5) Using the third three-dimensional reflection characteristic model, the color signal C of the glittering agent contained in the cosmetic product G Then, the first color signal C1(λ), ​​the second color signal C2(λ), and the color signal C of the metallic color pigment can be calculated. G By calculating the second RGB value from the third color signal C3(λ) calculated based on (λ), it is possible to generate the second three-dimensional object 30b in which the luster of the lustrous pigment in the cosmetic state is reproduced. G By calculating (λ), it is possible to reproduce the particle shape of the shining pigment and the dispersion state of the shining pigment actually contained in a cosmetic.

[0130] (1-6) By calculating the first color signal C1(λ) and the second color signal C2(λ) based on the illumination spectral distribution E(λ) for each illumination environment and the spatial distribution of the illumination light source, the illumination environment is reflected in the second three-dimensional object 30b as a simulation result. Therefore, a makeup simulation with high reproduction accuracy that reflects the illumination environment can be performed.

[0131] (1-7) By using the GPU 23 to execute the processes of steps S2 to S8 in the makeup simulation system 1, real-time image processing becomes possible, and the makeup simulation can be performed more appropriately.

[0132] (1-8) By using the makeup simulation system 1 to perform simulations of various cosmetics for the user 2, it is also possible to provide counseling to suggest the most suitable cosmetics to suit the needs of the user 2.

[0133] Second Embodiment In the above first embodiment, an example has been described in which the third color signal C3(λ) is calculated based on the first color signal C1(λ) and the second color signal C2(λ) calculated by the calculation process based on the spectral reflectance, and the second RGB value is calculated based on the third color signal C3(λ). Instead of the above process, the second RGB value can be obtained by performing the calculation process based on the RGB value. As the second embodiment, the operation when the calculation process based on the RGB value is performed will be described below.

[0134] The RGB value-based calculation process for obtaining the second RGB value is executed in an area of ​​the makeup area 31 where high reproduction accuracy is not required. That is, in an area where high reproduction accuracy is required, the second RGB value is calculated by the spectral reflectance-based calculation process described in the first embodiment. In the second embodiment, the RGB value-based calculation process for obtaining the second RGB value is executed by the second RGB value calculation unit 23g as the process after step S2 shown in FIG.

[0135] First, the second RGB value calculation unit 23g calculates the first RGB value and the first diffuse reflection function D a , the first specular reflection function G a, and the RGB value of the illumination light, a process is executed to calculate a third RGB value. The third RGB value is the RGB value of the object to be coated in a state in which the illumination light is irradiated onto the object to be coated and lighting effects such as shadows and gloss are reflected. The third RGB values ​​R3, G3, and B3 can be expressed by the following formula (17). In addition, when the RGB value of the illumination light R L ,G L ,B L can be expressed by the following equation (18) using the illumination spectral distribution E(λ) and RGB color matching functions: The RGB values ​​of the illumination light are stored in the illumination environment database 24b for each illumination environment.

[0136]

number

[0137] Next, the second RGB value calculation unit 23g executes a process of calculating a fourth RGB value based on the first RGB value and the cosmetic conversion coefficient. The fourth RGB value is the RGB value of the cosmetic when the cosmetic is applied to the application target and blends with the application target. In other words, the fourth RGB value does not reflect lighting effects such as shadows and gloss caused by the lighting light in the lighting environment selected in step S2. The cosmetic conversion coefficient is a coefficient calculated based on the second conversion function T(λ) and is a coefficient for converting the first RGB value to the fourth RGB value. The cosmetic conversion coefficient is stored in advance in the cosmetic database 24c for each cosmetic. The fourth RGB values ​​R4, G4, B4 can be expressed by the following formula (19). The cosmetic conversion coefficient R T, G T, B T can be expressed by the following equation (20) using the second conversion function T(λ) and the RGB color matching functions.

[0138]

number

[0139] Next, the second RGB value calculation unit 23g calculates the fourth RGB value and the second diffuse reflection function D b , the second specular reflection function Gb and the RGB values ​​of the illumination light, a process is executed to calculate a fifth RGB value. The fifth RGB value is an RGB value of the cosmetic that reflects the illumination effects such as shadows and gloss caused by irradiating the cosmetic with the illumination light. The fifth RGB values ​​R5, G5, and B5 can be expressed by the following formula (21).

[0140]

number

[0141] Then, the second RGB value calculation unit 23g executes a process of calculating a second RGB value based on the third RGB value, the fifth RGB value, and the weighting coefficient w. As a specific example, in an area of ​​the makeup area 31 where the foundation and the blush overlap, the second RGB value calculation unit 23g executes a process of calculating a second RGB value based on the following formula (22). 5f ,G 5f ,B 5f is the fifth RGB value of the foundation. 5c ,G 5c ,B 5c is the fifth RGB value of blush.

[0142]

number

[0143] It is difficult to perform calculations based on RGB values ​​for the shining pigments contained in cosmetics. Therefore, the sixth RGB value that reproduces the interference color of the shining pigment is calculated as the color signal C of the shining pigment based on the spectral reflectance using the above formula (12). G After calculating (λ), the color signal C of the glittering agent G The sixth RGB values ​​R6, G6, and B6 are calculated from (λ) and RGB color matching functions. The process of calculating the sixth RGB value is executed by the second RGB value calculation unit 23g. The sixth RGB values ​​R6, G6, and B6 can be expressed by the following formula (23).

[0144]

number

[0145] The second RGB value calculation unit 23g can also execute a process of calculating the second RGB values ​​in a state where a cosmetic containing a shining agent has been applied, based on the third RGB value, the fifth RGB value, the sixth RGB value, and the weighting coefficient w. As a specific example, the second RGB value calculation unit 23g executes a process of calculating the second RGB values ​​R2, G2, B2, based on the following formula (24), in an area of ​​the makeup area 31 where a foundation and an eye color containing pearl and glitter as a shining agent are superimposed. 5e ,G 5e ,B 5e is the fifth RGB value of the eye color excluding the glitter. 6p ,G 6p ,B 6p is the 6th RGB value of pearl, an example of a glittering agent contained in eye color. 6l ,G 6l ,B 6l is the 6th RGB value of glitter, which is an example of a glittering agent contained in eye color.

[0146]

number

[0147] The second RGB values ​​calculated by the RGB value-based calculation process as described above are used by the object generating unit 23a to execute the process of generating the second three-dimensional object 30b in step S9 shown in FIG.

[0148] When the second RGB value is calculated by RGB value-based calculation, three-dimensional calculation is performed for the three colors of red, green, and blue. On the other hand, when the second RGB value is calculated based on the color signal calculated from the spectral reflectance as in the first embodiment, 61-dimensional calculation is performed by dividing the visible light region of 400 nm to 700 nm into 5 nm intervals.

[0149] In other words, by calculating the second RGB values ​​by RGB value-based calculation processing, the calculation processing in the makeup simulation can be simplified compared to the processing of calculating the second RGB values ​​based on color signals calculated from spectral reflectance. Therefore, by performing RGB value-based calculation processing in an area of ​​the makeup area 31 where high reproduction accuracy is not required, the calculation processing can be simplified without significantly decreasing the reproduction accuracy of the makeup simulation.

[0150] [Effects of the second embodiment] According to the second embodiment, the following effects can be obtained. (2-1) By calculating the second RGB values ​​by RGB value-based calculation processing, the calculation processing in the makeup simulation can be simplified compared to the processing of calculating the second RGB values ​​based on color signals calculated from spectral reflectance. Therefore, by performing RGB value-based calculation processing in an area of ​​the makeup area 31 that does not require high reproduction accuracy, the calculation processing can be simplified without significantly decreasing the reproduction accuracy of the makeup simulation.

[0151] The second embodiment can be further modified as necessary as follows. The configuration in which the second RGB value is calculated based on the third RGB value and the fifth RGB value calculated using the RGB value-based calculation process has been exemplified. However, the present invention is not limited to this, and either the third RGB value or the fifth RGB value may be calculated by a spectral-based calculation. Specifically, instead of the third RGB value, a value obtained by converting the first color signal C1(λ) into an RGB value using an RGB color matching function may be used. Also, instead of the fifth RGB value, a value obtained by converting the second color signal C2(λ) into an RGB value using an RGB color matching function may be used. With this configuration, only any calculation process that does not require accuracy can be calculated based on RGB values, so that the calculation process of the makeup simulation can be more suitably simplified.

[0152] The above first and second embodiments can also be modified as necessary as follows. In the above first and second embodiments, a configuration has been exemplified in which the subject on which the makeup simulation is performed is a face, and the application object on which the cosmetics are applied is the skin or lips of the face. However, this is not limited thereto, and the present makeup simulation system 1 can be applied to any area on which the cosmetics are applied, even if it is not the face. The application object on which the cosmetics are applied may be, for example, hair. In that case, the applied cosmetics is, for example, hair nail polish. Furthermore, the application object on which the cosmetics are applied may be, for example, nails. In that case, the applied cosmetics is, for example, nail polish.

[0153] Although the configuration in which the GPU 23 is used to execute the processing of steps S2 to S8 in the makeup simulation system 1 has been exemplified, the present invention is not limited to this, and for example, the processing of steps S2 to S8 may be executed by the CPU 22 without providing a GPU 23 in the server 20. In this case, although the processing speed of steps S2 to S8 in the server 20 will decrease, it is possible not to provide a GPU 23. Also, for example, the CPU 22 may execute any of the processing of steps S2 to S8 depending on the performance of the CPU 22.

[0154] The configuration in which the first color signal C1(λ) and the second color signal C2(λ) are calculated based on the illumination spectral distribution E(λ) and the spatial distribution of the illumination light source for each illumination environment has been exemplified. However, the present invention is not limited to this, and for example, the first color signal C1(λ) and the second color signal C2(λ) may be calculated using only one type of illumination spectral distribution E(λ) and the spatial distribution of the illumination light source. In this case, although it becomes impossible to perform a makeup simulation for each illumination environment, since a procedure for selecting an illumination environment is not required, the process of calculating the first color signal C1(λ) and the second color signal C2(λ) is simplified. In addition, for example, if a spectral distribution having a uniform intensity in the visible light region of 400 to 700 nm is used as the illumination spectral distribution E(λ), the calculation of the illumination spectral distribution E(λ) can be omitted in the first to third three-dimensional reflection characteristic models.

[0155] In step S5, the second spectral reflectance S2(λ) is calculated based on the first spectral reflectance S1(λ) and the second conversion function T(λ) corresponding to the color of the object to be applied. However, the present invention is not limited to this, and the spectral reflectance of the cosmetic may be actually measured using a measuring device such as a spectrophotometer, and the actual measured value of the spectral reflectance of the cosmetic may be used as the second spectral reflectance S2(λ). In this case, although the reproduction accuracy of the second color signal C2 is slightly lower than when the second spectral reflectance S2(λ) is calculated based on the first spectral reflectance S1(λ) and the second conversion function T(λ), it is possible to easily perform a makeup simulation for various cosmetics. For example, when a new cosmetic is developed, the second spectral reflectance and the second reflection characteristic parameter of the new cosmetic can be actually measured to perform a makeup simulation in the present makeup simulation system 1. In this case, the process of step S5 is omitted.

[0156] The configuration in which the first spectral reflectance S1(λ) is calculated based on the first conversion function M according to the color of the application target has been exemplified. However, the present invention is not limited to this, and the first spectral reflectance S1(λ) may be calculated based on a general-purpose first conversion function M set for each application target such as skin or lips, regardless of the color of the application target. In this case, although the difference between the second three-dimensional object 30b as a simulation result and the actual appearance after makeup becomes slightly larger, the process of determining the first conversion function M according to the first RGB value constituting the first three-dimensional object 30a or the procedure of the user 2 selecting a color close to the application target from a color sample is not required. Therefore, the process of calculating the first spectral reflectance S1(λ) from the first RGB value can be simplified.

[0157] Although an example has been given of a configuration in which a facial image of the user 2 is acquired using the imaging unit 14 provided in the user terminal 10, this is not limited to this. For example, the first three-dimensional object 30a may be generated using a facial image acquired by an external device such as a digital camera.

[0158] The configuration has been exemplified in which the object generating unit 23a executes a process of generating the first three-dimensional object 30a from a facial image of the user 2 without makeup. However, the present invention is not limited to this. For example, a three-dimensional object of a face without makeup generated by other software may be imported and used as the first three-dimensional object 30a. In this case, the RGB values ​​constituting the imported three-dimensional object become the first RGB values.

[0159] In the present embodiment, a configuration has been exemplified in which the GPU 23 included in the server 20 is used to execute the processing of steps S2 to S8 in the makeup simulation system 1. However, this is not limited to this, and for example, a GPU may be provided in the control unit 11 of the user terminal 10, a makeup simulation program may be installed in the user terminal 10, and the processing of steps S2 to S8 may be executed by the GPU included in the user terminal 10.

[0160] The makeup simulation system 1 may be configured to be capable of executing a process of purchasing the cosmetics applied to the second three-dimensional object 30b as a result of the simulation, for example, by using a three-dimensional object display program. In this case, the server 20 is provided with a payment unit that executes a payment process. This allows, for example, a user to search for cosmetics that suit their desires in the makeup simulation system 1 and then purchase the cosmetics. [Explanation of symbols]

[0161] 1. Makeup simulation system 10...User terminal 11...Control section 12…Memory 13...Touch panel 14…Imaging unit 20…Server 21...Control unit 23…GPU 23a…Object generation unit 23b...First spectral reflectance calculation section 23c…First color signal calculation unit 23d...Second spectral reflectance calculation section 23e... second color signal calculation unit 23f...Third color signal calculation section 23g…Second RGB value calculation section 24…Memory 24a...Spectral conversion database 24b…Lighting environment database 24c…Cosmetics Database 30…Three-dimensional object 30a…First three-dimensional object 30b…Second 3D object 31…Cosmetic area 32…Glossy area 40…App image

Claims

1. a first spectral reflectance calculation unit that calculates a first spectral reflectance of the application target before makeup from a first RGB value of the application target to which the cosmetic material is applied before makeup; a first color signal calculation unit that calculates a first color signal of the object to be coated before the makeup is applied, using the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristic of the object to be coated before the makeup is applied, and a first three-dimensional reflection characteristic model that is expressed by a spectral distribution of illumination light and geometric conditions; a second color signal calculation unit that calculates a second color signal of the cosmetic using a second three-dimensional reflection characteristic model expressed by a second spectral reflectance of the cosmetic, second reflection characteristic parameters that define the reflection characteristics of the cosmetic, the spectral distribution of the illumination light, and the geometric conditions; a third color signal calculation unit that calculates a third color signal of the application target in a cosmetic state after the cosmetic material is applied to the application target from the first color signal and the second color signal; a second RGB value calculation unit that calculates a second RGB value of the application target object in the makeup state from the third color signal; an object generation unit that generates a three-dimensional object including the application target object in the makeup state from the second RGB values.

2. a spectral conversion database that stores a first conversion function for converting the first RGB value into the first spectral reflectance, the first conversion function corresponding to a color of the application target before the makeup application; The first spectral reflectance calculation unit calculates the first spectral reflectance by using the first conversion function corresponding to a color of the application target before the makeup is applied. The makeup simulation system according to claim 1 .

3. a cosmetic database that stores a second conversion function for calculating the second spectral reflectance from the first spectral reflectance, the second conversion function corresponding to a color of the application target before the makeup is applied; A second spectral reflectance calculation unit calculates the second spectral reflectance by using the first spectral reflectance and the second conversion function according to a color of the application target object before the makeup application, The second color signal calculation unit calculates the second color signal using the second three-dimensional reflection characteristic model including the second spectral reflectance calculated by the second spectral reflectance calculation unit. The makeup simulation system according to claim 2 .

4. the cosmetic database stores the second conversion function and the second reflection characteristic parameter for each cosmetic product, the second spectral reflectance calculation unit calculates the second spectral reflectance by using the second conversion function for each cosmetic product; The second color signal calculation unit calculates the second color signal using the second three-dimensional reflection characteristic model including the second spectral reflectance calculated by the second spectral reflectance calculation unit and the second reflection characteristic parameter for each cosmetic product. The makeup simulation system according to claim 3 .

5. the cosmetic database stores a spectral reflectance of a shining agent contained in the cosmetic, a reflection characteristic parameter of the shining agent that defines the reflection characteristic of the shining agent, shape information that defines the size of the shining agent, and dispersion information that defines the dispersion state of the shining agent; the second color signal calculation unit calculates the second color signal of the cosmetic using the second three-dimensional reflection characteristic model, and further calculates a color signal of the lustrous agent in a region in which the lustrous agent is scattered, the region being estimated from the shape information and the dispersion information of the lustrous agent, using a third three-dimensional reflection characteristic model expressed by the spectral reflectance of the lustrous agent, the reflection characteristic parameters of the lustrous agent, the spectral distribution of the illumination light, and the geometric conditions; The third color signal calculation unit calculates the third color signal of the application target in the made-up state in which the cosmetic material containing the glitter is applied to the application target before makeup from the first color signal, the second color signal, and the color signal of the glitter. The makeup simulation system according to claim 4 .

6. At least one of the second spectral reflectance and the second reflection characteristic parameter is an actual measurement value obtained by actually measuring the cosmetic material. The makeup simulation system according to claim 1 .

7. a lighting environment database that stores, for each lighting environment, the spectral distribution of the illumination light and a spatial distribution of an illumination light source that generates the illumination light, the spatial distribution in all directions centered on the three-dimensional object; the first color signal calculation unit calculates the first color signal using the first three-dimensional reflection characteristic model including the spectral distribution of the illumination light for each of the lighting environments; The second color signal calculation unit calculates the second color signal using the second three-dimensional reflection characteristic model including the spectral distribution of the illumination light for each of the lighting environments. The makeup simulation system according to any one of claims 1 to 6.

8. The image forming apparatus further includes a GPU that functions as at least one of the first spectral reflectance calculation unit, the first color signal calculation unit, the second color signal calculation unit, the third color signal calculation unit, the second RGB value calculation unit, and the object generation unit. The makeup simulation system according to any one of claims 1 to 7.

9. calculating a first spectral reflectance of the application target before makeup from a first RGB value of the application target to which the cosmetic material is applied before makeup; calculating a first color signal of the object to be coated before makeup is applied using a first three-dimensional reflection characteristic model expressed by the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristic of the object to be coated before makeup is applied, and a spectral distribution and geometric conditions of illumination light; calculating a second color signal of the cosmetic using a second three-dimensional reflection characteristic model expressed by a second spectral reflectance of the cosmetic, second reflection characteristic parameters defining the reflection characteristics of the cosmetic, the spectral distribution of the illumination light, and the geometric conditions; calculating a third color signal of the application target in a made-up state in which the cosmetic material is applied to the application target before makeup from the first color signal and the second color signal; calculating a second RGB value of the application target object in the makeup state from the third color signal; generating a three-dimensional object including the application target object in the makeup state from the second RGB values. Makeup simulation method.

10. On the computer, A process of calculating a first spectral reflectance of the application target before makeup from a first RGB value of the application target to which the cosmetic material is applied before makeup; A process of calculating a first color signal of the application target before makeup is applied using a first three-dimensional reflection characteristic model expressed by the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristic of the application target before makeup is applied, and a spectral distribution and geometric conditions of illumination light; a process of calculating a second color signal of the cosmetic using a second three-dimensional reflection characteristic model expressed by a second spectral reflectance of the cosmetic, a second reflection characteristic parameter that defines the reflection characteristic of the cosmetic, the spectral distribution of the illumination light, and the geometric conditions; A process of calculating a third color signal of the application target in a made-up state in which the cosmetic material is applied to the application target before makeup from the first color signal and the second color signal; A process of calculating a second RGB value of the application target object in the makeup state from the third color signal; and generating a three-dimensional object including the application target object in the makeup state from the second RGB value. Makeup simulation program.

11. a first spectral reflectance calculation unit that calculates a first spectral reflectance of the application target before makeup from a first RGB value of the application target to which the cosmetic material is applied before makeup; a first color signal calculation unit that calculates a first color signal of the object to be coated before the makeup is applied, using the first spectral reflectance, a first reflection characteristic parameter that defines the reflection characteristic of the object to be coated before the makeup is applied, and a first three-dimensional reflection characteristic model that is expressed by a spectral distribution of illumination light and geometric conditions; a second color signal calculation unit that calculates a second color signal of the cosmetic using a second three-dimensional reflection characteristic model expressed by a second spectral reflectance of the cosmetic, second reflection characteristic parameters that define the reflection characteristics of the cosmetic, the spectral distribution of the illumination light, and the geometric conditions; a third color signal calculation unit that calculates a third color signal of the application target in a cosmetic state after the cosmetic material is applied to the application target from the first color signal and the second color signal; a second RGB value calculation unit that calculates a second RGB value of the application target object in the makeup state from the third color signal; an object generating unit that generates a three-dimensional object including the application target object in the makeup state from the second RGB value; Makeup simulation device.

Images