Image generation method, makeup simulation method, image generation apparatus, and image generation program

Abstract

To provide a technology that can capture the characteristics of glittering powder and reproduce them as an image with high precision and realism.SOLUTION: An image generation method comprises: a glittering agent information acquisition step for acquiring at least one of optical property information and shape information of glittering powder; a color signal calculation step for calculating a color signal of the glittering powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes a light reflection process of the glittering powder; a color value calculation step for calculating a color value of the state in which the glittering powder is applied based on the color signal; and an image generating step for generating an image of the state in which the glittering powder is applied based on the color value and the shape information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image generation method, a makeup simulation method, an image generation device, and an image generation program. [Background technology]

[0002] Traditionally, the use of glittering powders (pearl pigments) in cosmetics and automotive paints has given products a beautiful shine and a luxurious feel. These powders play an important role in a wide range of fields, including makeup products and automotive exteriors, due to their unique reflective properties and texture. Recent advances in computer graphics technology have made it possible to simulate these glittering powders on computers, and cosmetic manufacturers and automotive designers are attempting to replicate the properties of these powders to conduct more precise product evaluations and design studies.

[0003] For example, Patent Document 1 discloses a method for generating a glitter pattern image that imitates the pattern of a glitter pattern coating film formed by applying a glitter pattern forming paint onto a base of any solid color. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-22540 Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional simulation techniques have found it difficult to accurately and realistically reproduce the visual texture of glittering powder, particularly its color changes, brightness fluctuations, and light reflection characteristics due to particle shape.

[0006] Therefore, a main object of the present invention is to provide a technology that can capture the characteristics of these glittering powders and reproduce them as images with high precision and realism. [Means for solving the problem]

[0007] The present invention provides a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the state in which the glitter powder is applied based on the color signal; and an image generating step of generating an image of a state in which the glitter powder is applied based on the color value and the shape information. the glitter powder contains at least one of an interference pearl pigment and a colored pearl pigment, In the color signal calculation step, the optical property information of the interference pearl pigment is given to an interference system three-dimensional reflection property model to calculate the color signal of the interference pearl pigment; In the color signal calculation step, optical property information of the colored pearl pigment may be provided to a color system three-dimensional reflection property model to calculate a color signal of the colored pearl pigment. The coloring system three-dimensional reflection characteristic model may include at least one of a physical model that describes the physical light reflection characteristics of the lustrous powder and an empirical model that describes the light reflection characteristics of the lustrous powder based on a hypothesis. The glittering powder is configured by coating a base powder with one or more thin layers, In the color signal calculation step, the wavelength of the interference light that depends on the thickness of the thin film may be applied to the three-dimensional reflection characteristic model to calculate the color signal. The optical property information acquired in the metallic color information acquisition step may include the wavelength. The shape information acquired in the glittering agent information acquisition step includes an average particle size and a standard deviation of the glittering powder, In the image generating step, the image may be generated using a probability distribution based on the average particle size and the standard deviation. The image may be represented using a normal map. The glittering powder in the image may have a polygonal shape in plan view. The glittering powder in the image may have a particle size of 3 mm or less. the optical characteristic information includes a spectral reflectance, In the lustrous powder information acquisition process, the spectral reflectance acquired by the spectrophotometer may be the spectral reflectance of light reflected in a direction between an illumination direction vector and the surface of the object to which the lustrous powder is applied, or the spectral reflectance of light reflected in a direction between a specular reflection direction vector of the illumination direction vector and the surface. When the horizontal direction with respect to the surface is set to 0 degrees, the reflection angle of light acquired by the spectrocolorimeter may be 25 degrees or less. The present invention also provides a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder which is a cosmetic; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the state in which the glitter powder is applied based on the color signal; and an image generating step of generating an image of the glitter powder applied based on the color value and the shape information. The present invention also provides a glittering agent information acquisition unit that acquires at least one of optical property information and shape information of the glittering powder; a color signal calculation unit that calculates a color signal of the glitter powder by providing the optical property information to a three-dimensional reflection property model that is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation unit that calculates a color value of the state in which the glitter powder is applied based on the color signal; and an image generating unit that generates an image of the state in which the glitter powder is applied based on the color value and the shape information. The present invention also provides a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the state in which the glitter powder is applied based on the color signal; and an image generating step of generating an image of the state in which the glittering powder is applied based on the color value and the shape information. [Effects of the Invention]

[0008] The present invention provides a technology that can capture the characteristics of glittering powder and reproduce them as an image with high precision and realism. Note that the effects of the present invention are not limited to the effects described herein, and may be any of the effects described in this specification. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a flowchart illustrating an example of the flow of an image generation method according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram illustrating a geometric model of reflection according to an embodiment of the present invention. [Figure 3] 3 is a schematic diagram showing a reflected light distribution in the geometric model of reflection shown in FIG. 2. FIG. [Figure 4] FIG. 2 is a schematic cross-sectional view showing an example of the configuration of a glittering powder. [Figure 5] 1 is a schematic diagram showing an example of the configuration of an image generation system to which an image generation method according to an embodiment of the present invention is applied; [Figure 6] FIG. 1 is a block diagram showing an example of the configuration of a user terminal to which an image generation method according to an embodiment of the present invention is applied. [Figure 7] 1 is a block diagram showing an example of the configuration of a server to which an image generation method according to an embodiment of the present invention is applied; [Figure 8] 8 is a block diagram showing an example of the functional configuration of the GPU shown in FIG. 7. [Figure 9] 10 is an example of a screen displayed in a makeup simulation method according to an embodiment of the present invention. [Figure 10] 10 is an example of a screen displayed in a makeup simulation method according to an embodiment of the present invention. [Figure 11] 1 is a block diagram showing an example of the configuration of an image generating device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] [1. First embodiment (example of image generation method)] [(1) Overview] Pearlescent powders (pearl pigments) are special pigments that exhibit luster and interference colors, providing unique visual effects in a wide range of products, including cosmetics, automotive paints, and plastics.

[0011] The present invention provides a technology for reproducing such glittering powder with high accuracy on a computer. Specifically, the present invention provides an image generation method including: a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder; a color signal calculation step of calculating a color signal of the glittering powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model describing the light reflection process of the glittering powder; a color value calculation step of calculating a color value of the glittering powder based on the color signal; and an image generation step of generating an image of the glittering powder in an applied state based on the color value and the shape information.

[0012] In the following description, "image" is a concept that includes two-dimensional and three-dimensional still images as well as two-dimensional and three-dimensional videos, and "video" is a concept that includes video footage captured in real time and video footage that has already been captured.

[0013] In the present invention, the "image of the state in which the glittering powder is applied" refers to an image that depicts in detail the distribution and shape of the glittering powder. This image may be based on actual measurement data obtained using a measuring device such as a microscope, or may be automatically generated using the method described below.

[0014] The image generation method provided by the present invention will be described with reference to Fig. 1. Fig. 1 is a flowchart showing an example of the flow of an image generation method according to one embodiment of the present invention.

[0015] As shown in FIG. 1, in the glittering agent information acquisition step (step S1), an acquisition unit included in a computer acquires at least one of optical property information and shape information of the glittering powder.

[0016] The optical property information is a set of indices for quantifying the optical behavior of the glittering powder, including information on how the glittering powder interacts with light, such as reflectance, color, and interference light.

[0017] Shape information indicates the shape and dimensions of the glittering powder. This shape information includes, for example, particle shape, particle size distribution, surface roughness, particle size, and the like. Furthermore, this shape information also includes minute variations in particle shape. For example, to represent particles with non-uniform shapes, shape variations are taken into account by distorting each side of polygonal particles. This distortion of each side of the polygonal particles is intended to reproduce the non-uniformity and natural appearance of particles seen in reality, and polygonal particles are represented by applying random numbers to the length of each side.

[0018] The acquisition unit acquires one or both of the optical property information and the shape information. The optical property information may be acquired by any method, such as measurement using a measuring device or input from a computer display.

[0019] An example of an acquisition unit is a spectrophotometer. A spectrophotometer is a measuring device for measuring the color of an object. Unlike conventional colorimeters, a spectrophotometer can acquire more detailed color information by measuring the reflectance of each wavelength of light.

[0020] A spectrophotometer consists of elements such as a light source, a spectroscope, and a detector. The light source shines light onto an object. The spectroscope separates the light into wavelengths. The detector detects the separated light and measures the reflectance of each wavelength (spectral reflectance). The data obtained from these elements is converted into a color space and displayed as color values. Color values ​​are a numerical representation of how a color appears. Color values ​​can be expressed in color systems such as RGB, CMYK, HSV, Lab, CIE XYZ, and yCbCr.

[0021] As mentioned above, color values ​​can also be acquired by inputting them from a computer display. Specifically, a user can directly input color information such as RGB values ​​through a computer display. In this case, the computer display, keyboard, etc. are used as the acquisition unit.

[0022] The image generation method of the present invention uses a mathematical model that describes the reflectance characteristics of the glitter powder based on the optical property information and shape information to calculate a color signal, and then calculates color values ​​based on this to generate an image of the powder applied. This process makes it possible to reproduce the visual effect of the actual glitter powder with high accuracy. By adopting this input method, users can directly and easily provide the color information they desire, realizing a more intuitive and flexible image generation process.

[0023] In the image acquisition step (step S2), a computing unit (e.g., a CPU, GPU, etc.) in a computer acquires an image of the object to be coated with the pigment. During this process, the computing unit receives image data from an external source and stores it in memory for processing. The acquired image is the basis for an image that can be analyzed, processed, or generated based on a specific processing procedure. This acquisition process is a necessary step prior to image generation, analysis, or other image processing techniques, and plays an important role in subsequent processing.

[0024] This image is an image of the state before the pigment is applied. Here, the image of the state before the pigment is applied is defined as a first image. This first image may be, for example, a single still image, multiple still images taken at different angles, or a three-dimensional model.

[0025] When the present invention is applied to makeup simulation, the first image can be a facial image. This captures the state of the face before pigment application and serves as a basis for simulating the visual impact of cosmetic colors and textures on the face. Similarly, when the present invention is used in automobile painting simulation, the first image can be an image of the automobile. In this case, the appearance of the automobile before painting is recorded and serves as a reference point for evaluating the visual effects that different paint colors and finishes have on the automobile. As such, the present invention has a wide range of applications and can be used to reproduce visual effects in a variety of fields, from cosmetic application simulation to automobile painting simulation.

[0026] When the first image is captured, it is affected by various factors such as lighting in the shooting environment. From the viewpoint of obtaining a first image that is more suitable for simulation, corrections to reduce the influence of the shooting environment can be made, for example, by automatically adjusting the brightness and saturation of the captured first image according to the surrounding lighting environment using an image processing program stored in memory. In addition, a database can be used to reproduce an image that assumes how the pigment will appear (color change) in a scene different from the scene at the time of capture.

[0027] In the shape estimation step (step S3), the calculation unit estimates the three-dimensional shape of the application target (e.g., a face, a car, etc.) from the first image. Specifically, the calculation unit calculates normal vectors required to reproduce makeup, skin texture, etc.

[0028] A normal vector is a vector that indicates the inclination or orientation of a surface at a point on that surface, and is a concept used in various fields such as geometry, graphics, and physically based rendering. It is perpendicular to a surface, such as a flat or curved surface, and indicates the direction in which the surface faces outward. Calculating this normal vector makes it possible to estimate the precise three-dimensional shape of the object to be painted and generate more realistic images based on that.

[0029] Normal vectors play an important role in the three-dimensional reflection property model described below and are closely related to the expression of shading and gloss. In particular, when utilizing augmented reality (AR) technology, it is important to continuously calculate normal vectors in real time in response to changes in facial orientation and facial expression in situations where the texture of the object being painted or the pigment changes from moment to moment. This continuous calculation of normal vectors makes it possible to respond immediately to changes in the object's texture and provide the user with a more realistic visual experience. This process is used in various real-time graphics processes, including AR technology, to more accurately reproduce the texture and light reflection of the object.

[0030] In the surface identification step (step S4), the calculation unit identifies the uneven shape of the surface of the application object from the first image as a mathematical model. In particular, when the application object is skin, the calculation unit identifies the minute unevenness of the skin surface using the mathematical model. These minute unevenness refers to the uneven shape of the skin surface, and specific examples include wrinkles, texture, pimples, and scars.

[0031] These features that affect skin texture are represented by a mathematical model as variations in shading and color, rather than providing shape information directly. In this way, differences in texture, such as wrinkles and texture, can be reproduced by adjusting the parameters of the mathematical model. Therefore, skin texture can be estimated by estimating these parameters from an image.

[0032] These mathematical models may include, for example, the Torrance-Sparrow model or the Oren-Nayar model, which mathematically represent optical properties and are used to simulate the surface reflectance properties of objects in the fields of computer graphics and computer vision.

[0033] The Torrance-Sparrow model is a physics-based model based on microfacet theory that describes the behavior of tiny surface elements (microfacets) when reflecting light. The V-groove model in this model mathematically represents the effect of surface irregularities on reflection.

[0034] On the other hand, the Oren-Nayar model takes into account the scattering of light when the surface is rough, and models the intensity and directionality of reflected light, making it particularly suitable for reproducing the texture of rough surfaces.

[0035] In the lighting environment estimation step (step S5), the calculation unit uses the first image to estimate the lighting environment in which the first image was captured. The lighting environment is a combination of the illumination light spectral distribution E(λ) and the spatial distribution of the illumination light source to reproduce an illumination effect suitable for the object to be coated in a specific scene. The illumination light spectral distribution E(λ) is the spectral distribution of the illumination light that may be irradiated onto the object to be coated.

[0036] Examples of specific lighting environments include indoor environments that use artificial direct light sources and outdoor environments that use natural direct light sources. Outdoor environments are sometimes divided into time periods such as morning, noon, evening, and night. Examples of artificial direct light sources include incandescent bulbs, fluorescent lights, and LED lights, while examples of natural direct light sources include the sun and the moon.

[0037] The processing in the lighting environment estimation step (step S5) will be described in detail. The calculation unit estimates an illumination direction vector from the first image. The illumination direction vector is a vector that indicates the direction from which light comes at a certain point or surface. At the same time, the calculation unit uses a system conversion matrix for obtaining the illumination light spectral distribution E(λ) from the color value, and derives the illumination light spectral distribution E(λ) from the color value of the first image. This system conversion matrix can be generated from a database that stores color signals obtained by multiplying the spectral distribution of a known light source by the spectral reflectance of various objects to be coated.

[0038] Based on the illumination direction vector and the illumination light spectral distribution E(λ), the calculation unit estimates the spatial distribution of the illumination light source, thereby estimating the illumination environment in the first image. This estimated illumination environment information (scene information) is also recorded as independent illumination environment information and can be applied to images captured in other scenes. For example, lighting environment information estimated in an office can be used to reproduce a video of a face captured in another location.

[0039] As an example of the process in which the calculation unit estimates the illumination direction vector, a case where a face image is used will be described. First, a normal vector corresponding to each pixel of the face image is calculated based on information about the three-dimensional shape of the face. A normal vector is a vector that indicates the perpendicular direction of the surface at a given point on the surface of an object. Normal vectors generally have different directions for each point on the surface, and change depending on the shape and curve of the object.

[0040] If the face can be assumed to be a convex hull object based on the cheeks, forehead, etc., then if the object is not shiny (matte reflection), the lighting direction can be estimated by finding the brightest pixel compared to its surroundings, as the normal vector of that pixel will match the lighting direction. If the object is shiny, the lighting direction can be estimated by finding the brightest pixel compared to its surroundings, as the specular reflection direction vector of the gaze direction vector related to the normal vector of that pixel will match the lighting direction. The gaze direction vector is a vector that indicates the direction of the viewpoint as seen by the observer or camera.

[0041] Furthermore, the calculation unit may use a lighting environment estimation image, which is an image different from the first image, to estimate the lighting environment in which the lighting environment estimation image was captured. This lighting environment estimation image may be, for example, an image that does not include an object to be coated, or an image that includes an object to be coated. Therefore, for example, by obtaining information on ambient light and cast match from an image that does not include an object to be coated, it becomes possible to estimate a lighting environment that is not limited to the first image. Note that the first image and the lighting environment estimation image may be the same.

[0042] In the application area receiving step (step S6), the computer receives information on the application area of ​​the object through input from the user. If the object is a face, the user can specify the specific area to apply the pigment using an interface such as a touch panel displaying a facial image. For example, if the user wants to apply pigment to the eyelids, they set the application area by directly selecting or drawing the eyelid area on the displayed facial image. Through this process, the user can intuitively and accurately specify the area to apply the pigment, and as a result, can freely control the position and range of the pigment in the generated image. This input information is processed by the calculation unit within the system and used in subsequent steps.

[0043] In the first spectral reflectance calculation step (step S7), the calculation unit calculates the first spectral reflectance S1(λ), ​​which is the spectral reflectance of the coated area, based on the first color values, which are the color values ​​that make up the first image. Spectral reflectance is a ratio that indicates how much light is reflected for each wavelength when an object receives light. Since the color of light usually varies depending on the wavelength, spectral reflectance is an index that represents the color of an object.

[0044] Specifically, the first spectral reflectance calculation step (step S7) involves the calculation unit reading out a first conversion function M corresponding to the color of the object in the application area, and calculating the first spectral reflectance S1(λ) in the application area. The first conversion function M may be stored in a database in a computer, for example.

[0045] The first conversion function M is a function for converting color values ​​into spectral reflectance. The first conversion function M is a conversion matrix obtained by measuring the color values ​​and spectral reflectance of objects to be treated for a large number of people and statistically analyzing the data. The first conversion function M is obtained by investigating the correspondence between the color value and spectral reflectance of the object to be treated for each measured subject and performing statistical analysis based on the results. The first conversion function M can be expressed, for example, as a 61 x 3 conversion matrix that divides the visible wavelength range of 400 nm to 700 nm into 5 nm intervals. This matrix indicates the relationship between the color of the object, its color value, and spectral reflectance, and is statistically derived from actual data. Using this conversion matrix, it is possible to estimate the spectral reflectance of the object from its color value.

[0046] The first conversion function M may be automatically determined based on the first color value of the application area of ​​the first image, or may be determined by the user selecting a color similar to the color of the application target from a color sample such as a color code.

[0047] The following description will be given taking the case where the color values ​​are RGB values ​​as an example. The first spectral reflectance S1(λ) can be approximated as a discrete 61-dimensional spectral reflectance vector s, which divides the visible wavelength range of 400 nm to 700 nm at 5 nm intervals, using the first conversion function M and the values ​​of each element in the first color value, R1, G1, and B1. The first spectral reflectance S1(λ) can be expressed by the following equations (1) and (2).

[0048]

number

[0049] The first spectral reflectance S1(λ) is calculated based on a first conversion function M corresponding to the color of the object to be coated, thereby making it possible to obtain a value close to the actual spectral reflectance. For example, in the case of facial skin, it can be difficult to directly measure the spectral reflectance of the entire face because the measurement range of measuring instruments such as spectrophotometers is usually narrow. In contrast, in this embodiment, the first spectral reflectance S1(λ) is calculated from the first color value of an image of a face without makeup, making it possible to effectively obtain the spectral reflectance of the object to be coated without makeup.

[0050] Note that the first spectral reflectance calculation step (step S7) does not need to be performed on areas where no pigment is applied, such as hair, i.e., areas other than the applied area, which simplifies the process of calculating the first spectral reflectance S1(λ).

[0051] In the first color signal calculation step (step S8), the calculation unit calculates the first color signal C1(λ) based on the first spectral reflectance S1(λ), ​​the first reflection characteristic parameter, the illumination light spectral distribution E(λ), and the first three-dimensional reflection characteristic model.

[0052] The 3D reflectance model is a mathematical model that represents the light reflection process of an object, and is an important element in defining the color and texture of an object. The core of this model is the spectral reflectance and reflectance parameters, which determine how an object appears.

[0053] The reflectance characteristic parameters depend on the surface properties of the object, especially the surface roughness, and define the intensity of the reflected light and the extent of the gloss depending on the angle of incidence of the illumination light on the object surface. This determines how the object surface reflects light, i.e., how its gloss and texture appear.

[0054] The first 3D reflectance model represents the state before the pigment is applied. By applying geometric conditions such as the spectral distribution and incident angle of the illuminating light to this model, it is possible to calculate the color signal when the illuminating light is reflected by the object. This makes it possible to predict how an object will look under specific lighting conditions and generate realistic images based on that prediction.

[0055] The first three-dimensional reflectance characteristic model is a first spectral reflectance S1(λ), ​​an illumination light spectral distribution E(λ), and a first diffuse reflectance function D a , and the first specular reflection function G a The first diffuse reflectance function D can be expressed by the following equation (3). a is a function that defines the diffuse reflection of the object before the pigment is applied. The first specular reflection function G a is a function that defines the specular reflection on the object to be coated before the pigment is applied. N represents the normal vector, V represents the line of sight vector, and L represents the lighting direction vector. The first term on the right-hand side represents the diffuse reflection component, and the second term on the right-hand side represents the specular reflection component.

[0056]

number

[0057] Here, the geometric model of reflection in this embodiment will be described with reference to Fig. 2. Fig. 2 is a schematic diagram showing the geometric model of reflection according to one embodiment of the present invention.

[0058] 2, virtual space 50 includes a virtual point 51, a virtual light source 52, and a virtual viewpoint 53. Virtual point 51 is a point on surface S of the object to be coated, and virtual light source 52 irradiates this point with illumination light. Virtual viewpoint 53 is a viewpoint from which virtual point 51 is observed.

[0059] The lighting direction vector L is a vector directed from the virtual point 51 to the virtual light source 52. The normal vector N is a vector in the normal direction at the virtual point 51 on the surface S. The viewing direction vector V is a vector directed from the virtual point 51 to the virtual viewpoint 53. The lighting direction vector L, the normal vector N, and the viewing direction vector V are parameters for defining the geometric conditions of the lighting light in the three-dimensional reflection property model.

[0060] Generally, when an object is exposed to illumination light, the color signal C0(λ) generated on the object surface is expressed as C0(λ) = E0(λ)S0(λ) using the object's spectral reflectance S0(λ) and the illumination light spectral distribution E0(λ).

[0061] On the other hand, in this embodiment, by using a three-dimensional reflection characteristic model that takes into account not only the spectral reflectance of the object and the spectral distribution of the illumination light, but also the diffuse reflection and specular reflection on the object surface, it is possible to more accurately reproduce the gloss and texture of the object surface.

[0062] The angle between the illumination direction vector L and the normal vector N is defined as the incident angle θ i The angle between the normal vector N and the line of sight vector V is the light receiving angle θ r When a virtual point 51 receives light from a virtual light source 52, diffuse reflected light is generated in all directions at the virtual point 51, and the incident angle θ i and acceptance angle θ r The reflected light is composed of specular reflected light that occurs when the and are close to each other.

[0063] This will be explained with reference to Fig. 3. Fig. 3 is a schematic diagram showing the distribution of reflected light in the geometric model of reflection shown in Fig. 2.

[0064] 3, the intensity distribution of the reflected light occurring at the imaginary point 51 is expressed as a reflected light distribution RD, which is an intensity distribution obtained by adding the diffuse reflected light intensity α and the specular reflected light intensity I. The intensity of the reflected light occurring at the imaginary point 51 varies with the angle of reception θ r Specifically, the distance from the virtual point 51 to the reflected light distribution RD is different depending on the light receiving angle θr represents the intensity of reflected light at

[0065] The diffuse reflection light intensity α is a reflection characteristic parameter for defining the intensity of the diffuse reflection light. The diffuse reflection light intensity α is determined by the angle of incidence θ i With respect to the acceptance angle θ r The specular reflection light intensity I is a reflection characteristic parameter that determines the relative intensity of specular reflection light to the diffuse reflection light intensity α. The specular reflection light intensity I is determined by the angle of incidence θ i With respect to the acceptance angle θ r The value of increases as the incident angle θ i and acceptance angle θ r The maximum value is reached when the values ​​are equal.

[0066] For example, the acceptance angle θ r is the angle of incidence θ i The acceptance angle θ is significantly different from r1 In this case, the intensity of the reflected light observed at the virtual viewpoint 53a is the value of the diffuse reflected light intensity α. r is the angle of incidence θ i The angle of acceptance θ is approximated as r2 In this case, the intensity of the reflected light observed at the virtual viewpoint 53b is the sum of the diffuse reflected light intensity α and the specular reflected light intensity I.

[0067] The first diffuse reflectance function D used in equation (3) a is expressed by the following equation (4) using the first diffuse reflected light intensity α1 and the illumination direction vector L and normal vector N as the geometric conditions of the illumination light. The first diffuse reflected light intensity α1 is a first reflection characteristic parameter for defining the intensity of the diffuse reflected light on the object to be coated before the pigment is applied. Here, the first diffuse reflected light intensity α1 is treated as α1=1. Note that the geometric conditions of the illumination light used in the following calculations are calculated from the spatial distribution of the illumination light source, the shape of the object to be coated, the viewpoint position from which the object is observed, etc.

[0068]

number

[0069] In the reflected light distribution RD shown in Figure 3, the range in which specularly reflected light occurs is expressed by the specular reflection parameter m. The specular reflection parameter m is a parameter that depends on the roughness and other surface properties of the object surface, and is a reflection characteristic parameter that determines the extent of gloss. The larger the specular reflection parameter m, the larger the range in which specularly reflected light occurs.

[0070] The first specular reflection function G used in Eq. (3) a is expressed by the following equations (5) to (7) using the first specular reflection light intensity I1, the first specular reflection parameter m1, the lighting direction vector L, the normal vector N, and the line of sight vector V. Note that B(N, V, L, m1) included in equation (6) is a Beckmann function.

[0071]

number

[0072] The first specular reflection light intensity I1 is a first reflection characteristic parameter for defining the intensity of specular reflection light on the object to be coated in a state before the pigment is applied. The first specular reflection parameter m1 is a first reflection characteristic parameter for defining the extent of gloss on the object to be coated in the same state. The first specular reflection light intensity I1 and the first specular reflection parameter m1 are set to arbitrary values ​​in advance.

[0073] The first specular reflection light intensity I1 and the first specular reflection parameter m1 can also be obtained from a plurality of images of the coating target before the pigment is applied, in which the angle of incidence of the illumination light is changed. For example, in the image acquisition step (step S2), when acquiring images of the coating target, the first specular reflection light intensity I1 and the first specular reflection parameter m1 can also be obtained by acquiring a plurality of images in which the angle of incidence of the illumination light is changed.

[0074] In this way, by calculating the first color signal C1(λ) using the first three-dimensional reflection characteristic model, the color of the object to be coated before the pigment is applied and the shading and gloss caused by the illumination light can be reproduced as the first color signal C1(λ).

[0075] In the first color signal calculation step (step S8), the calculation unit may calculate changes in texture (such as shading or gloss) due to the unevenness of the surface of the object to be coated, and may further use texture information including information about the texture of the surface of the object to be coated obtained by the calculation to calculate the first color signal C1(λ). This allows the texture corresponding to the unevenness of the object to be calculated, and texture information to be imparted to the color of the object to be coated.

[0076] In the characteristic part identification step (step S9), the calculation unit identifies characteristic parts of the object to be applied from the first image. This identification may, for example, involve identifying the position and area of ​​the object to be applied. As a specific example, if the first image is a face image, the calculation unit identifies facial components such as the eyes, nose, and mouth, thereby identifying the position and area of ​​the facial components (contours or areas surrounded by the contours). This identification can be achieved by using known image recognition technology. Examples of this image recognition technology include convolutional neural networks (CNNs).

[0077] In the second spectral reflectance calculation step (step S10), the calculation unit executes a process of calculating the second spectral reflectance S2(λ) of the pigment-coated region based on the first spectral reflectance S1(λ) and the second conversion function T(λ). In detail, the calculation unit reads out the second conversion function T(λ) corresponding to the color of the coating target from the database and uses it to calculate the second spectral reflectance S2(λ) of the pigment-coated region.

[0078] The second conversion function T(λ) is a conversion matrix for calculating the second spectral reflectance S2(λ), which is the spectral reflectance of the pigment, based on the first spectral reflectance S1(λ). Here, the pigment refers to a pigment that is applied to an object to be applied. If the object to be applied is, for example, skin, the pigment is a cosmetic product, such as foundation, blush, eye color, or lip color.

[0079] The second conversion function T(λ) is, for example, obtained by calculating the spectral reflectance S a and the spectral reflectance S b and the spectral reflectance S a and spectral reflectance S b The change in the amount of the test subject's skin color is statistically calculated for each color of the object to be applied.

[0080] The second spectral reflectance S2(λ) can be expressed by the following equation (8) using the first spectral reflectance S1(λ) and the second conversion function T(λ).

[0081]

number

[0082] By calculating the second spectral reflectance S2(λ) based on the second conversion function T(λ) and the first spectral reflectance S1(λ), ​​the second spectral reflectance S2(λ) is calculated, which is close to the spectral reflectance when the pigment is actually applied to the object to be applied and blended.

[0083] The second conversion function T(λ) may be determined, for example, in the first spectral reflectance calculation step (step S7) together with the first conversion function M. Alternatively, the second conversion function T(λ) may be determined by the user selecting a color close to the color of the object to be coated from a color sample such as a color code.

[0084] In the second color signal calculation step (step S11), the calculation unit calculates the second color signal C2(λ) based on the second spectral reflectance S2(λ), the second reflection characteristic parameter, the illumination light spectral distribution E(λ), and a second three-dimensional reflection characteristic model expressed by the geometric conditions of the illumination light.

[0085] Specifically, the calculation unit reads out the second reflection characteristic parameters, which are the reflection characteristic parameters of the pigment selected by the user, from the database. Then, the second three-dimensional reflection characteristic model is applied with the illumination light spectral distribution E(λ) and the spatial distribution of the illumination light source to calculate the second color signal C2(λ). The second color signal C2(λ) is the color signal obtained when the illumination light is irradiated onto the pigment and reflected.

[0086] The second three-dimensional reflectance model is the second spectral reflectance S2(λ), the illumination light spectral distribution E(λ), and the second diffuse reflectance function D b , and the second specular reflection function G b Using the above, the second diffuse reflectance function D can be expressed by the following equations (9) to (11). b is a function that defines the diffuse reflection in the pigment. The second specular reflection function G b is a function that defines the specular reflection in pigments. Also, B(N,V,L,m2) in equation (11) is the Beckmann function. In equation (9), the first term on the right-hand side represents the diffuse reflection component, and the second term on the right-hand side represents the specular reflection component.

[0087]

number

[0088] In the second three-dimensional reflection characteristic model expressed by equations (9) to (11), the second diffuse reflected light intensity α2 is a parameter set for each pigment and is a second reflection characteristic parameter that defines the intensity of the diffuse reflected light of the pigment. Here, to simplify the second diffuse reflected light intensity α2, α2 is treated as α2 = 1. Furthermore, the second specular reflected light intensity I2 is a parameter set for each pigment and is a second reflection characteristic parameter that defines the intensity of the specular reflected light of the pigment. Furthermore, the second specular reflection parameter m2 is a parameter set for each pigment and is a second reflection characteristic parameter that defines the extent of the pigment's gloss.

[0089] Furthermore, when multiple types of pigments are selected, in the second color signal calculation step (step S11), the calculation unit may calculate the second color signal C2(λ) for each pigment.

[0090] Furthermore, in the second color signal calculation step (step S11), the calculation unit may calculate changes in shading and gloss due to the unevenness of the surface of the object to be coated, and may further use the information on shading and gloss due to the surface of the object to be coated obtained by the calculation, i.e., texture information, to calculate the second color signal C2(λ). This allows shading and gloss (texture) corresponding to the unevenness of the object to be coated to be calculated, and then uses this to impart shading and gloss information to the pigment.

[0091] In this way, by calculating the second color signal C2(λ) using the second three-dimensional reflection characteristic model, the color of the pigment and the shading and gloss caused by the illumination light can be effectively reproduced as the second color signal C2(λ).

[0092] In the glitter color signal calculation step (step S12), the calculation unit inputs the optical property information to a third three-dimensional reflection property model, which is a mathematical model that describes the process of light reflection of the glitter powder, to calculate the glitter color signal C G In this case, the second color signal C2(λ) of the pigment excluding the glitter powder is calculated by the second three-dimensional reflection characteristic model, and the color signal C G (λ) is calculated by the third three-dimensional reflectance model.

[0093] Fig. 4 is a schematic cross-sectional view showing an example of the structure of a glittering powder. As shown in Fig. 4, glittering powder 60 is composed of a powder substrate 62 coated with one or more thin films 61, and the thickness of these thin films determines the color and luster of the glittering powder. Examples of substrate 62 include mica, alumina, silica, synthetic fluorophlogopite, glass powder, aluminum powder, and polyethylene terephthalate resin. Examples of thin films 61 include titanium dioxide, silicon dioxide, iron oxide, gold, silver, organic pigments, and resins.

[0094] When the glittering powder 60 is irradiated with light from the virtual light source 52, an interference light that depends on the thickness d of the thin film 61 is observed at the virtual viewpoint 53. Based on this interference light, a color signal C of the glittering powder 60 is calculated. G (λ) is calculated. The specific interference color that is selectively enhanced by the thickness d of the thin films 61 makes it possible to accurately reproduce the unique color produced by the glittering powder. Information such as the number and thickness d of the thin films 61 in the glittering powder 60 is stored in a database, and is used as reference data when the calculation unit calculates a color signal based on this information.

[0095] In the metallic pigment color signal calculation step (step S12), the calculation unit inputs the wavelength of the interference light that depends on the thickness d of the thin film 61 to the third three-dimensional reflection characteristic model to calculate the color signal C G This process is important for accurately simulating the color and luster characteristic of the glitter powder. The physical properties of the thin film 61, particularly its thickness d, determine the interference pattern of the irradiated light, which results in the unique color change and luster effect exhibited by the glitter powder. By applying the wavelength information of this interference light to the third three-dimensional reflection property model, it is possible to calculate and simulate how the glitter powder actually reflects light and what visual effects it produces.

[0096] In the metallic tin emitting material information acquisition step (step S12), the user can input a specific wavelength in a variety of ways. This includes inputting a wavelength derived from visual judgment or using values ​​obtained by a spectrophotometer. In other words, the optical property information acquired in the metallic tin emitting material information acquisition step (step S12) can include wavelength.

[0097] Photoluminescent powders have the ability to react to different wavelengths of light based on their optical properties and display various colors. Therefore, by inputting specific wavelengths, users can more accurately reproduce the color of the powder under actual application conditions. Through this process, the simulated color more closely matches the actual application state, reducing the gap between expected and actual results. This feature provides more accurate visual feedback during the product development and design process, contributing to improving the quality of the final product.

[0098] The third three-dimensional reflection characteristic model is the spectral reflectance S G Using the reflectance characteristic parameters of the glittering powder, the spectral distribution of the illumination light E(λ), and the geometric conditions of the illumination light, the following equations (12) and (13) can be expressed. G ) is the Beckmann function. The glittering powder has a spectral reflectance S that depends on the geometric conditions of the illumination light. G In this model, the color signal C is calculated based only on the specular reflection component of the glitter powder. G (λ) is calculated, ignoring the diffuse reflection component.

[0099]

number

[0100] The distribution function used is not limited to the Beckmann function, and other distribution functions can also be used. For example, it may be desirable to select a different distribution function to capture the characteristics of a material to which the Beckmann function cannot be applied.

[0101] In the third three-dimensional reflection characteristic model expressed by equations (12) and (13), the specular reflection function G G is a function that defines the specular reflection of the glitter powder. Specular reflection light intensity I G is a reflection characteristic parameter of the glittering powder, which defines the intensity of specular reflection light specific to each glittering powder. The specular reflection parameter m G are reflection characteristic parameters of the glitter powder, and are parameters that define the extent of gloss for each glitter powder. These reflection characteristic parameters can be obtained, for example, from actual measurements of the actual glitter powder using a measuring instrument such as a gloss meter.

[0102] Furthermore, the glitter color signal calculation step (step S12) is performed in the area within the application area where the glitter powder is applied. Specifically, the area within the application area where the glitter powder is applied can be estimated based on shape information that defines the particle size of the glitter powder, and dispersion information that defines the dispersion state of the glitter powder within the area where the pigment is actually applied. The area within the application area where the glitter powder is applied is estimated, and the color signal C of the glitter powder is calculated in that area. G By calculating (λ), it is possible to reproduce the particle shape and dispersion state of the glittering powder contained in the actual pigment. Information on the shape and dispersion of the glittering powder is stored in a database.

[0103] In addition, the color signal C of the glitter powder G When calculating (λ), a correction may be made to increase the spectral intensity at a specific wavelength. The third three-dimensional reflection characteristic model may also include parameters such as an orientation parameter that defines the orientation of the glittering powder and the thickness d of the thin film 61.

[0104] In the third color signal calculation step (step S13), the calculation unit calculates a third color signal C3(λ), which is a color signal of a second image including the object to be coated with the pigment, based on the first color signal C1(λ) and the second color signal C2(λ). In areas of the coated area where multiple types of pigment are superimposed, the third color signal C3(λ) is calculated by adding up the second color signals C2(λ) of the respective pigments.

[0105] As a specific example, the calculation unit calculates a first color signal C1(λ) and a color signal C of the first pigment in a region where the first pigment and the second pigment are superimposed in the application region. 2f (λ) and the color signal C of the second pigment 2c The third color signal C3(λ) is calculated based on the following equation (14) using (λ) and weighting coefficients w1 to w3 for each color signal. The weighting coefficients w1 to w3 are coefficients that appropriately weight each color signal, and are determined based on the thickness of the pigment in the simulation, the order of application, etc.

[0106]

number

[0107] In this way, by calculating the third color signal C3(λ) based on the first color signal C1(λ) and the second color signal C2(λ), it is possible to calculate the third color signal C3(λ) in a state in which the color of the object to be coated is reflected in the pigment as the color signal of the object to be coated when the pigment is being applied.

[0108] Even when simulating a state in which multiple types of pigments are superimposed, by calculating the third color signal C3(λ) based on the first color signal C1(λ) and the second color signal C2(λ) of each pigment, it is possible to calculate a color signal in which the color of the object to be coated and the color of the underlying pigment are reflected in the outermost pigment.

[0109] Furthermore, in the third color signal calculation step (step S13), the calculation unit calculates the first color signal C1(λ), ​​the second color signal C2(λ), and the glitter powder color signal C G (λ), the third color signal C3(λ) is calculated.

[0110] As a specific example, the calculation unit calculates the third color signal C3(λ) in the area where the first pigment, the second pigment, and the glittering powder (pearl pigment) overlap within the application area based on the following equation (15).

[0111]

number

[0112] Here, the second color signal C 2e (λ) indicates the color signal of the applied pigment excluding the glitter powder. Color signal C Gp (λ) is the color signal of the first glittering powder. Color signal C Gl (λ) is the color signal of the second glittering powder. Weighting coefficients w1 to w3 are coefficients that appropriately weight each color signal, and are determined appropriately depending on the thickness of the pigment in the simulation, the order of application, etc.

[0113] In this way, the first color signal C1(λ), ​​the second color signal C2(λ), and the glitter powder color signal C G Based on (λ), a color signal for the state in which the pigment is applied can be calculated, thereby obtaining a third color signal C3(λ) that reflects the color and luster of the glitter powder contained in the pigment.

[0114] In the color value calculation step (step S14), the calculation unit calculates the color value (second color value) in the state where the glitter powder is applied, based on the third color signal C3(λ).

[0115] Specifically, the calculation unit calculates the second color value based on the following equation (16) using the third color signal C3(λ) and color functions such as RGB: Note that R2, G2, and B2 on the left side of equation (16) are the values ​​of each element in the second color value.

[0116]

number

[0117] In the second image generation process (step S15), a calculation unit generates a second image based on the second color value and the shape information of the glitter powder. In this process, an image representing the glitter powder applied to the target object is created by combining the color value defined in the previous step with shape information such as the particle shape, size, and distribution of the glitter powder. This image generation process takes into account the unique optical properties and particle shape of the glitter powder, resulting in a simulation result that is very close to the appearance of the glitter powder when actually applied. This process improves the accuracy of the final visual effect, making it possible to provide users with a more realistic visual experience.

[0118] In the superimposed image generation process (step S16), the calculation unit generates an image of the applied photoluminescent powder based on the second color value and the shape information of the photoluminescent powder. Specifically, the calculation unit superimposes the first and second images to create a combined image. This process visualizes the relationship between the area where the photoluminescent powder is applied and its surroundings in real time, allowing the user to confirm the final appearance. Using this superimposition technology, the subtle color variations and shape characteristics of the photoluminescent powder can be accurately reproduced on the surface of the applied object. As a result, a more realistic image with finer details is generated, allowing the user to grasp in detail the final appearance of the applied powder.

[0119] In this way, by calculating the first color signal C1(λ) and the second color signal C2(λ) based on the illumination light spectral distribution E(λ) and the spatial distribution of the illumination light source, the superimposed image that results from the simulation reflects the lighting environment where the first image was captured. This method makes it possible to perform a simulation that takes into account the influence of the lighting environment on the first image.

[0120] Furthermore, various lighting environments can be easily reproduced by using the illumination light spectral distribution E(λ) calculated from the measured values ​​of the spectral distribution in the actual lighting environment. This makes it easy to perform a simulation under a specific lighting environment, for example. Similarly, it also becomes easy to compare the results of simulations under different lighting environments.

[0121] 1 may be changed as appropriate within the scope of not interfering with the processing. For example, the lighting environment estimation step (step S5) and the first spectral reflectance calculation step (step S7) may be performed in this order, or the order may be reversed. In other words, the first spectral reflectance calculation step (step S7) may be performed after the lighting environment estimation step (step S5), or the lighting environment estimation step (step S5) may be performed after the first spectral reflectance calculation step (step S7). Furthermore, for example, the surface identification step (step S4) may be performed after the first color signal calculation step (step S8) when the unevenness of the surface of the object to be coated is not taken into consideration.

[0122] Furthermore, by selecting a different pigment while the first and second images are superimposed, the first spectral reflectance calculation step (step S7) through the color value calculation step (step S14) can be executed again to generate and display a superimposed image to which a different pigment has been applied. In this case, for example, the first spectral reflectance calculation step (step S7) and the first color signal calculation step (step S8) may be omitted by storing the first color signal C1(λ) calculated once in memory. In this way, by selecting various pigments and repeatedly performing simulations, the user can find a pigment that suits their own preferences.

[0123] [(2) Interference pearl pigments / colored pearl pigments] Luminous powders include interference pearlescent pigments and colored pearlescent pigments. Interference pearlescent pigments are pigments that utilize the interference effect of specific light to produce various colors and brilliance. This effect, called thin-film interference, occurs when light strikes a thin film, reflecting and interfering within the film. Interference pearlescent pigments are primarily made by coating the surface of minerals such as mica with metal oxides such as titanium dioxide (TiO2), but are not limited to these pigments as long as they exhibit an interference effect. Examples include glitter agents in lamination resins. Resins used in lamination include polyethylene terephthalate resin, epoxy resin, polyolefin resin, and polyester resin.

[0124] A characteristic of these pigments is that their color changes depending on the viewing angle. For example, when light from a light source hits the pigment and reaches the viewer's eye, it reflects and interferes within the pigment's thin film, causing different colors to be observed depending on the viewing angle. This can result in the appearance of different colors such as red, green, blue, yellow, and purple. Furthermore, interference pearlescent pigments do not have a color themselves; instead, color is generated by the interference of light, resulting in extremely vivid and deep colors.

[0125] Interference pearlescent pigments are used in a variety of industries, including cosmetics, paints, inks, plastics, and automotive coatings. In particular, they are used in cosmetics to create a natural shine and three-dimensional effect, and in automotive coatings to create a metallic shine that changes color depending on the angle. Using these pigments can add unique beauty and appeal to skin and products.

[0126] Colored pearlescent pigments are glittering powders made by coating fine metal or mineral powders with pigments, with color directly added. The characteristic of these pigments is that they retain a consistent color regardless of changes in lighting or viewpoint, due to the inherent color of the glittering powder itself. Unlike interference pearlescent pigments, colored pearlescent pigments rarely change color significantly depending on the viewing angle or the way the light hits them, but the brightness and depth of color do change depending on the light.

[0127] Colored pearlescent pigments are primarily used in industries such as cosmetics, paints, and plastics, where metallic or pearlescent effects are desired. Using colored pearlescent pigments can give products rich colors and unique textures, enhancing their appeal. In addition, by combining multiple colorants, it is possible to produce pigments with a variety of colors, broadening the scope of design.

[0128] The production of colored pearlescent pigments involves coating interference pearlescent pigments with organic or inorganic pigments, which improves brilliance and color reproducibility under natural light, making the pigment suitable for a variety of applications.

[0129] Examples of applications of colored pearlescent pigments include eye shadow, blush, and lip products in cosmetics. By using colored pearlescent pigments in these products, it is possible to create a natural luster and three-dimensional effect, giving the user's skin a rich look. They are also used in automobile paints and plastic products, contributing to the expression of a luxurious feel and unique texture.

[0130] The present invention enables highly accurate and realistic reproduction of interference pearl pigments and colored pearl pigments. The glitter powder reproduced by the present invention contains at least one of an interference pearl pigment and a colored pearl pigment. In other words, the present invention can reproduce one or both of an interference pearl pigment and a colored pearl pigment.

[0131] It is preferable that the three-dimensional reflection characteristic model used to reproduce the interference pearl pigment and the three-dimensional reflection characteristic model used to reproduce the colored pearl pigment are different from each other. The former is defined as an interference-based three-dimensional reflection characteristic model, and the latter is defined as a colored-based three-dimensional reflection characteristic model. In the present invention, either one of the interference-based three-dimensional reflection characteristic model and the colored-based three-dimensional reflection characteristic model may be used, or both may be used.

[0132] In the third color signal calculation step (step S13), the calculation unit provides the optical property information of the interference pearl pigment to the interference system three-dimensional reflection property model to calculate the color signal of the interference pearl pigment.

[0133] Furthermore, in a third color signal calculation step (step S13), the calculation unit provides the optical property information of the colored pearl pigment to the colored system three-dimensional reflection property model to calculate the color signal of the colored pearl pigment.

[0134] The interference-based three-dimensional reflection characteristic model and the color-based three-dimensional reflection characteristic model are constructed based on different principles and characteristics.

[0135] A physical model utilizing the principle of thin film interference is used to reproduce interference pearlescent pigments. This physical model simulates the interference effect that occurs when light enters a thin film of pearlescent pigment and is reflected multiple times internally. This effect reproduces the phenomenon in which different colors appear depending on the observation angle. In order to reproduce the characteristics of interference pearlescent pigments with high precision, the present invention defines the distribution of reflected light based on, for example, the Torrance-Sparrow model, taking into account the effect of thin film interference. This allows for accurate reproduction of the characteristics of interference pearlescent pigments, whose color and brightness change depending on the angle.

[0136] On the other hand, a separate model formula has been developed to directly color highlights in colored pearlescent pigments. Unlike interference pearlescent pigments, colored pearlescent pigments are inherently colored. Because there are still unknown aspects of their underlying principles, in order to match actual appearance and colorimetric values, the color system three-dimensional reflection characteristic model includes at least one of two models: a physical model that describes the physical light reflection characteristics of the glittering powder, and an empirical model that describes the light reflection characteristics of the glittering powder based on hypotheses. In other words, a flexible model is used that utilizes both a physical model with an elucidated principle and an empirical model based on hypotheses. This method allows for the construction of mathematical models that capture the unique reflection blur and interference characteristics of colored pearlescent pigments, and then combines these to represent a single reflection characteristic. By primarily using empirical models, the phenomenon of light reflection and interference enhancing the specific color tone of the coating is precisely reproduced, recreating the unique color effects of colored pearlescent pigments. This process does not rely solely on physical models; it is a practical approach to reproducing actual colors, and this method is suited to reproducing the color of the real thing. In the future, technological advances will make it possible to replace empirical models with physical models.

[0137] The difference between these approaches comes from the differences in the physical properties and reflection mechanisms of interference pearlescent pigments and colored pearlescent pigments. Interference pearlescent pigments reproduce color changes due to the interference of light, while colored pearlescent pigments reproduce colors based on the color of the pigment itself and its reflection properties. In this way, the present invention uses an appropriate model to reproduce the unique brilliance effects of both types of pearlescent pigments with high accuracy and realism.

[0138] (3) Variability Photochromic powders are manufactured using a variety of methods, including milling, synthesis, precipitation, and vapor deposition. These processes result in variations in the final particle size due to factors such as uneven particle size in the raw materials, subtle differences in reaction conditions, or differences in the settings of the milling machine. For example, during the milling process, particles are broken down into irregular shapes, resulting in particles of various sizes. To accurately and realistically reproduce photochromic powders, it is desirable to represent this variation. For example, when photochromic powders are used in cosmetics, the products are viewed up close, making it particularly important to accurately reproduce variations in the small particle size range (approximately 20 to 200 μm).

[0139] Therefore, in the present invention, by inputting numerical parameters such as the average particle size and standard deviation of the glittering powder, it is possible to automatically calculate the shape and particle distribution of the glittering powder. In other words, the shape information acquired in the glittering agent information acquisition step (step S12) includes the average particle size and standard deviation of the glittering powder. The standard deviation is an index that quantitatively indicates the variation in particle size. This information can be acquired using, for example, a microscope or a particle size analyzer. This information is very important because the particle size and its distribution have a significant impact on the appearance of the powder when actually applied.

[0140] Then, in the second image generation step (step S15), the image is generated using a probability distribution based on the average particle size and standard deviation. That is, based on the acquired average particle size and standard deviation, the size variation of the glitter powder particles is expressed based on a probability distribution. This makes it possible to reflect the fact that the particle sizes are not constant but have random variations.

[0141] There are many different types of probability distributions, including normal distribution, log-normal distribution, binomial distribution, and Poisson distribution. These probability distributions can be applied to the present invention. In particular, the log-normal distribution is suitable for the size distribution of particles generated when a thin glass plate or the like is crushed. This is because the size of particles generated when a glass-like material is crushed tends to follow a log-normal distribution. On the other hand, in the case of a material that has a specific shape from the beginning, for example, particles processed to a certain shape or size during the manufacturing process, this distribution does not necessarily follow. Therefore, it is important to select an optimal probability distribution depending on the characteristics of the material and particles used.

[0142] The lognormal distribution is a probability distribution defined over positive real numbers where the logarithm of a variable follows a normal distribution. This distribution can be applied to size distributions found in nature and social phenomena, such as the size of individuals, income distributions, and particle size distributions.

[0143] The log-normal distribution can represent a wide range of sizes, from very small to large. Glittering powder particles exist in a wide range of sizes, from very fine to relatively large, and this wide range can be represented by a single distribution.

[0144] The lognormal distribution has the characteristic of having a long right tail and a wide tail. Many material manufacturing processes typically produce a small number of large particles and a large number of small particles, and the lognormal distribution naturally represents this phenomenon.

[0145] In particular, the particle size distribution of glittering powders is thought to be particularly suitable for a log-normal distribution. This is because glittering powders are produced through the processes of pulverization and synthesis, and then the particle size is selected through a process called classification, so the particle size distribution tends to be close to a log-normal distribution. However, with technological advances in the future, it may be possible to use probability functions that can more accurately represent particle size distributions.

[0146] When the log-normal distribution is applied purely, depending on certain parameters, glitter powders with very large particle sizes may occasionally appear. This can cause a significant sense of incongruity in cosmetics where the application area is small and the finish is often checked at close range from the skin. Therefore, the present invention solves this problem by adding a process that does not display glitter powders with particle sizes exceeding a certain size. This process makes it possible to accurately reproduce particle size variations within a natural range while maintaining a natural texture and appearance of the finished cosmetic product.

[0147] [(4) Generated image] The generated image is expressed using a normal map. Normal maps are a three-dimensional representation technique used to simulate minute surface irregularities. Specifically, the direction of the surface normal vector at each pixel is encoded as color information, and this information is used to calculate light reflection and shading. Using normal maps, it is possible to visually express three-dimensional irregularities and textures without actually creating three-dimensional shapes. This technique enables realistic representation while keeping calculation costs low. Normal maps are particularly effective in reproducing fine details, and play an important role in reproducing the texture and luster of glitter powder.

[0148] In addition to normal mapping, various other techniques can be used in generating this image. For example, displacement mapping is used to more realistically represent the unevenness of an object's surface. While normal mapping primarily modifies the appearance of the surface, displacement mapping involves actual geometric deformation. This changes the height of the surface, allowing for a more three-dimensional appearance. Environment mapping simulates the reflection of the environment, creating realistic reflection effects within the environment in which the object is placed. It is particularly effective for expressing glossy surfaces and metallic textures. Subsurface scattering (SSS) simulates the effect of light scattering inside translucent objects. It is used to realistically render partially transparent materials.

[0149] The shape of the glittering powder in the generated image in plan view is preferably polygonal, such as a triangle, a square, a pentagon, or a hexagon.

[0150] Photochromic powders are typically produced by milling thin glass plates or other materials. The particles produced by this milling process are typically polygonal rather than perfectly circular. This polygonal shape significantly affects the light reflection and refraction characteristics, resulting in the unique luster and color variations characteristic of photochromic powders. The polygonal shape of powder particles is particularly important for reproducing interference effects, in which the color and luster appear to change depending on the angle, and metallic textures, and these visual effects are reproduced richly. Therefore, in image generation, reflecting the actual photochromic powder manufacturing process and representing the particles as polygons can achieve more realistic textures and visual effects.

[0151] The particle size of the glitter powder in the generated image can be 3 mm or less. Glitter powder of this particle size is used in cosmetics. Powders with a fine particle size of 3 mm or less are produced by grinding, and this fine particle size provides a smooth texture and uniform luster to the skin. Such fine particles have a high ability to capture and reflect light, and the color and luster change depending on the viewing angle, giving cosmetics a rich look.

[0152] Fine-grained luminous powders are used in a wide range of cosmetics, including highlighters, eye shadows, and foundations, to add a natural glow to the skin and create a three-dimensional, deep look.

[0153] [(5) Photochromic Agent Information Acquisition Process] When acquiring optical property information, including spectral reflectance, the angles of incidence and reflection of light play an important role, and these depend on the geometric relationship between the line-of-sight vector, normal vector, and lighting direction vector. In particular, the angle at which light from the light source strikes an object (light source angle) has a significant impact on the accurate reproduction of color. To accurately capture the color characteristics of lustrous powders such as colored pearlescent pigments, it is best to measure spectral reflectance using a spectrophotometer. During this measurement process, the angle at which color is measured affects the results, so selecting the correct angle is important.

[0154] This will be explained again with reference to Figure 4. Figure 4 shows the angle at which color is measured. With the illumination direction vector L as the reference, the angle between the illumination direction vector L and the normal vector N is 45°.

[0155] In order to accurately reproduce the primary colors of the glittering powder, it is necessary to measure the color at a specific angle. In the glittering agent information acquisition step (step S12), the spectral reflectance acquired by the spectrophotometer is preferably the spectral reflectance of light reflected in the direction between the illumination direction vector L and the surface S of the object to which the glittering powder is applied, or the spectral reflectance of light reflected in the direction between the specular reflection direction vector of the illumination direction vector L and the surface S.

[0156] Although the reflection angle of light acquired by the spectrophotometer is not particularly limited, it is more desirable that the reflection angle of light acquired by the spectrophotometer be 25 degrees or less when the horizontal direction relative to surface S is defined as 0 degrees. In other words, when light emitted from virtual light source 52 is specularly reflected by surface S, the direction of the reflected light observed at virtual viewpoint 53b is defined as 0 degrees. In this case, it is desirable to acquire the spectral reflectance of light reflected between the 110-degree direction and surface S, or between the -15-degree direction and surface S. This makes it possible to capture actual colors excluding the texture elements of the cosmetic film, thereby improving the reproducibility of the observed colors.

[0157] By using an appropriate three-dimensional reflection characteristic model, it is possible to obtain the reflection characteristics of light at angles that have not been measured by interpolation or estimation.

[0158] (6) System Configuration Fig. 5 is a schematic diagram showing an example of the configuration of an image generation system to which an image generation method according to one embodiment of the present invention is applied. As shown in Fig. 5, the image generation system 1 includes a user terminal 10 and a server 20. The user terminal 10 is an information processing device owned by a user 2, and an example thereof is a smartphone. The user terminal 10 may also be an information processing device such as a tablet, laptop computer, or desktop computer.

[0159] If the application object is, for example, a face, in the image acquisition step (step S2), the control unit 11 of the user terminal 10 instructs the imaging unit 14 to acquire a facial image of the user 2 without makeup (a first image including the skin and lips of the face without makeup). Thereafter, the control unit 11 executes a process of transmitting the first image composed of the first color value to the server 20.

[0160] For example, a face image acquired by an external device such as a digital camera can be used as the first image. The image for estimating the lighting environment does not need to be an image acquired by the imaging unit 14 of the user terminal 10, and an image acquired by an external device other than the user terminal can also be used.

[0161] The server 20 is a server computer that executes various processes in the image generation system 1 and is an example of an image generation device for generating images. Specifically, the server 20 generates a second image that is an image including the object to be applied with the pigment. The server 20 also superimposes the second image on a first image that is an image including the object to be applied before the pigment is applied. In this embodiment, the first image only needs to be an image including the object to be applied, and may also include an object other than the object to be applied. For example, if the object to be applied is lips, the first image may be an image including lips and parts other than the lips, specifically, a face image.

[0162] The user terminal 10 and the server 20 are connected via a network 3, which is a communication line. The network 3 is, for example, a mobile communication system such as 4G or 5G for mobile phones, or a wireless LAN communication system such as Wi-Fi (registered trademark).

[0163] An image display program serving as an application program is installed in the user terminal 10. The image display program is a program that causes the user terminal 10 to execute processing to display various images, such as a first image and a superimposed image in which the first image and a second image are superimposed. The image display program is downloaded from an application distribution device via a network and installed in the user terminal 10.

[0164] The user terminal 10 includes a touch panel 13. On the touch panel 13, various images such as a first image 30a and a superimposed image 30b as a simulation result are displayed.

[0165] [(7) User terminal] Fig. 6 is a block diagram showing an example of the configuration of a user terminal to which an image generation method according to an embodiment of the present invention is applied. As shown in Fig. 6, the user terminal 10 includes a control unit 11, a memory 12, a touch panel 13, an imaging unit 14, and a communication unit 15. The control unit 11 is, for example, a CPU, and controls the overall operation of the user terminal 10.

[0166] The memory 12 includes a main memory and a data storage unit. The main memory included in the memory 12 is, for example, a RAM, and temporarily stores data, programs, and the like of the user terminal 10. The data storage unit included in the memory 12 is, for example, a non-volatile memory, for example, a flash memory. The data storage unit included in the memory 12 stores programs for executing various processes in the user terminal 10, such as an image display program.

[0167] The touch panel 13 is a display that allows touch operation input by combining a display device such as a liquid crystal panel or an organic EL panel with a position input device. That is, the touch panel 13 is a display unit on the user terminal 10 that displays the image 30, and also an operation unit that performs operation input to the server 20.

[0168] The imaging unit 14 is, for example, a camera equipped with an imaging element such as a CCD or CMOS. The imaging unit 14 detects light such as visible light and outputs image data including color information. Hereinafter, the color value of the image acquired by the imaging unit 14 is referred to as the first color value.

[0169] The imaging unit 14 acquires a facial image of the user 2 (a first image including the face without makeup) configured with a first color value before the pigment is applied. The imaging unit 14 may also acquire an image for lighting environment estimation used in a lighting environment estimation unit described below. Note that although the present invention relates to the application of pigments, its range of application is not limited to cosmetics. For example, it may also be applied to the painting of automobiles.

[0170] The communication unit 15 communicates with the server 20 via the network 3. Specifically, the communication unit 15 transmits signals from the user terminal 10 to the server 20, and receives data such as images generated by the server 20. The communication unit 15 may also be capable of communicating with an information processing device (not shown) other than the server 20 via the network 3. In this case, the communication unit 15 may receive, for example, the first image or the above-mentioned lighting environment estimation image from an information processing device other than the server 20.

[0171] [(8) Server] Fig. 7 is a block diagram showing an example of the configuration of a server to which an image generation method according to one embodiment of the present invention is applied. As shown in Fig. 7, the server 20 includes a calculation unit 21, a memory 24, and a communication unit 25. The calculation unit 21 includes, as an example, a CPU 22 and a GPU 23. The CPU 22 controls the overall operation of the server 20 except for the image generation process. The GPU 23 executes the image generation process in the server 20.

[0172] The memory 24 includes a main memory and a data storage unit. The main memory included in the memory 24 is, for example, a RAM, and temporarily stores data, programs, etc. of the server 20. The data storage unit included in the memory 24 is, for example, a non-volatile memory, and is, for example, a flash memory such as an HDD or SSD.

[0173] The data storage unit included in the memory 24 stores programs for executing various processes in the server 20, such as an image generation program. The image generation program is a program that causes the server 20 to execute a process of generating a second image and generating a superimposed image in which the first image and the second image are superimposed. The processing in the image generation program is controlled by the GPU 23 included in the calculation unit 21.

[0174] However, this configuration is not limited to this, and for example, it is also possible to provide a GPU in the control unit 11 of the user terminal 10, install an image generation program in the user terminal 10, and perform processing using the GPU provided in the user terminal 10.

[0175] The memory 24 includes a spectral conversion database 24a as a data storage unit, a lighting environment database 24b, and a pigment database 24c. The memory 24 may further include a conversion matrix storage unit 24d that stores a system conversion matrix for determining the spectral distribution of illumination light from RGB values.

[0176] The spectral conversion database 24a stores a first conversion function M. The first conversion function M is a function for converting color values ​​into spectral reflectance. For example, the first conversion function M converts a first color value constituting the object to be coated into the spectral reflectance of the object to be coated in a state before the pigment is applied, i.e., the first spectral reflectance S1(λ). In this embodiment, the spectral conversion database 24a stores the first conversion function M for each part of the object to be coated, such as bare skin and lips, as examples of the object to be coated.

[0177] As a specific example, the spectral conversion database 24a classifies skin into three types—a first type, a second type, and a third type—according to the amount of melanin contained in skin, which is an example of an application target, and stores a first conversion function M corresponding to each skin color. The first type of skin is skin whose melanin amount is within a first range that is lower than the second and third types, and whose relative reflectance at each wavelength when light is applied to the skin is high. The second type of skin is skin whose melanin amount is within a second range that is higher than the first range, and whose relative reflectance at each wavelength when light is applied to the skin is medium. The third type of skin is skin whose melanin amount is within a third range that is higher than the second range, and whose relative reflectance at each wavelength when light is applied to the skin is low. Note that the spectral conversion database 24a may, for example, classify the RGB values ​​of skin into predetermined ranges and store a first conversion function M for each classification of the RGB values ​​of skin. The spectral conversion database 24a uses skin as an example, classifying it into, for example, three types based on the amount of melanin pigment, and stores a first conversion function M appropriate for each skin color. Specifically, skin of type 1 has the least amount of melanin pigment and has a high relative reflectance when illuminated with light. Skin of type 2 has more melanin pigment than type 1 and has a medium reflectance. Skin of type 3 has even more melanin pigment and has a low reflectance. This database can also classify skin color values ​​into specific ranges and store a first conversion function M corresponding to each range.

[0178] The lighting environment database 24b stores, for each lighting environment, the type of illumination light irradiated on the object to be coated and the arrangement of the light sources that generate that illumination light. The illumination light referred to here includes direct light sources that directly illuminate the object to be coated and reflected light sources that reflect off surrounding objects and illuminate the object to be coated, but these two types are not clearly distinguished in this embodiment. Therefore, the lighting environment database 24b comprehensively records the arrangement of light sources, including both direct and reflected light sources, from all directions around the object to be coated, and the illumination light spectral distributions E(λ) from these light sources. The light source arrangement information is used to define geometric conditions, such as the angle of incidence, of how the illumination light strikes the object to be coated.

[0179] The illumination light spectral distribution E(λ) and the spatial distribution of the illumination light source can be obtained, for example, by acquiring spectral images from all directions in an actual lighting environment and sampling the spectral distribution every 5 nm in the visible wavelength range (e.g., 400 to 700 nm). In this case, the illumination environment database 24b stores the illumination light spectral distribution E(λ) for each pixel of the spectral image. The position of each pixel serves as information specifying the spatial distribution of the illumination light source distributed in all directions around the object to be coated. The assigned illumination light spectral distribution E(λ) is the spectral distribution of the illumination light irradiated onto the object to be coated. In other words, by regarding each pixel in the spectral image as an illumination light source and assigning it an illumination light spectral distribution E(λ), the spatial distribution of the illumination light source in all directions and the spectral distribution of the illumination light irradiated from each illumination light source can be obtained. This method makes it possible to process the illumination light source of the entire scene at once without distinguishing between direct and reflected light sources in the actual lighting environment, making it possible to reproduce the lighting environment even when there are multiple illumination light sources.

[0180] The pigment database 24c stores a second conversion function T(λ) for each pigment. The second conversion function T(λ) is a conversion matrix for calculating a second spectral reflectance S2(λ), which is the spectral reflectance of the pigment, based on the first spectral reflectance S1(λ). When the pigment is, for example, a cosmetic product, the pigment database 24c stores the second conversion function T(λ) for each type of cosmetic product, such as foundation or blush, and for each variation of the color and purpose of each cosmetic product.

[0181] Specifically, the pigment database 24c stores the second conversion function T(λ) corresponding to the color of the object to be applied for each product. For example, in the case of cosmetics to be applied to the skin, the second conversion function T(λ) corresponding to each of the first, second, and third skin types is stored for each product.

[0182] The second conversion function T(λ) is, for example, the spectral reflectance S a and the spectral reflectance S after applying the pigment bis measured, and the change in spectral reflectance is statistically calculated for each color of the object to be coated.

[0183] The pigment database 24c also stores reflection characteristic parameters for each pigment. The reflection characteristic parameters depend on the surface properties of the object, such as surface roughness, and define reflection characteristics such as the intensity of reflected light and the extent of gloss relative to the angle of incidence of illumination light on the object surface. The reflection characteristic parameters for a pigment are calculated from actual measurements of the pigment surface using a measuring instrument such as a gloss meter.

[0184] Additionally, the pigment database 24c can store the spectral reflectance and reflection characteristic parameters of glitter powder. The spectral reflectance of glitter powder is not easily affected by the color of the object to which it is applied, even when the pigment is applied to the skin, so it is calculated as a reflection component independent of the spectral reflectance of the skin itself. Therefore, calculation processing using the second conversion function T(λ) is not necessary. Regarding the spectral reflectance of glitter powder, actual measured values ​​obtained by a measuring device are stored in the database.

[0185] The conversion matrix storage unit 24d stores a system conversion matrix for obtaining the illumination light spectral distribution E(λ) from the color value.

[0186] The communication unit 25 communicates with the user terminal 10 via the network 3. Specifically, the communication unit 25 receives signals from the user terminal 10 and transmits data such as images generated by the server 20 to the user terminal 10.

[0187] [(9)GPU] Fig. 8 is a block diagram showing an example of the functional configuration of the GPU shown in Fig. 7. As shown in Fig. 7, the GPU 23 included in the calculation unit 21 has and functions as an image acquisition unit 23a, a shape estimation unit 23b, a surface identification unit 23c, a lighting environment estimation unit 23d, a first spectral reflectance calculation unit 23e, a first color signal calculation unit 23f, a feature portion identification unit 24g, a second spectral reflectance calculation unit 23h, a second color signal calculation unit 23i, a metallic color signal calculation unit 23j, a third color signal calculation unit 23k, a color value calculation unit 23l, a second image generation unit 23m, and a superimposed image generation unit 23n.

[0188] The image acquisition unit 23a executes the process of the image acquisition step (step S2).

[0189] The shape estimation unit 23b executes the processing of the shape estimation step (step S3).

[0190] The surface identification unit 23c executes the processing of the surface identification step (step S4).

[0191] The lighting environment estimation unit 23d executes the processing of the lighting environment estimation step (step S5).

[0192] The first spectral reflectance calculation unit 23e executes the process of calculating the first spectral reflectance (step S7).

[0193] The first color signal calculation unit 23f executes the process of the first color signal calculation step (step S8).

[0194] The characteristic part identification unit 24g executes the processing of the characteristic part identification step (step S9).

[0195] The second spectral reflectance calculation unit 23h executes the process of calculating the second spectral reflectance (step S10).

[0196] The second color signal calculation unit 23i executes the process of the second color signal calculation step (step S11).

[0197] The metallic shining agent color signal calculation unit 23j executes the processing of the metallic shining agent color signal calculation step (step S12).

[0198] The third color signal calculation unit 23k executes the process of the third color signal calculation step (step S13).

[0199] The color value calculation unit 23l executes the processing of the color value calculation step (step S14).

[0200] The second image generating unit 23m executes the processing of the second image generating step (step S15).

[0201] The superimposed image generating unit 23n executes the process of the superimposed image generating step (step S16).

[0202] The GPU 23 installed in the server 20 executes these processes, enabling high-speed processing and real-time image generation. For example, simulations using augmented reality (AR) and mixed reality (MR) become possible, allowing users to instantly check various application effects.

[0203] This technology allows users to visually experience various finishes and designs before actually touching the product, such as in cosmetics trials or car customization simulations. The high-speed processing power of the GPU23 makes it possible to reproduce the reflective properties and texture of complex glitter powders in real time, providing more realistic simulation results. This allows product developers and designers to quickly evaluate user reactions to product appearance and effects, which can be used to improve products and develop marketing strategies.

[0204] [(10) Effects] According to this embodiment, the following effects can be obtained: Furthermore, this embodiment can also achieve other effects described in this specification.

[0205] (1) The unique luster of photoluminescent powder and color changes in response to changes in viewpoint and light source can be accurately reproduced based on a physical model, making it possible to accurately predict and display the behavior of powder under actual usage conditions.

[0206] (2) The behavior of the luminescent powder under different light sources and the changes in color and gloss with changes in viewpoint can be confirmed in real time, allowing users to understand in advance the effects of pigments in various environments.

[0207] (3) The unique brilliance effects of interference pearlescent pigments and colored pearlescent pigments can be reproduced with high precision. For interference pearlescent pigments, a physical model utilizing the principle of thin-film interference is used to accurately simulate the phenomenon in which color changes depending on the observation angle due to the interference effect caused by multiple internal reflections. Meanwhile, to reproduce colored pearlescent pigments, a separate model formula was developed to achieve a metallic appearance, using a mathematical model to express the color characteristics of the pigment itself and the blurring and interference of light reflection characteristics. These models are based on the differences in the physical properties and reflection mechanisms of interference pearlescent pigments and colored pearlescent pigments, and further improvements in accuracy are expected in the future as technology advances.

[0208] (4) Not only can it realistically reproduce the particle size variation of photoluminescent powders, but by changing the average particle size and standard deviation, it is also possible to adjust the particle size distribution in real time and predict and visualize various situations. By using a log-normal distribution, the wide range of size variation seen in actual photoluminescent powders, from very fine particles to relatively large particles, can be captured in a single distribution. By adjusting material information parameters based on this, it is possible to predict through computer simulations during the product prototyping process without changing the actual material formulation, and to explore the optimal appearance and usability. This function is a significant advantage in accurately predicting actual usability and visual effects, greatly streamlining the product development process.

[0209] (5) Image generation technology using normal maps makes it possible to realistically reproduce the unique texture and luster of glittering powder. It is particularly effective in reproducing fine details, as it can visually express three-dimensional unevenness and texture while keeping calculation costs low.

[0210] (6) This technology accurately captures the optical properties of glitter powders, enabling accurate reproduction of the color characteristics of pigments, particularly those of colored pearlescent pigments. Optical property information, including spectral reflectance, is acquired based on the geometric relationship between the light incidence and reflection angles—i.e., the line-of-sight vector, normal vector, and lighting direction vector. A spectrophotometer is used for this measurement, emphasizing the importance of the angle at which light from the light source strikes the object (light source angle) for color reproducibility. It is particularly important to measure the spectral reflectance of light reflected between the lighting direction vector and the surface at a specific angle. This captures the actual color, eliminating the texture element of the cosmetic film, improving the reproducibility of the primary color of glitter powders. This technology significantly improves the color reproducibility of the surface of objects to which glitter powders are applied.

[0211] (7) By calculating the first color signal C1(λ) using the first three-dimensional reflection characteristic model and the second color signal C2(λ) using the second three-dimensional reflection characteristic model, it is possible to calculate a third color signal C3(λ) in which the color of the object to be coated is reflected in the pigment as the color signal of the object to be coated in a pigment-coated state. Then, by calculating the second color value from the third color signal C3(λ), it is possible to generate a second image including the object to be coated in a pigment-coated state. This allows for image generation with higher reproduction accuracy than simply superimposing the color of the pigment on the area of ​​the object to be coated. Furthermore, because the gloss caused by the illumination light can be reproduced, the texture of the object to be coated in a pigment-coated state can be reproduced in the second image.

[0212] In this specification, "reproduction accuracy" means the ability to reproduce in real time the three-dimensional texture of an actual pigment, such as its gloss, color, and shading, which changes from moment to moment in response to the position, orientation, or lighting environment of the object to be coated.

[0213] (8) By calculating the first spectral reflectance S1(λ) based on the first conversion function M corresponding to the color of the skin or lips as the object to be applied, it is possible to calculate a first spectral reflectance S1(λ) that is closer to the spectral reflectance of actual skin or lips.

[0214] (9) By calculating the second spectral reflectance S2(λ) based on the first spectral reflectance S1(λ) and the second conversion function T(λ) corresponding to the color of the object to be coated, it is possible to calculate the second spectral reflectance S2(λ) that is closer to the spectral reflectance of the pigment when it is actually applied to the object to be coated and blends in with the object to be coated.

[0215] (10) By calculating the second color signal C2(λ) using the second spectral reflectance S2(λ) and the second reflection characteristic parameter for each pigment, a second image that reflects the color and reflection characteristics of each pigment can be generated.

[0216] (11) By using the third three-dimensional reflection characteristic model, the color signal C of the glitter powder G Then, the first color signal C1(λ), ​​the second color signal C2(λ), and the glitter powder color signal C G By calculating the second color value from the third color signal C3(λ) calculated based on (λ), it is possible to generate a second image that reproduces the gloss of the glitter powder when the pigment is applied. Also, in the area where the glitter powder is scattered, estimated from the shape information and dispersion information, the color signal C of the glitter powder is G By calculating (λ), the particle shape of the glittering powder and the dispersion state of the glittering powder can be reproduced.

[0217] (12) By calculating the first color signal C1(λ) and the second color signal C2(λ) based on the illumination spectral distribution E(λ) of the illumination environment estimated as the illumination environment of the first image and the spatial distribution of the illumination light source, the illumination environment is reflected in the second image. Therefore, a makeup simulation with high reproduction accuracy that reflects the illumination environment in the first image can be performed.

[0218] (13) By using the GPU 23 to execute processing in the image generation system 1, real-time image processing becomes possible, enabling more appropriate image generation. For example, image generation can be performed using augmented reality (AR) or mixed reality (MR).

[0219] (14) The results of measurements made by various measuring instruments such as spectrophotometers, gloss meters, and colorimeters can be stored in the pigment database 24c as parameters that indicate the characteristics of pigments. This allows the measurement results from the measuring instruments to be directly used in image generation.

[0220] (15) By appropriately configuring a mathematical model (e.g., a three-dimensional reflection property model), it is possible to reproduce glittering powders with high accuracy and realism, without relying on the designer's sensitivity or technical ability. Unlike conventional CG technology, which uses RGB values ​​to specify colors, this invention uses material information as input and performs physical calculations to automatically generate colors. This allows for natural and accurate reproduction of the color changes of glittering powders that change depending on the angle of the viewpoint and light source.

[0221] (16) It is possible to decompose each material that makes up a pigment into an individual mathematical model and describe each as an independent block structure. The advantages of adopting this structure are as follows:

[0222] (16-1) It is possible to reproduce images that correspond to three-dimensional changes, such as gloss, shadows, and color changes depending on lighting and the angle of the viewpoint, so it is possible to reproduce the texture of real pigments and changes in texture depending on changes in observation conditions in an identical way to the real thing.

[0223] (16-2) In pigment development, describing each individual composition using a block structure of a mathematical model makes it possible to not only remove or add specific components, but also to change parameters in the simulation. The parameters here refer to the numerical data given to the mathematical model, such as changes in content, the addition or subtraction of materials that enhance specific wavelengths, or changes in particle size. This makes it possible to perform computer simulations (visual simulations) that respond to changes in the component composition and properties of pigments without having to prepare actual pigments.

[0224] (16-3) Increasing the accuracy of the mathematical model can improve the functionality of the image generation system 1. However, increasing the functionality of the system can potentially result in a decrease in processing speed. Taking this into consideration, it is possible to increase the speed of the image generation system 1 by lowering the accuracy of the mathematical model.

[0225] (16-4) When it is difficult to model the entire system, such as pigments with complex compositions, it is possible to independently describe low-precision models, such as partially simplified models or empirical models that include hypotheses, as blocks. By improving measurement accuracy, etc., it is possible to gradually improve the accuracy of the entire model by replacing low-precision models (blocks) with high-precision models.

[0226] (16-5) In particular, this system makes it possible to effectively reproduce pigments with complex optical effects, such as color travel pearls, which have been difficult to reproduce using conventional methods. Color travel pearls have the characteristic of changing color depending on the angle of the viewpoint and light source, and their reproduction requires advanced modeling. In this invention, an empirical model is used to capture these characteristics, allowing the unique color changes of color travel pearls, which could not be reproduced using conventional methods, to be accurately simulated on a computer.

[0227] (16-6) Furthermore, one of the major advantages of this method is that it allows us to predict the properties of unknown materials and build models from available measurement data. This makes it possible to create models with a certain degree of accuracy even for materials with complex compositions, greatly facilitating the understanding and development process of materials.

[0228] The above description of the image generating method according to the first embodiment of the present invention can be applied to other embodiments of the present invention unless there is a particular technical contradiction.

[0229] [2. Second embodiment (example of makeup simulation method)] The image generation method according to the first embodiment can be applied to makeup simulation. This makeup simulation involves treating glitter powder as a cosmetic and acquiring its optical property information and shape information. The acquired information is supplied to a mathematical model, i.e., a three-dimensional reflection property model, which explains the light reflection process of the glitter powder, and used to calculate a color signal for the glitter powder. Next, a step is performed in which the color value of the glitter powder is calculated based on this color signal, and an image of the glitter powder applied is generated based on the color value and shape information. This provides a makeup simulation method.

[0230] That is, the present invention includes a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of glittering powder, which is a cosmetic; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the glitter powder based on the color signal; and an image generating step of generating an image of the glitter powder applied based on the color value and the shape information.

[0231] The metallic color generating agent information acquisition step, color signal calculation step, color value calculation step, and image generation step have been described with reference to FIG. 1, so a repeated description will be omitted.

[0232] A makeup simulation method according to this embodiment will be described with reference to FIGS. 9 and 10. FIGS. 9 and 10 show examples of screens displayed in a makeup simulation method according to one embodiment of the present invention. As shown in FIG. 9, an application image 40, which is an application image displayed by an image display program, is displayed on the touch panel 13 of the user terminal 10. The application image 40 includes an image display area 41 and an operation area 42. The image display area 41 is an area where an image 30, such as a first image and a superimposed image in which the first image and a second image are superimposed, is displayed.

[0233] The operation area 42 includes, for example, a switching object 42a, a makeup area selection object 42b, a cosmetic selection object 42c, a lighting selection object 42d, and an operation object 42e. The switching object 42a is an object that switches the simulation results for each cosmetic between ON and OFF.

[0234] The makeup area selection object 42b is an object for manually specifying the makeup area 31, which is the area on the application target where the cosmetic is to be applied. One example of a method for specifying the makeup area 31 is to specify the area to which the cosmetic is to be applied by tapping on the first image 30a displayed in the image display area 41. For example, the makeup area 31 may be the entire facial skin for foundation, the skin around the cheekbones for blush, the skin around the eyes for eye color, or the lips for lipstick. Note that the makeup area 31 may also be set automatically using an image recognition function or the like.

[0235] The cosmetic selection object 42c is an object for selecting the cosmetic product to be simulated. As shown in FIG. 10, by tapping the cosmetic selection object 42c, a cosmetic selection area 43 is displayed in the image display area 41. In the cosmetic selection area 43, cosmetics registered in the pigment database 24c are displayed for each product as product objects 43a. By tapping the product object 43a, the cosmetic product to be applied in the makeup simulation is selected.

[0236] As a specific example, in the case of foundation, the product object 43a of the product to be simulated is selected from the product objects 43a displayed for each type of foundation, such as liquid foundation or powder foundation, and for each color.

[0237] As another example, the selection of the makeup area and the cosmetics may be performed without using the makeup area selection object 42b and the cosmetics selection object 42c. For example, by tapping the area to which the cosmetics are to be applied in the first image displayed in the image display area 41, the cosmetics corresponding to the tapped area may be automatically selected. As a specific example, if the area near the cheekbone is tapped in the first image, blush is automatically selected as the cosmetics, and the cosmetics are applied using an application method suitable for that blush, as specified by a makeup expert. This allows the image of cosmetics applied by a makeup expert with various makeup know-how to be reproduced. The application method may be selected from multiple options, and an application method may be selected according to, for example, age, mood, or time, place, and occasion.

[0238] The lighting selection object 42d is an object for selecting a lighting environment in the makeup simulation. By tapping the cosmetic selection object 42c, a lighting selection area (not shown) is displayed in the image display area 41. In the lighting selection area, various lighting environments registered in the lighting environment database 24b are displayed as lighting objects for each lighting environment. By tapping the lighting object, the lighting environment to be applied in the makeup simulation is selected. For example, if a user wants to perform the makeup simulation in a lighting environment different from the lighting environment in which the first image was captured, the user may tap the lighting selection object to select any lighting environment.

[0239] The operation object 42e is an object for moving, rotating, enlarging, and reducing the image displayed in the image display area 41.

[0240] For example, by using a makeup simulation system (image generation system 1) to simulate various cosmetics for user 2, it becomes possible to provide counseling that suggests the most suitable cosmetics to meet user 2's needs. Utilizing this system, advice can be given on choosing cosmetics that are optimized for user 2's skin type, preferences, and even lighting environment. Before actually using the cosmetics, user 2 can check the effects of various products through a digital simulation and find the cosmetics that suit them best. This type of counseling reduces mistakes when purchasing cosmetics and supports product selection that results in greater satisfaction. This system can also be effectively used in counseling by professionals such as beauticians and cosmetics salespeople, enabling personalized suggestions tailored to each customer.

[0241] The makeup simulation system of this embodiment can also provide a function for the user to purchase cosmetics selected through the makeup simulation. Specifically, this system supports a process for purchasing cosmetics applied to a superimposed image as a simulation result generated using an image display program. To realize this process, the server 20 can include a payment unit for executing payment processing.

[0242] Users can use the makeup simulation system to search for cosmetics that match their preferences and purchase the ones they choose directly. This function allows users to check the effects of various cosmetics through a digital simulation before purchasing the actual cosmetics, allowing them to find the product that is best suited to them. This reduces mistakes when purchasing cosmetics and helps users make more satisfying choices.

[0243] The present invention is expected to be applicable not only to cosmetic simulation but also to a wide range of fields, such as vehicle painting, art works, fashion design, etc. It is particularly effective when realistic color expression is required in the design and presentation of products that use glittering powder.

[0244] Furthermore, the color reproduction method based on physical calculations of this invention is also very useful in the fields of education and research. Students and researchers can use simulations based on this technology as a supplement to experimental observations when gaining a deeper understanding of the optical properties of materials.

[0245] The above description of the makeup simulation method according to the second embodiment of the present invention can be applied to other embodiments of the present invention unless there is a particular technical contradiction.

[0246] [3. Third embodiment (example of image generating device)] The present invention provides an image generation device comprising: an acquisition unit that acquires at least one of optical property information and shape information of a glittering powder; a color signal calculation unit that applies the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glittering powder, to calculate a color signal of the glittering powder; a color value calculation unit that calculates a color value of the glittering powder based on the color signal; and an image generation unit that generates an image of the glittering powder in a state where it is applied, based on the color value and the shape information.

[0247] An example of the configuration of an image generating device according to one embodiment of the present invention will be described with reference to Fig. 11. Fig. 11 is a block diagram showing an example of the configuration of an image generating device according to one embodiment of the present invention.

[0248] As shown in FIG. 11, an image generating device 70 according to one embodiment of the present invention includes an acquisition unit 71, a color signal calculation unit 72, a color value calculation unit 73, and an image generating unit 74.

[0249] The acquisition unit 71 acquires at least one of optical property information and shape information of the glittering powder. As described above, the acquisition unit 71 performs the glittering agent information acquisition step (step S1). The acquisition unit 71 can be, for example, a spectrophotometer or a computer display.

[0250] The color signal calculation unit 72 calculates a color signal of the glittering powder by inputting the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glittering powder. As described above, the color signal calculation unit 72 may be the glittering agent color signal calculation unit 23j, etc.

[0251] The color value calculation unit 73 calculates the color value of the glittering powder based on the color signal. As described above, the color value calculation unit 73 can be the color value calculation unit 23l or the like.

[0252] The image generating unit 74 generates an image of the state in which the glittering powder has been applied based on the color value and shape information. As described above, the image generating unit 74 can be the second image generating unit 23m, the superimposed image generating unit 23n, or the like.

[0253] The above description of the image generating device according to the third embodiment of the present invention can be applied to other embodiments of the present invention unless there is a particular technical contradiction.

[0254] [4. Fourth embodiment (example of image generation program)] The present invention provides an image generation program that causes a computer to execute the following steps: a lustrous agent information acquisition step of acquiring at least one of optical property information and shape information of the lustrous powder; a color signal calculation step of applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the lustrous powder, to calculate a color signal of the lustrous powder; a color value calculation step of calculating a color value of the lustrous powder based on the color signal; and an image generation step of generating an image of the lustrous powder in a state where it has been applied, based on the color value and the shape information.

[0255] As described above, the image generation program executes various processes in the server 20. The image generation program is a program that causes the server 20 to execute a process of generating a second image and generating a superimposed image in which the first image and the second image are superimposed. The process in the image generation program is controlled by the GPU 23 included in the calculation unit 21.

[0256] The present invention includes an image generation program recorded on a non-volatile computer-readable storage medium (e.g., a flash drive, an SSD (Solid State Drive), a ROM (Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory), etc.), a magnetic storage medium (e.g., a hard disk drive or a floppy disk), or an optical storage medium (e.g., a CD-ROM, a DVD-ROM, or a Blu-ray disk). When the storage medium is inserted into a computer system, the computer can execute the image generation program.

[0257] The above description of the image generating program according to the fourth embodiment of the present invention can be applied to other embodiments of the present invention unless there is a particular technical contradiction.

[0258] The embodiments of the present invention are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present invention. The specific numerical values, shapes, materials (including compositions), etc. described are merely examples and are not limiting.

[0259] The present invention can also have the following configuration. [1] a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the state in which the glitter powder is applied based on the color signal; and an image generating step of generating an image of a state in which the glitter powder is applied based on the color value and the shape information. [2] the glitter powder contains at least one of an interference pearl pigment and a colored pearl pigment, In the color signal calculation step, the optical property information of the interference pearl pigment is given to an interference system three-dimensional reflection property model to calculate the color signal of the interference pearl pigment; In the color signal calculation step, optical property information of the colored pearl pigment is provided to a color system three-dimensional reflection property model to calculate a color signal of the colored pearl pigment. [1] The image generation method according to [1]. [3] The coloring system three-dimensional reflection characteristic model includes at least one of a physical model that describes the physical light reflection characteristics of the glitter powder and an empirical model that describes the light reflection characteristics of the glitter powder based on a hypothesis, [2] The image generation method according to [2]. [4] The glittering powder is configured by coating a base powder with one or more thin layers, In the color signal calculation step, the wavelength of the interference light that depends on the thickness of the thin film is input to the three-dimensional reflection characteristic model to calculate the color signal. The image generating method according to any one of [1] to [3]. [5] The optical property information acquired in the metallic color information acquisition step includes the wavelength, [4] The image generation method according to [4]. [6] The shape information acquired in the glittering agent information acquisition step includes an average particle size and a standard deviation of the glittering powder, In the image generating step, the image is generated using a probability distribution based on the average particle size and the standard deviation. [1] to [5]. An image generating method according to any one of [1] to [5]. [7] the image is represented using a normal map; [1] to [6]. An image generating method according to any one of [1] to [6]. [8] The planar shape of the glitter powder in the image is polygonal. [1] to [7]. An image generating method according to any one of [1] to [7]. [9] The particle size of the glitter powder in the image is 3 mm or less. [1] to [8]. An image generating method according to any one of [1] to [8].

[10] the optical characteristic information includes a spectral reflectance, In the glittering agent information acquisition step, the spectral reflectance acquired by the spectrophotometer is the spectral reflectance of light reflected in a direction between an illumination direction vector and the surface of the object to which the glittering powder is applied, or the spectral reflectance of light reflected in a direction between a specular reflection direction vector of the illumination direction vector and the surface. [1] to [9]. An image generating method according to any one of [1] to [9].

[11] When the horizontal direction with respect to the surface is set to 0 degrees, the reflection angle of light acquired by the spectrophotometer is 25 degrees or less.

[10] The image generation method according to

[10] .

[12] a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder which is a cosmetic; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the state in which the glitter powder is applied based on the color signal; and an image generating step of generating an image of the glitter powder applied based on the color value and the shape information.

[13] a glittering agent information acquisition unit that acquires at least one of optical property information and shape information of the glittering powder; a color signal calculation unit that calculates a color signal of the glitter powder by providing the optical property information to a three-dimensional reflection property model that is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation unit that calculates a color value of the state in which the glitter powder is applied based on the color signal; an image generating unit that generates an image of a state in which the glitter powder is applied based on the color value and the shape information.

[14] a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the state in which the glitter powder is applied based on the color signal; an image generating step of generating an image of a state in which the glittering powder is applied based on the color value and the shape information. [Explanation of symbols]

[0260] S1 Photochromic material information acquisition process S2 Image acquisition process S3 Shape estimation process S4 Surface identification process S5 Lighting environment estimation process S6 Application area reception process S7 First spectral reflectance calculation process S8 First color signal calculation process S9 Characteristic part identification process S10 Second spectral reflectance calculation process S11 Second color signal calculation process S12 Photochromic material information acquisition process S13 Third color signal calculation process S14 Color value calculation process S15 Second image generation process S16 Superimposed image generation process 1. Image generation system 2 users 3 Network 10 User terminal 11 Control section 12 Memory 13 Touch Panel 14 Imaging unit 15 Communications Department 20 servers 21 Arithmetic section 23a Image acquisition unit 23b Shape estimation part 23c Surface specific part 23d Lighting environment estimation section 23e 1st spectral reflectance calculation section 23f 1st color signal calculation section 23h 2nd spectral reflectance calculation section 23i 2nd color signal calculation section 23j Fluorescent pigment color signal calculation unit 23k 3rd color signal calculation section 23l Color value calculation unit 23m 2nd image generation section 23n Superimposed image generation unit 24 memory 24a Spectral Conversion Database 24b Lighting Environment Database 24c Pigment Database 24d Transformation matrix storage 24g Feature identification section 25 Communications Department 60 Bright powder 61 Thin Film 62 Base material 70 Image generation device 71 Acquisition Department 72 Color signal calculation section 73 Color value calculation unit 74 Image Generation Unit

Claims

1. a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the state in which the glitter powder is applied based on the color signal; and an image generating step of generating an image of a state in which the glitter powder is applied based on the color value and the shape information.

2. the glitter powder contains at least one of an interference pearl pigment and a colored pearl pigment, In the color signal calculation step, the optical property information of the interference pearl pigment is given to an interference system three-dimensional reflection property model to calculate the color signal of the interference pearl pigment; In the color signal calculation step, optical property information of the colored pearl pigment is provided to a color system three-dimensional reflection property model to calculate a color signal of the colored pearl pigment. The image generating method according to claim 1 .

3. The coloring system three-dimensional reflection characteristic model includes at least one of a physical model that describes the physical light reflection characteristics of the glitter powder and an empirical model that describes the light reflection characteristics of the glitter powder based on a hypothesis, The image generating method according to claim 2 .

4. The glittering powder is configured by coating a base powder with one or more thin layers, In the color signal calculation step, the wavelength of the interference light that depends on the thickness of the thin film is input to the three-dimensional reflection characteristic model to calculate the color signal. The image generating method according to any one of claims 1 to 3.

5. The optical property information acquired in the metallic color information acquisition step includes the wavelength, The image generating method according to claim 4.

6. The shape information acquired in the glittering agent information acquisition step includes an average particle size and a standard deviation of the glittering powder, In the image generating step, the image is generated using a probability distribution based on the average particle size and the standard deviation. The image generating method according to any one of claims 1 to 3.

7. the image is represented using a normal map; The image generating method according to any one of claims 1 to 3.

8. The planar shape of the glitter powder in the image is polygonal. The image generating method according to any one of claims 1 to 3.

9. The particle size of the glitter powder in the image is 3 mm or less. The image generating method according to any one of claims 1 to 3.

10. the optical characteristic information includes a spectral reflectance, In the glittering agent information acquisition step, the spectral reflectance acquired by the spectrophotometer is the spectral reflectance of light reflected in a direction between an illumination direction vector and the surface of the object to which the glittering powder is applied, or the spectral reflectance of light reflected in a direction between a specular reflection direction vector of the illumination direction vector and the surface. The image generating method according to any one of claims 1 to 3.

11. When the horizontal direction with respect to the surface is set to 0 degrees, the reflection angle of light acquired by the spectrophotometer is 25 degrees or less. The image generating method according to claim 10.

12. a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder which is a cosmetic; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the state in which the glitter powder is applied based on the color signal; and an image generating step of generating an image of the glitter powder applied based on the color value and the shape information.

13. a glittering agent information acquisition unit that acquires at least one of optical property information and shape information of the glittering powder; a color signal calculation unit that calculates a color signal of the glitter powder by providing the optical property information to a three-dimensional reflection property model that is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation unit that calculates a color value of the state in which the glitter powder is applied based on the color signal; an image generating unit that generates an image of a state in which the glitter powder is applied based on the color value and the shape information.

14. a glittering agent information acquisition step of acquiring at least one of optical property information and shape information of the glittering powder; a color signal calculation step of calculating a color signal of the glitter powder by applying the optical property information to a three-dimensional reflection property model, which is a mathematical model that describes the light reflection process of the glitter powder; a color value calculation step of calculating a color value of the state in which the glitter powder is applied based on the color signal; an image generating step of generating an image of a state in which the glitter powder is applied based on the color value and the shape information.

Images