Interactive 3D physical-realistic rendering system for color visualization

Abstract

A method comprising: Generating one or more user-selectable virtual menus on a display (4), the user-selectable virtual menus; Interactively updating one or more perspective, hue, tone, or shade values of a three-dimensional virtual rendering of a body area, such as hair, by applying a transformation (8) configured to convert or map user-selectable menu entries (5). Figure: 9

Description

Title of the invention: Interactive 3D physical-realistic rendering system for color visualization

[0001] The present invention relates to a method for constructing an interactive 3D model for visualizing a color of a body area, in particular hair, in the virtual world. It also relates to the use of this method for reproducing and visualizing, in the virtual world, on a calibrated display unit, a color applied in the real world, to a body area, in particular hair.

[0002] More generally, a cosmetic product is a product as defined in Regulation EC No. 1223 / 2009 of the European Parliament and of the Council dated November 30, 2009, relating to cosmetic products.

[0003] The body area can be the skin, lips, eyelashes, eyebrows, nails and especially the hair. Technological background

[0004] While our eye is a wonderful partner in showing us the world, it is unfortunately quite incapable of helping us calibrate a graphics chain. However, good color management between a photo of an object displayed on a screen and the real object can only be achieved if the screen is calibrated.

[0005] All color management is centered on this step: calibration. It is thanks to the calibration (also called calibration) of the different peripherals and the knowledge of the colorimetric characteristics that we can virtually reproduce the color closest to the real color.

[0006] The aim of calibration and then characterization is to know precisely the colorimetric characteristics of a given device or, in a pictorial way, how the different peripherals "distort" the colors - which colors they "see" for a given signal - in order to be able to "correct" them afterwards and which colors they are capable of reproducing.

[0007] For example, we know software for creating ICC profiles sold with probes and other colorimeters which compare a colored target whose characteristics are precisely known, that is to say the one and only L*a*b* color of a colored sample, to the color actually printed by default by a printer, to the exact wavelength of the RGB displayed on the screen.

[0008] Patent US9924778B2 discloses an apparatus for measuring the optical properties of skin, comprising:

[0009] a spectrometer and a light source under the control of a processing circuit;

[0010] an optical cavity in which illumination from the light source is provided, the cavity having an opening for illuminating a portion of the skin to be measured;

[0011] a calibration / normalization reference having a surface to be positioned such that the aperture illuminates the calibration / normalization reference;

[0012] a storage circuit coupled to the processing circuit that stores linearization coefficients and spectral data for the calibration / normalization reference and stores data for a classifier prediction algorithm;

[0013] a display under control of the processing circuit; and

[0014] a wireless module under control of the processing circuit;

[0015] the processing circuit controls the display of the control information on the display;

[0016] in response to user activation based on the displayed control information, a measurement of the calibration / normalization reference is performed and stored in the storage circuit.

[0017] Patent EP2661222B1 discloses a system for analyzing at least one characteristic of the skin by means relating to optics and computer image processing, in particular for detecting skin cancer or for proposing at least one product suitable for the skin and / or hair, comprising:

[0018] a consumer electronic device, comprising a lens, capable of acquiring at least one image by digital means and capable of producing data relating to said at least one acquired image; and characterized in that it comprises:

[0019] specific means, distinct from said consumer electronic device, for producing an optical magnification of the image capable of analyzing at least one characteristic of the skin; and

[0020] means for introducing at least one piece of information into the image, these means being included in said specific means for carrying out an optical magnification of the image; and

[0021] means for processing said data in relation to said at least one acquired image with a view to producing information relating to at least one characteristic of the skin,

[0022] said specific means for producing an optical magnification of the image further comprise means for calibrating at least one characteristic of the image.

[0023] However, the calibration and visualization methods known to date can still be improved from the point of view of their reliability in the real world.

[0024] The problem posed by the invention is to provide a system allowing:

[0025] To digitally visualize all possible colorings, in particular hair colorings, so that the user can visualize the result of a coloring on different supports, including but not limited to straight and curly supports, that is to say in particular more or less straight hair and more or less wavy hair (i.e. on the whole head, or in single samples);

[0026] To explore different colors on a medium by starting with a selected color and adjusting it to achieve the desired result by selecting different parameters (ash, gold, matte etc.);

[0027] To link each digital color visualization to a characterized result in the real world via standard measurements (such as Lab values, brightness or other) to make the connection between the two worlds;

[0028] To propose a color formula which produces the desired color on a given support;

[0029] To reproduce any color of a body area, including hair in real life on a calibrated screen by providing an accurate digital representation that can be adjusted to achieve any desired color;

[0030] To predict the result of a coloring on different supports, in particular on hair. Definition of invention

[0031] The invention relates to a method comprising:

[0032] generating one or more user-selectable virtual menus on a display, the user-selectable virtual menus comprising one or more:

[0033] menu for selecting a viewpoint or perspective;

[0034] menu for selecting a library of colors of a body area, in particular hair;

[0035] window selection menu;

[0036] tool selection menu;

[0037] color lock selection menu;

[0038] quick save selection menu; and

[0039] interactively updating one or more perspective, hue, tone, or shade values of a three-dimensional virtual rendering of a body area, including hair, by applying a transformation configured to convert or map user-selectable menu entries into a set of parameters of a color space, including CIELAB, which, when applied to the three-dimensional virtual rendering of the body area, including hair, results in a change to one or more perspective, hue, tone, or shade values of the virtual rendering of the body area, including hair.

[0040] By integrating state-of-the-art rendering algorithms and meticulous color mapping, the system ensures that every color of a virtually rendered body area, including hair, aligns with the real world. By leveraging real-time 3D rendering capabilities, the method achieves physico-realistic results. realistic, setting a new benchmark in hair color representation. The ability to instantly and seamlessly visualize and reproduce every color of a body area, including hair, in real life on a calibrated display, opens new perspectives in various industries, such as beauty, fashion, entertainment, offering a truly immersive and realistic digital experience.

[0041] Thanks to the invention, it is now possible to instantly visualize and reproduce each hair color in real life on a calibrated screen equipped with an appropriate expert interpretation user interface.

[0042] The invention also relates to a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method as described above.

[0043] The invention also relates to a computer program product comprising instructions which, when the program is executed by a computer, cause the latter to: Display on an interactive interface one or more virtual menus selectable by a user, the virtual menus selectable by the user comprising one or more:

[0044] menu for selecting a viewpoint or perspective;

[0045] menu for selecting a library of colors of a body area, in particular hair;

[0046] window selection menu;

[0047] tool selection menu;

[0048] color lock selection menu;

[0049] quick save selection menu; and

[0050] Receiving user input to interactively update one or more perspective, hue, tone, or shade values of a three-dimensional virtual rendering of a body area, including hair;

[0051] Processing user input by applying a transformation configured to convert or map user-selectable menu entries into a set of parameters of a color space, including CIELAB, which, when applied to the three-dimensional virtual rendering of a body area, including hair, results in a change to one or more perspective, hue, tone, or shade values of the virtual rendering of the body area, including hair.

[0052] The invention also relates to a data processing system comprising:

[0053] A memory in which virtual menus are stored that can be selected by the user including one or more instances of

[0054] A menu for selecting a viewpoint or perspective;

[0055] A menu for selecting a library of colors of a body area, especially hair;

[0056] a window selection menu;

[0057] a tool selection menu;

[0058] a color lock selection menu;

[0059] a quick save selection menu;

[0060] a three-dimensional virtual rendering of hair

[0061] A processing unit configured to implement the computer program product as described above,

[0062] A screen configured to display selectable virtual menus and three-dimensional virtual rendering of hair.

[0063] Preferably, the system comprises a color calibration unit by an expert comparison between the three-dimensional virtual rendering of hair and real colored hair, natural or synthetic, until an identical color detection between the three-dimensional virtual rendering of hair and the real colored hair.

[0064] The invention also relates to the use of the system as described previously, to predict the result of a coloring from a three-dimensional virtual rendering of hair.

[0065] The invention also relates to the use of the system as described above, for recommending a coloring on a body area.

[0066] The invention also relates to the use of the system as described above, to interactively align with a person on a desired coloring and, possibly, to associate a cosmetic composition with the desired color.

[0067] The invention also relates to the use of the system as described above, for a virtual fitting in photo or video.

[0068] The invention also relates to the use of the system as described above, to display a virtual half-head section and to directly compare the result of two colorings.

[0069] The present invention introduces an advanced technique that harnesses the power of a state-of-the-art real-time 3D rendering product to accurately characterize the vast color space of a body area, including hair, present in the real world. By collecting a diverse data set comprising real samples of a body area, including hair, whether obtained from a medium, including real or synthesized hair, the proposed method enables expert analysis and mapping of hair color information onto a digital representation.

[0070] From a limited number of samples of a body area, in particular carefully selected hair, the method constructs a color space model, and this model allows instant and transparent visualization of any hair color in real life on a properly calibrated screen.

[0071] The method for constructing an interactive 3D model according to the invention comprises three components:

[0072] 1) Realistic 3D rendering of hair in real time

[0073] State-of-the-art technologies use physically based rendering techniques to accurately simulate light scattering, light absorption, reflection, and transmission, in media, including hair strands of various types, ensuring that the rendered hair appears visually convincing and exhibits realistic shades, shadows, and highlights.

[0074] 2) A synthesis and mapping of expertise

[0075] The characterization of the physical coloring space of a body area, in particular hair, makes it possible to ensure the correspondence between a real hair color and its digital representation.

[0076] 3) Construction of a color space model through samples of data validated by expert physical-visual experiments. Preferred embodiments

[0077] Preferably, the method according to the invention has one or more of the following characteristics, taken alone or in combination:

[0078] The interactive update includes an expert comparison between the three-dimensional virtual rendering of the body area, including hair, and the real body area, including real colored, natural, or synthetic hair, up to a visually identical color detection between the three-dimensional virtual rendering of the body area, including hair, and the real body area, including real colored hair, in order to calibrate the color display.

[0079] The update is carried out by an interface comprising levers to vary the colors of the virtual rendering.

[0080] The three-dimensional virtual rendering of the body area, in particular hair, is obtained by "path tracing", "ray tracing", "inverse rendering", generative AI or diffusion AI.

[0081] The update is carried out by levers of a color space, in particular CIELAB or by intrinsic levers of three-dimensional virtual rendering.

[0082] The transformation includes a construction of a 3D model in the color space, in particular CIELAB and an interpolation from each point of the 3D model.

[0083] Interpolation is 3D interpolation.

[0084] Each point of the 3D model is associated with a numerical value, in particular an intrinsic value of an expert language, a rendering engine, colony values or others.

[0085] A selection menu is associated with hair softness, hair type, hair quality, hair dimensions, skin quality, skin thickness, or skin type. Description of the figures

[0086] Other characteristics and advantages of the invention will emerge from reading the detailed description which follows, from non-limiting examples of implementation thereof, and from examining the appended schematic and partial drawing, in which: Description of the figures

[0087] [Fig.l]

[0088] [Fig.l] represents a display unit presenting a virtual reference image and virtual reference digital parameters, before determining the reference 3D rendering associated with the real wick according to [Fig.2],

[0089] [Fig.2]

[0090] [Fig.2] represents a lock of real hair,

[0091] [Fig.3]

[0092] [Fig.3] represents a 3D model in Lab space,

[0093] [Fig.4]

[0094] [Fig.4] represents a tetrahedron associated with a point of the 3D model of [Fig.3],

[0095] [Fig.5]

[0096] [Fig.5] represents a color chart in the real world,

[0097] [Fig.6]

[0098] [Fig.6] represents a list of expert notions on the user interface,

[0099] [Fig.7]

[0100] [Fig.7] represents a target photo in the real world,

[0101] [Fig.8]

[0102] [Fig.8] represents a user interface after selection of a reference 3D rendering associated with the real wick according to [Fig.2],

[0103] [Fig.9]

[0104] [Fig.9] represents a block diagram of the method according to the invention.

[0105] In order to obtain an accurate replication of the colors, a first step is to establish a connection between the real world 1 and the virtual world 2, that is, between the domain of digital colors and the domain of physical colors. To do this, strands of synthetic or real hair 3 are used, which simulate real colorings 31, as tangible references for coloring the hair in the real world 1.

[0106] In the real world 1, a lock of hair 3 is placed on a support 40 to which a real target color 31 has been applied.

[0107] In Figure 2, such a wick 3 can be seen carefully arranged in a predefined environment, subjected to predetermined lighting conditions. In the example shown, each wick 3 of actual target color 31 is suspended from a support 40 placed in the interior space of a closed box 41, a front side wall 42 of which is transparent so that the wick 3 is visible to an expert.

[0108] 3D rendering

[0109] In [Fig.l], a display unit 4 displays a virtual reference image 21 of colored hair (or 3D rendering) having a digital reference color 22. The unit 4 further presents virtual digital reference parameters 5 in relation to each virtual reference image 21. The virtual digital reference parameters 5 are the user-selectable menu entries.

[0110] In practice, the expert modulates the reference digital color 22 of the reference virtual image 21 (3D rendering) by modifying the reference virtual parameters 5, until a visual correspondence is obtained with the real target color 31 of the strand 3.

[0111] As illustrated in [Fig.3], a 3D model is constructed in the CIELAB color space, based on the calibrated hair color references.

[0112] As a reminder, the CIE 1976 Lab color space, generally called CIELAB, is a color space particularly used for the characterization of surface colors. Three quantities characterize the colors: the lightness L derived from the luminance of the surface; two parameters a and b which express the deviation of the color from that of a gray surface of the same lightness. The existence of a gray, non-colored, achromatic surface implies explicitly indicating the composition of the light which illuminates the colored surface.

[0113] The Lab system parameters are:

[0114] the clarity L takes values between 0 (black) and 100 (reference white);

[0115] the parameter a represents the value on an axis going from green to red;

[0116] parameter b represents the value on an axis going from blue to yellow.

[0117] In the example shown, the 3D model defines nodes 35, 36, 37, 38, 39, with segments indicating the distance between digital colors calibrated according to LAB values.

[0118] The 3D model transforms for example; into a 3D Delaunay triangulation (tetrahedralization), where each tetrahedron (A, B, C, D) is composed of four shades ( [Fig.4]), which makes it possible to obtain the LAB value of any point. Because the color of each strand of color chart 7 in [Fig.5] has been calibrated, as evidenced by the hair rendering images in [Fig.l], it is possible to generate any desired hair color in real time by exploiting this 3D model of the CIELAB color space. The tetrahedron 46 in [Fig.4] is associated with a any point 45 of the 3D model.

[0119] In [Fig.5], a color chart 7 has been shown schematically without showing the shades but in reality, all the hair in the color chart is shown with more or less dark or colored shades.

[0120] Several libraries exist for triangulation, such as CGAL.

[0121] For each node 35, 36, 37, 38, 39 in the 3D space, its Lab value was measured (it can be measured in any other color or appearance space, CIELAB having been used for simplicity). Navigation in this color space can be done directly via the Lab values variables.

[0122] To be more concrete, this 3D space is translated into expert notions in the field of hair color, by introducing “semantic” orientations in this colorimetric space, such as different dimensions such as:

[0123] Color shade ranging from darkest to lightest,

[0124] Coppery ranging from most coppery to least coppery,

[0125] Gold ranging from most golden to least golden.

[0126] We can then move along these dimensional axes of semantic orientations, instead of using Lab values, which is more intuitive for the end user.

[0127] [Fig.6] represents a user interface offering examples of expert notions in connection with the 3D space represented in [Fig.3].

[0128] In [Fig.7], a photo 32 is shown representing a head of hair 3 with a new shade constituting a real target color 31, this photo can for example be a selfie or an image captured on the internet. The photo could be replaced by a text description. In the example shown, the hair is wavy.

[0129] An expert searches in the real color chart 7, a real color 310 which visually corresponds best to the color 31 of the target represented by the hair 3.

[0130] In [Fig.8], a color approximately close to the color 31 of the target 3 is displayed on the screen, then the expert refines the color until obtaining the desired and visually satisfactory shade closest to the actual target color 31, by modulating the expert notions 5, until the desired color 310.

[0131] [Fig.9] illustrates the method according to the invention by a block diagram.

[0132] To calibrate the colors in the virtual world 2, a real hair sample 3 is provided to which a real target color 31 has been applied, in the real world 1 shown on the right of [Fig. 9]. This real sample 3 is placed under predetermined lighting and spatial placement conditions. In the virtual world 2 shown on the left of [Fig. 9], a display unit 4 is provided for displaying a plurality of virtual reference images 21 of hair colored by a color digital reference 22. These virtual reference images 21 are 3D renderings. Unit 4 initially presents virtual digital reference parameters 5 in relation to each rendering 21.

[0133] An expert selects, from among the plurality of virtual reference images 21, the virtual reference image 210 having the digital reference color 220 which is visually closest to the actual target color 31 and which constitutes the selected reference rendering 220.

[0134] The selected reference rendering 220 undergoes a transformation 8 comprising the determination 9 of a 3D model from the selected rendering 200 and an interpolation 10 of the 3D model, in order to determine the virtual digital parameters 50 associated with the selected reference rendering 220.

[0135] The invention is not limited to the illustrated embodiments. Other presentations of the display are notably part of the present invention. Those skilled in the art will understand that the previously described embodiments can be combined to form new embodiments provided that they are technically compatible.

Claims

Claims

1. A method comprising: generating one or more user-selectable virtual menus on a display (4), the user-selectable virtual menus comprising one or more of: a viewpoint or perspective selection menu; a body area, especially hair, color library selection menu; a window selection menu; a tool selection menu; a color lock selection menu; a quick save selection menu;and interactively updating one or more perspective, tint, tone or shade values of a three-dimensional virtual rendering (22) of a body area, including hair, by applying a transformation (8) configured to convert or map user-selectable menu entries (5) into a set of parameters of a color space, including CIELAB, which, when applied to the three-dimensional virtual rendering (22) of the body area, including hair, results in a change in one or more perspective, tint, tone or shade values of the virtual rendering of the body area, including hair.;

2. Method according to claim 1, characterized in that the interactive update comprises an expert comparison between the three-dimensional virtual rendering (22) of the body area, in particular the hair and the real body area (3), in particular real colored, natural or synthetic hair, up to a visually identical color detection (220) between the three-dimensional virtual rendering (210) of the body area, in particular the hair and the real body area (3), in particular the real colored hair, in order to calibrate the color display.

3. Method according to any one of the preceding claims, characterized in that the update is carried out by an interface comprising levers for varying the colors (22) of the virtual rendering (21).

4. Method according to any one of the preceding claims, characterized in that the three-dimensional virtual rendering of the area body (21), especially hair, is obtained by "path tracing", "ray tracing", "inverse rendering", generative AI or diffusion AI.

5. Method according to any one of the preceding claims, characterized in that the update is carried out by levers of a color space, in particular CIELAB or by intrinsic levers of the three-dimensional virtual rendering (21).

6. Method according to any one of the preceding claims, characterized in that the transformation comprises a construction of a 3D model in the color space, in particular CIELAB and an interpolation from each point of the 3D model.

7. Method according to the preceding claim, characterized in that the interpolation is a 3D interpolation.

8. Method according to the preceding claim, characterized in that a numerical value is associated with each point of the 3D model, in particular an intrinsic value of an expert language, of a rendering engine, of colon values or others.

9. Method according to the preceding claim, characterized in that a selection menu is associated with a hair flexibility, a hair type, a hair quality, hair dimensions, a skin quality, a skin thickness or a skin type.

10. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method of any preceding claim.

11. A computer program product comprising instructions which, when executed by a computer, cause the computer to: Display on an interactive interface (4) one or more user-selectable virtual menus, the user-selectable virtual menus comprising one or more: a viewpoint or perspective selection menu; a body area, including hair, color library selection menu; a window selection menu; a tool selection menu; a color lock selection menu; a quick save selection menu; and Receive user input to interactively update one or more perspective, tint, tone, or shade values of a three-dimensional virtual rendering of a body area, including hair; Processing the user input by applying a transformation (8) configured to convert or map user-selectable menu entries into a set of parameters of a color space, including CIELAB, which, when applied to the three-dimensional virtual rendering (21) of a body area, including hair, results in a modification of one or more perspective, hue, tone, or shade values of the virtual rendering of the body area, including hair.

12. Data processing system comprising: A memory in which are stored user-selectable virtual menus comprising one or more instances of A menu for selecting a viewpoint or perspective; A menu for selecting a library of colors of a body area, in particular hair; a window selection menu; a tool selection menu; a color locking selection menu; a quick save selection menu; a three-dimensional virtual rendering of hair (21); A processing unit configured to implement the computer program product according to the preceding claim, A screen configured to display the selectable virtual menus and the three-dimensional virtual rendering of the hair (21).

13. System according to the preceding claim, characterized in that it comprises a color calibration unit by an expert comparison between the three-dimensional virtual rendering of hair (21) and real colored hair (31), natural or synthetic, until an identical color detection (220) between the three-dimensional virtual rendering of hair and the real colored hair.

14. Use of the system according to any one of claims 12 or 13, for predicting the result of a coloring from a three-dimensional virtual rendering of hair (21).

15. Use of the system according to any one of claims 12 or 13, for recommending a coloring on a body area (3).

16. Use of the system according to any one of claims 12 or 13, to interactively align with a person on a co- desired color and, possibly, combine a cosmetic composition with the desired color.

17. Use of the system according to any one of claims 12 or 13, for virtual fitting in photo or video.

18. Use of the system according to any one of claims 12 or 13, to display a virtual half-head section and to directly compare the result of two colorings.

Images