Dynamic update of automatic makeup treatment

Abstract

A dynamically updated automated makeup application system and method. The system includes a machine having an airbrush mounted on a robotic arm movable with five degrees of freedom, the airbrush mounted on the robotic arm. The system uses the machine to perform a makeup application plan on a recipient to achieve a desired appearance for the recipient or to apply corrective makeup. The system is configured to automatically and dynamically update the makeup application plan during the implementation of the makeup application plan in response to identifying movements of the recipient based on sensor readings. The airbrush is attachable to a number of alternative nozzles to achieve different air pressure levels and different materials according to the makeup application plan. The system designs and fabricates a four-dimensional (4D) stencil that is applied to the recipient while applying makeup according to the makeup application plan.

Description

[Technical field]

[0001] Cross-reference to related applications

[0001] This application claims the benefit of Provisional Patent Application No. 63 / 325,322, filed March 20, 2022, and entitled "AUTOMATIC MAKEUP APPLICATION," the entire contents of which are incorporated herein by reference without undue disclaimer. Technical Field

[0002] FIELD OF THE DISCLOSURE This disclosure relates generally to automated application of materials on fine line features and sharp edges, and more particularly to dynamic updating of automated makeup applications. [Background technology]

[0002]

[0003] Applying makeup is a difficult task. Traditional methods of applying makeup tend to be messy, imprecise, time-consuming, expensive, and unhygienic. For example, applying makeup manually can consume several hours per week. Also, makeup and brushes are often not properly cleaned after each use, which can lead to unhygienic conditions. Applying makeup can also be very expensive, as multiple products from different makeup brands, colors, and styles are required to achieve a desired look. Furthermore, whether applied by an individual or professional, makeup requires skill, precision, and materials that are not always sufficient to accurately achieve a desired look. Summary of the Invention [Problem to be solved by the invention]

[0003]

[0004] There is a strong need for new products, methods, and techniques that help consumers save time and money and improve their hygiene when applying makeup. [Means for solving the problem]

[0004]

[0005] One representative embodiment of the disclosed subject matter is a method that includes the steps of obtaining a cosmetic treatment plan, the cosmetic treatment plan including instructions for an automated makeup applicator that, when implemented by the automated makeup applicator, are configured to apply a cosmetic material onto a recipient to achieve a desired appearance for the recipient, executing a portion of the cosmetic treatment plan with the automated makeup applicator, obtaining sensor readings from a sensor during said execution, the sensor configured to monitor movements of the recipient, updating the cosmetic treatment plan in response to identifying movements of the recipient based on the sensor readings to obtain a dynamically updated cosmetic treatment plan, and executing the dynamically updated cosmetic treatment plan, or a portion thereof, to achieve the desired appearance for the recipient, taking into account the movements of the recipient and the execution of the portion of the cosmetic treatment plan.

[0005]

[0006] Optionally, the cosmetic plan instructions include instructions to apply the cosmetic material from a predefined position in space a defined distance away from the facial surface, and updating the cosmetic plan includes modifying the instructions based on movement of the recipient to maintain the defined distance.

[0006]

[0007] Optionally, the instructions include movement instructions that yield a 3D trajectory to be followed by the automated cosmetic application tool to achieve a desired look, and the instructions include material application instructions, each material application instruction including an application location, a material to be applied, and attributes of the treatment to be performed by the automated cosmetic application tool.

[0007]

[0008] Optionally, the step of obtaining the cosmetic treatment plan includes the steps of obtaining a three-dimensional (3D) surface of the recipient's face, obtaining a desired appearance, the desired appearance being determined based on user input indicating a cosmetic treatment result desired on the recipient's face, and developing a cosmetic treatment plan based on the user input and further based on the 3D surface of the face.

[0008]

[0009] Optionally, the planning step includes determining a material to be applied to a target area on the 3D surface of the face, application attributes for the application of the material, and an application distance and orientation at which the material is to be applied to the target area to achieve a desired appearance in the target area, and generating one or more instructions to an automated cosmetic application tool configured to apply the material using the application attributes from the application distance and orientation onto the target area.

[0009]

[0010] Optionally, the application attributes include an application pressure to use when applying the material onto the target area.

[0011] Optionally, the planning step is performed for a first target area and a second target area, wherein a coating distance in the first target area is different from a coating distance in the second target area.

[0010]

[0012] Optionally, the planning step is performed for a first target area and a second target area, the application attribute being a pressure used by the automated cosmetic application tool, an application distance in the first target area being equal to an application distance in the second target area, and the pressure used by the automated cosmetic application tool in the first target area being different from the pressure in the second target area.

[0011]

[0013] Optionally, the planning step is performed based on safety considerations, including a response time of the automated cosmetic application tool to movements of the recipient, thereby ensuring sufficient time to avoid injury to the recipient.

[0012]

[0014] Optionally, the updating step is performed to avoid injury to the recipient.

[0015] Optionally, the method further includes a step of simulating an outcome of the process of applying a cosmetic treatment plan on a recipient, the simulating step including a step of simulating execution of instructions of the cosmetic treatment plan on a 3D model of the recipient, thereby obtaining a simulation result depicting the recipient wearing makeup in accordance with the cosmetic treatment plan, the step of simulating execution of the instructions including a step of simulating a first application of makeup on a target area of ​​the recipient and a step of simulating a second application of makeup on the target area, and a step of displaying the simulation result.

[0013]

[0016] Optionally, the simulating step includes generating an intermediate simulation result depicting a recipient made up according to partial application of a cosmetic treatment plan, and the method includes displaying the intermediate simulation result.

[0014]

[0017] Optionally, the cosmetic treatment plan includes instructions for creating a four-dimensional (4D) stencil configured to be applied to the recipient, and the method further includes a step of creating the 4D stencil, and the performing step is performed while the 4D stencil is being applied to the recipient.

[0015]

[0018] Optionally, a 4D stencil is created based on a 3D model of the recipient.

[0019] Optionally, the automated makeup application tool comprises an airbrush movable with five degrees of freedom, the airbrush being capable of translational movement along three axes and rotational movement about two axes.

[0016]

[0020] Optionally, the airbrush has multiple nozzles with varying sizes and shapes.

[0021] Optionally, the cosmetic application plan defines a first application trajectory for the first nozzle and a second application trajectory for the second nozzle, the cosmetic application plan defining a relative application order between the first nozzle and the second nozzle.

[0017]

[0022] Another exemplary embodiment of the disclosed subject matter is a machine that includes a robotic arm movable with five degrees of freedom, the five degrees of freedom including translational movement along three axes and rotational movement about two axes, an airbrush mounted on the robotic arm, a sensor for monitoring movement of a recipient, and a control unit for controlling movement of the robotic arm in accordance with a cosmetic treatment plan, the control unit further configured to control application of the airbrush in accordance with the cosmetic treatment plan, the control unit configured to modify the cosmetic treatment plan based on sensor readings from the sensor.

[0018]

[0023] Optionally, the machine further comprises an air compressor, said air compressor configured to effect application of material via said airbrush, and said control unit configured to instruct said air compressor to supply different air pressure levels according to a cosmetic treatment regimen.

[0019]

[0024] Optionally, said machine further comprises a material mixer for supplying materials to be applied by said airbrush, the makeup application plan defining different materials to be mixed to apply the makeup on the recipient.

[0020]

[0025] Optionally, the airbrush is attachable to a number of alternative nozzles having different sizes and shapes, thereby allowing different application patterns with the airbrush.

[0021]

[0026] Optionally, the machine is configured to automatically attach and detach the nozzle from the airbrush.

[0027] Optionally, the machine is coupled to a stencil making device for making a four-dimensional (4D) stencil, the 4D stencil being configured to be applied to a recipient during application of the makeup according to a makeup application plan.

[0022]

[0028] Optionally, the aforementioned machine further comprises a chin rest and a forehead rest.

[0029] Optionally, said machine further comprises a proximity sensor that monitors the distance of said airbrush from a physical object.

[0023]

[0030] Yet another exemplary embodiment of the disclosed subject matter is a computerized device having a processor configured to perform the steps of: obtaining a cosmetic treatment plan, the cosmetic treatment plan including instructions for an automated cosmetic application tool that, when executed by the automated cosmetic application tool, are configured to apply a cosmetic material onto a recipient to achieve a desired appearance for the recipient; executing a portion of the cosmetic treatment plan with the automated cosmetic application tool; obtaining sensor readings from a sensor during said execution, the sensor being configured to monitor movements of the recipient; updating the cosmetic treatment plan in response to identifying movements of the recipient based on the sensor readings, thereby obtaining a dynamically updated cosmetic treatment plan; and taking into account the movements of the recipient and the execution of the portion of the cosmetic treatment plan while executing the dynamically updated cosmetic treatment plan or the portion thereof to achieve the desired appearance for the recipient. [Brief description of the drawings]

[0024] The subject matter disclosed herein will be better understood and appreciated by considering the following detailed description in conjunction with the drawings, in which corresponding or like numbers or letters indicate corresponding or like components. Unless otherwise indicated, the drawings depict exemplary embodiments or aspects of the present disclosure and are not intended to limit the scope of the disclosure. In the drawings: [Figure 1A]1 shows a schematic diagram of an exemplary machine according to certain exemplary embodiments of the disclosed subject matter. [Figure 1B] 1 shows a schematic diagram of an exemplary machine according to certain exemplary embodiments of the disclosed subject matter. [Figure 1C] 1 shows a schematic diagram of an exemplary machine according to certain exemplary embodiments of the disclosed subject matter. [Figure 2A] 1 illustrates a flow diagram of a method according to certain exemplary embodiments of the disclosed subject matter. [Figure 2B] 1 illustrates a flow diagram of a method according to certain exemplary embodiments of the disclosed subject matter. [Figure 3A] 1 illustrates a flow diagram of a method according to certain exemplary embodiments of the disclosed subject matter. [Figure 3B] 1 illustrates a flow diagram of a method according to certain exemplary embodiments of the disclosed subject matter. [Figure 4A] 1 shows a schematic diagram of an exemplary architecture according to certain exemplary embodiments of the disclosed subject matter. [Figure 4B] 1 shows a schematic diagram of an exemplary architecture according to certain exemplary embodiments of the disclosed subject matter. [Figure 4C] 1 shows a schematic diagram of an exemplary architecture according to certain exemplary embodiments of the disclosed subject matter. [Figure 4D] 1 shows a schematic diagram of an exemplary architecture according to certain exemplary embodiments of the disclosed subject matter. [Figure 4E] 1 shows a schematic diagram of an exemplary architecture according to certain exemplary embodiments of the disclosed subject matter. [Diagram 5] 1 shows a schematic diagram of an exemplary simulation according to certain exemplary embodiments of the disclosed subject matter. [Figure 6] 1 shows a block diagram of an apparatus according to certain exemplary embodiments of the disclosed subject matter. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025]

[0008] One technical problem addressed by the disclosed subject matter is to enable efficient automated makeup application for both professional and personal use. Application of makeup, particularly traditional or manual makeup application, can be time consuming and requires learning, skill, and technique. Furthermore, application of makeup can be unhygienic, because the tools utilized, i.e., tools such as brushes and sponges, even if designated for personal use, can accumulate bacteria, especially when the same tools are used for different users.

[0026]

[0009] On the other hand, existing automatic or semi-automatic makeup tools or devices are developed to simplify the process and provide accurate and consistent makeup application, so they are very basic, lack many functions, are less precise, less safe, and expensive. In some cases, they may cause confusion in the application of makeup or may cause injury to the person being applied, leading to suboptimal or even worse results.

[0027]

[0010] In one exemplary embodiment, airbrushing can be used for the application of materials such as paints on the face. Airbrushing can be said to be the free manipulation of the airbrush, medium, air pressure, distance, etc., from the surface to be sprayed, with or without the use of a shield or stencil, to consistently produce a specific and predictable result. Airbrushing can be used in fine art and illustration, pre-digital photo retouching, painting murals, makeup application, mehndi, nail art, clothing industry, automotive industry, etc. Airbrush makeup can be makeup sprayed on the skin from a relative distance using an airbrush machine, instead of being applied by direct contact, as when using a sponge, brush, finger, etc. Some airbrush systems use a compressor to create an air stream that passes through a hose connected to a trigger-controlled spray paint gun. The airbrush pressure can be adjusted to apply various types of makeup, such as lighter, darker, or more detailed styles. Airbrush systems can be used for both professional applications and personal home use. For home use, it can be utilised by miniaturised airbrush systems designed to operate at lower pressures than those used in professional applications.

[0028]

[0011] In one exemplary embodiment, airbrush techniques can also be utilized for makeup application because they are more hygienic, last longer, and look more natural. However, self-application of airbrush makeup can be a very complex and challenging multitasking procedure. The airbrush tool must be held by the hand while other organs must be manipulated and coordinated to operate it. The hand and arm may have to move along a trajectory in 3D space to change the position and orientation of the airbrush. The index finger may have to control the flow of makeup by pulling the airbrush trigger. Both eyes may have to continuously track the moving airbrush and the makeup being sprayed. All of these tasks require high precision, which can be even more difficult during the application of makeup to sensitive areas, such as when applying eyeshadow, because one eye must be completely closed during the application. Also, as another example, application of makeup to the under-eye area can be even more difficult due to the risk of spraying the makeup into the eye. As a result, airbrush makeup is most often best applied by professional makeup artists who are specially trained in airbrushing techniques and is not feasible for personal use.

[0029]

[0012] Another technical problem addressed by the disclosed subject matter is enabling precise application of cosmetic materials to achieve a desired appearance. In one exemplary embodiment, automated application of materials on non-stationary surfaces in general, and specifically on a person's body or face using airbrush techniques, can be difficult due to the difficulty of precisely adjusting the application means, the difficulty of keeping the user stationary and motionless, etc. There is a need for a cosmetic application method that takes into account the movement of the recipient during the application process to achieve a desired appearance for the recipient.

[0030]

[0013] Additionally or alternatively, the application of makeup with an airbrush is believed to produce smooth results. This type of application may be appropriate for makeup when a natural, smooth airbrushed result is desired. However, some makeup applications may require more precise application with sharp, defined edges or thin lines, such as eyeliner or lip liner. In one exemplary embodiment, when an airbrush artist manually applies airbrush materials or paints, such thin lines can often be drawn by removing the airbrush cap (i.e., exposing the airbrush needle) or by spraying the material with the needle almost in contact with the paper or canvas. Such techniques may not be practical for automated makeup applications, because it may be dangerous to bring the needle so close to the user's face, much less the user's eyes. On the other hand, when an airbrush artist manually applies makeup using an airbrush, a stencil may be utilized. The stencils may be general or generic ready-made stencils produced by airbrush makeup manufacturers, or may be generic stencils included in any airbrush makeup starter kit for drawing fine shapes. There may be a set of standard generic stencils for each treatment, such as eyeliner, brow clarification, lip clarification, etc. As an example, there may be several stencils with different brow shapes, or several different widths and shapes for an eyeliner. Such stencils are very difficult to use on oneself. They rarely fit the user perfectly. Such stencils may be made of plastic or other similar hard materials, may not be flexible, and may not be easily attached to the user's face. As an example, the shape of an eyeliner stencil may not adapt to the shape of the user's eyes.As a result, a lot of work is required to get these stencils to line up perfectly during the airbrush application, and the makeup artist must move and adjust the stencil until the desired result is achieved.

[0031]

[0014] Yet another technical problem addressed by the disclosed subject matter is to enable high-precision corrective makeup that is adapted to different users in mixed-different situations. One of the most challenging aspects of being a makeup artist is to create illusions or camouflage using makeup to correct imperfections, especially in the facial area, to enable the attainment of the exact desired appearance. Such imperfections may include facial dissimilarity, skin imperfections, pigmentation, scars, inappropriate size of certain organs, or any other facial defects such as acne, bums, vitiligo, rosacea, age spots, birthmarks, dark circles around the eyes, etc. Corrective makeup may be a technique that utilizes light and dark shades and colors to highlight, as well as contour features to create effects of harmony and balance, etc. Dark shades can be seen farther away, and the lighter the shade, the closer it can be seen. When dark and light shades are used next to each other in the right places, the illusion of shape can be created. As an example, a highlight or light shade can accentuate a feature, while a dark shade outline can make the feature appear less prominent. As another example, a light color can make an object appear forward, while a dark color can bring it back to the foreground. As yet another example, a precise outline can provide precise shadows and help define a particular area. Corrective makeup requires precise identification of areas on a person's face that need to be weakened or enhanced in appearance by makeup. It is not surprising that the identification process is difficult to perform without training, skill and expertise, and without the use of correct measuring equipment. As an example, corrective makeup may be required in certain situations that require precise measurements of the face, such as when the nose and forehead are at different heights, or when the eyes are separated by a distance greater than the width of one eye. However, it is difficult to identify these features and perform the correction precisely with makeup.Aside from being mostly inaccurate, manual corrective makeup naturally requires practice and patience, both in choosing the correct combination of colors and materials, and in the accurate application of such materials.

[0032]

[0015] One technical solution is an automated airbrush makeup application system that allows for automatic application of makeup based on user input to achieve a desired makeup look, and can dynamically update the makeup look during the application process. In one exemplary embodiment, the desired look can be obtained based on direct user input, such as a photo of the desired look, a selection of a look from a catalog, etc. Additionally or alternatively, the desired look can be generated manually or automatically based on user input according to the recipient's face. A user can utilize a graphical user interface (GUI) that shows a simulation of the recipient's face to indicate the area to which the makeup is to be applied, the makeup color, style, brush type, application technique, etc. The user can start from scratch or from an initial state that can be automatically determined.

[0033]

[0016] In an exemplary embodiment, a visual input may be obtained that captures a surface to which the cosmetic material is to be applied, such as a face of a recipient, a neck of a recipient, a chest of a recipient, etc. The visual input may be obtained from a visual sensor that monitors the recipient in real time, such as a camera, a scanner, a range imaging sensor, etc. Additionally or alternatively, the visual input may be a photograph of the recipient in an initial state, from which attributes of the recipient may be extracted. In an exemplary embodiment, the visual input may be analyzed to determine attributes of the treatment surface, such as structure (e.g., facial structure, object, or organ structure, etc.), surface color (e.g., skin color of a user, color of a background, etc.), surface texture, etc. Additionally or alternatively, the visual input may be analyzed to determine the location or positioning of surface features, such as coordinates of points of interest, borders, precise location of facial features, dimensions, reference points, etc. As an example, eyes, nose, eyebrows, lips, etc. may be identified. In an exemplary embodiment, the analysis may be performed using image analysis techniques, machine learning, geometric analysis, etc.

[0034]

[0017] In an exemplary embodiment, the system can be configured to determine an optimal path (also called a trajectory) for application of material, such as an airbrush application path, a makeup application process path, etc. In an exemplary embodiment, the system can be configured to calculate a trajectory in 3D space that emulates the movements of a human expert when manually applying material to a surface to achieve a desired result. As an example, the system can be configured to calculate a trajectory in 3D space that emulates the hand movements of an airbrush makeup artist to achieve a selected makeup look. Each trajectory can comprise a collection of oriented 3D points that represent the position and orientation of a makeup application tool in 3D space.

[0035] In one exemplary embodiment, an automated machine can be configured to apply a material onto a surface according to a calculated path. The automated machine can be a computer numerically controlled (CNC) machine, an airbrush machine, a machine with automated airbrush equipment, etc. One or more automated machines can also be configured to follow the calculated path and trajectory and apply (e.g., spray) a material (e.g., cosmetic product) onto a surface (e.g., a user's face).

[0036] Additionally or alternatively, the automated machine may be a robotic system designed for automated makeup application. The automated machine may be comprised of a robotic arm capable of moving in exactly five degrees of freedom, including translational movements along three axes and rotational movements around two axes. It is noted that rotational movements around a third axis may also be realized in some embodiments. In some exemplary embodiments, one or more airbrushes may be mounted on the robotic arm and used to apply makeup to a recipient. The automated machine may also include sensors to monitor the movements of the recipient and a control unit to control the movements of the robotic arm and the application of the airbrush according to a makeup application plan.

[0037]

[0020] In one exemplary embodiment, the control unit may modify the cosmetic application plan based on sensor readings from the sensor, thereby allowing for adjustments to the cosmetic application in real time. Additionally or alternatively, the automated machine may include a material mixer that provides different materials to be applied by the airbrush, allowing for application of a wide range of cosmetic products. The airbrush may also be attachable to multiple alternative nozzles of different sizes and shapes to achieve different application patterns.

[0038]

[0021] In one exemplary embodiment, the system can be configured to calculate and instruct the machine to generate a combination of materials to be used along each trajectory to provide a customized color or style to be applied (e.g., sprayed, printed, etc.) onto the surface. The customized color or style can be calculated based on the desired result (e.g., the selected cosmetic look), the background color of the surface (e.g., the skin tone of the user), etc. The customized color or style can be obtained by mixing materials (e.g., basic cosmetic shades) stored in designated reservoirs associated with the machine. In one exemplary embodiment, the materials can be mixed according to the Cyan, Magenta, Yellow, and Key (Black) (CMYK) color model and the White color model. The system can be configured to generate any shade by mixing the CMYK basic shades, allowing the desired look to be reached with a minimum amount and variety of materials. In this way, a limited variety of cosmetics and colors can be used to generate any desired look, thereby providing a foundational effect for cosmetic applications. It should be noted that the disclosed subject matter is not limited to any particular color model, and other color models may certainly be applicable.

[0039] In one exemplary embodiment, the system may be configured to apply material in layers to achieve a desired result (e.g., a selected cosmetic look). Each layer may be applied separately, using a separately calculated trajectory. Each layer may have a different composition of material applied.

[0040]

[0023] Another technical solution is to dynamically update the cosmetic application plan based on sensor information monitoring the movement of the recipient during application of the cosmetic material. The cosmetic application plan can be dynamically updated in response to identifying the movement of the recipient that may affect the cosmetic application process. In a representative embodiment, the system can be configured to continuously track the movement of the user, i.e., the recipient, during application of the cosmetic material and dynamically adjust the calculated trajectory based on such movement. The system can also be configured to dynamically stop or pause the cosmetic process, adjust automated cosmetic application tools, etc. based on the movement of the recipient. The movement of the recipient can be tracked using motion sensors, visual sensors, range of motion (ROM) sensors, etc.

[0041]

[0024] Yet another technical solution is to dynamically update the cosmetic treatment plan based on user input for real-time simulation of the results of the cosmetic process based on the treatment plan.

[0042]

[0025] In one exemplary embodiment, the system can be configured to provide a step-by-step augmented reality (AR) or virtual reality (VR) preview of a layer-by-layer application on a surface, such as a step-by-step AR or VR preview of a selected cosmetic look on a 3D model of a user's face, using a graphical simulation based on the calculated trajectory, shade, and style. The graphical simulation can simulate an actual cosmetic application based on the actual trajectory and movement of both the user and the application machine.

[0043]

[0026] Yet another technical solution is to use four-dimensional (4D) printing technology to produce custom-designed self-folded 3D stencils that can be automatically personalized and adapted to the user's face shape, the desired results, etc. Custom-designed self-folded 3D stencils can be placed in precise positions (automatically by machine or manually by user) and utilized to draw thin lines and sharp edges by airbrushing. This method defines an alternative to traditional 2D generic stencils.

[0044] In one exemplary embodiment, a custom-designed, auto-folding 3D stencil can be created that is custom-fitted to the user's face based on an analysis of visual input from a scan of the user's face.

[0045] In one exemplary embodiment, the custom-designed self-folding 3D stencil can be a 2D shape that can transform into different forms in response to environmental stimuli, with the fourth dimension being a time-dependent shape change after printing. The custom-designed self-folding stencil can be created using programmable 4D printing, and after the fabrication process, the printed stencil responds to parameters in the environment (humidity, temperature, voltage, etc.) and changes its form accordingly. The self-folding stencil can perfectly conform to the user's face, allowing for a much less challenging application of the material.

[0046]

[0029] In a representative embodiment, 4D printing can be configured to encode self-actuating deformation during the printing process so that an object can be created flat and then transformed into a target 3D shape. 4D printing can include printing a 3D structure, or can include printing a 2D structure that can assume a 3D form, such as by folding a 2D surface. The fourth dimension involved in 4D printing can be the time dimension, since a 3D object (e.g., a 3D printed object, a 2D printed surface that is folded into a 3D object, etc.) can change its form over time. The change in form can be caused by heat, contact with other materials, etc. As an example, Fused Deposition Modeling (FDM) printing techniques can be used to injection mold molten thermoplastics through a narrow nozzle, which stretches the material in the printing direction. The pre-stretching causes the material to shrink in the printing direction during heating. In addition to the shrink direction, the amount of shrink can also be controlled by the printing thickness of each layer. As another example, a printed 2D flatsheet, such as a personalized stencil, can be uniformly heated in a hot water bath at a high temperature, such as about 90°C, and self-deformed into a target 3D surface. As yet another example, a printed stencil (2D or 3D) can be automatically deformed by being brought close to a user's face or by contacting a base material applied to the face using an airbrush machine. In one exemplary embodiment, a barcode can be printed on the 2D stencil to aid in positioning the stencil using a camera and computer vision algorithms.

[0047]

[0030] Yet another technical solution is to dynamically perform corrective makeup using automatic airbrushing. In a representative embodiment, the system can be configured to automatically detect facial asymmetry, pigmentation, or any other deficiencies in the face. The system can be configured to automatically determine a correction process and apply it to the user's face using airbrushing techniques. The correction process can include the use of multiple 3D trajectories and paths, the use of different types of materials, etc. The correction process can be applied to the user's face using cosmetic procedures and techniques, for example, the automatic application of a technique that uses the golden rule to harmonize facial features. Corrective contouring can be used to further balance the face, to increase the symmetry of features, or to change features or facial shapes altogether, such as minimizing the size of the forehead by adding a darker shade around the edge of the forehead when the size of the forehead exceeds a certain threshold with respect to the overall size of the face, or to make the nose appear thinner or shorter when the length of the nose exceeds a certain threshold, or to hide a double chin when it is detected. When two areas should appear to be of equal length but are not, a darker shade can be used to soften part of one area, thus creating the illusion that this area is shorter, while a lighter shade can be used to highlight the equal length parts.

[0048]

[0031] It should be noted that the above solutions, products, methods, and systems have been described with respect to airbrush makeup application. However, each of these technical solutions, products, methods, and systems may also be adapted and applied to other makeup application techniques, such as 3D printing applications, sponge-based makeup tools, etc. Additionally or alternatively, each of these technical solutions, products, methods, and systems may also be adapted and applied to other uses of airbrush techniques, such as in fine art applications, painting, drawing, temporary tattoos, nail art, clothing industry, automotive industry, etc.

[0049]

[0032] One technical effect of utilizing the disclosed subject matter is to provide efficient, hygienic, and precise application of makeup materials to achieve a desired look while overcoming the challenges of manual airbrush makeup. The disclosed automated application of airbrush makeup overcomes the challenges of existing makeup techniques, both automated and manual, and provides precise, fast, and hygienic application of makeup. The disclosed automated application of airbrush makeup can utilize freehand techniques to manipulate aspects such as distance and air pressure while applying makeup to create specific effects and coverage. Additionally, the disclosed subject matter provides a stencil making device that creates 4D stencils. The 4D stencils can be attached to the recipient's face during makeup application, addressing customization, personalization, and flexibility in the makeup application process while allowing for a highly automated solution to apply precise and precise makeup.

[0050] Another technical effect of utilizing the disclosed subject matter is the elimination of human intervention in the selection, application, application, and cleanup of makeup while performing these tasks accurately, cost-effectively, and under acceptably sanitary conditions. The disclosed automated application of airbrush makeup can be particularly useful to those who wish to achieve a flawless, long-lasting application of makeup without having to spend a long time or a lot of effort in the process.

[0051]

[0034] Yet another technical effect of utilizing the disclosed subject matter is to allow free movement of the recipient during automated makeup application without exposing the recipient to danger. The disclosed subject matter utilizes airbrush techniques to allow makeup to be applied automatically to obtain fine lines and precise shapes without requiring a means to keep the user motionless. Additionally or alternatively, a chin rest or forehead rest may be utilized to stabilize the recipient during the makeup application process without restricting the recipient's movement or requiring the recipient to sit in a specific position during the makeup application process.

[0052]

[0035] Yet another technical effect of utilizing the disclosed subject matter is to provide an auto-correcting makeup application tool that uses airbrush technology to aid in blending and application with five degrees of freedom of movement without the use of manual tools or specialized knowledge.

[0053]

[0036] Yet another technical effect utilizing the disclosed subject matter is to assist individuals with disabilities who have limited or impaired ability to apply makeup themselves. A voice interface can be used to operate and communicate with the machine, which can also help make the makeup process more inclusive for individuals with disabilities.

[0054]

[0037] The disclosed subject matter provides one or more technical improvements over any existing techniques and over any techniques that are already routine or customary in the art.

[0055]

[0038] Still other technical problems, solutions, and advantages will become apparent to those skilled in the art in view of this disclosure.

[0039] Reference is now made to Figure 1A, which illustrates a schematic diagram of an exemplary machine according to certain exemplary embodiments of the disclosed subject matter.

[0056] In one exemplary embodiment, the machine 100 may be an automated makeup application tool that utilizes airbrush technology to apply makeup on surfaces, organs, bodies, etc., such as the face of the recipient 190. The machine 100 may be configured to use airbrush technology to provide a quick, easy, and accurate method of applying makeup with professional looking results.

[0057]

[0041] In a representative embodiment, the machine 100 may include a robot arm 110 that is movable with five degrees of freedom. The robot arm 110 may be designed to move in three axes of translation, e.g., linear motion, such as forward / backward (z-axis), up / down (y-axis), and left / right (x-axis) in a plane facing the face of the patient 190. The three-axis translation may be utilized in conjunction with the pitch and yaw rotation of the arm or wrist to emulate the hand movements of a makeup artist. The three-axis movement may be enabled by two-axis movement of the robot arm 110, e.g., forward / backward (z-axis), up / down (y-axis), and by a left / right (x-axis) movement of the body 102 that holds the robot arm 110. Additionally or alternatively, the robot arm 110 may be movable with four degrees of freedom. The robot arm 110 can also be designed to move in three axes of translation, e.g., linear motion, such as forward / backward (z-axis), up / down (y-axis), and left / right (x-axis) in a plane facing the face of the patient 190. The three-axis translation can be used in conjunction with a yaw rotation of the arm or wrist to approximate the hand movements of a cosmetician. The three-axis movement may be enabled by two-axis movement of the robot arm 110, e.g., forward / backward (z-axis), up / down (y-axis), and by a left / right (x-axis) movement of the body 102 holding the robot arm 110.

[0058] Additionally or alternatively, the robot arm 120 may include a wrist 130 having one or two joints. One joint may be configured to connect between two portions of the robot arm 120, a first portion configured to move forward / backward (z-axis) and a second portion configured to move up / down (y-axis) and left / right (x-axis). Additionally or alternatively, the robot arm 110 may be directly actuated by three-axis movement without relying on the movement of other components of the machine 100.

[0059]

[0043] In one exemplary embodiment, the airbrush 120 can be mounted to a robotic arm 110. The robotic arm 110 can be configured to allow rotational movement of the airbrush 120 about two axes. Additionally or alternatively, the airbrush 120 can be connected to the robotic arm 110 using a wrist 130 (such as using a second joint) to allow for its rotational movement. Additionally or alternatively, the robotic arm 110 can be directly actuated by five-axis movement without relying on the movement of other components of the machine 100.

[0060]

[0044] In an exemplary embodiment, the airbrush 120 may be configured to utilize compressed air to spray makeup onto the skin of the recipient 190. The airbrush 120 may utilize a refillable makeup reservoir. The airbrush 120 may be configured to spray makeup material onto the skin in a fine mist to create a uniform, natural-looking finish. In an exemplary embodiment, the airbrush 120 may be associated with an air compressor configured to effect application of the material from the airbrush 120. The air compressor may be an integral component of the airbrush 120, may be connected to the airbrush 120, may be connected to other components of the machine 100, etc.

[0061]

[0045] In one exemplary embodiment, the airbrush 120 may be attachable to a number of alternative nozzles having different sizes and shapes. Different nozzles allow for different application patterns by the airbrush 120. The machine 100 may also be configured to automatically attach and detach nozzles to the airbrush 120. Additionally or alternatively, the airbrush 120 may be attachable to other types of attachments for application of different types of makeup, such as foundations, blushes, and highlighters. Having interchangeable attachments allows for variety in application.

[0062]

[0046] In an exemplary embodiment, the airbrush 120 may also be associated with a depth camera. The depth camera may be located at a center of the machine 100, such as near or within the sensor 140. The sensor 140 may be or may include a depth camera. Additionally or alternatively, the depth camera may be an integrated component of the airbrush 120, a separate sensor located elsewhere on the machine 100, etc. The depth camera may be configured to scan the face of the recipient 190 to maintain a predetermined distance from the face of the recipient 190 according to a cosmetic treatment plan.

[0063] Additionally or alternatively, the machine 100 may include other sensors in different locations, such as a sensor 145 located at the bottom right of the machine 100, a sensor on the wrist 130 (not shown), multiple sensors in non-fixed locations (not shown), etc. Similar to the sensor 140, the sensor 145 may include a depth camera. The combination of two or more sensors or depth cameras in different locations allows the machine 100 to address occlusion that can occur when one camera or sensor is hidden behind a second camera or sensor located in a different location where it cannot be hidden.

[0064] In one exemplary embodiment, the machine 100 can include a control unit 150 that controls the movement of the robotic arm 110 according to a cosmetic application plan. The control unit 150 can be configured to control the application of the airbrush 120 according to the cosmetic application plan, the movement of the wrist 130, etc. Additionally or alternatively, the control unit 150 can be configured to modify the cosmetic application plan based on sensor readings from the sensor 140.

[0065]

[0049] In a representative embodiment, the machine 100 may include a sensor 140 for monitoring the movement of the subject 190. The sensor 140 may be a visual sensor, a motion sensor, a combination thereof, etc. The sensor 140 may be designed to detect and measure the movement of an object, person, or animal within its range, specifically the movement of the subject 190, the movement of a particular organ, part, or point of the subject 190, such as the face, eyes, forehead, chest, neck, shoulders, etc.

[0066]

[0050] In one exemplary embodiment, the sensor 140 may comprise a camera or other optical device configured to continuously capture and analyze images of the recipient 190 to detect motion. The analysis may include computer vision and image analysis techniques. Additionally or alternatively, the sensor 140 may comprise other types of motion sensors. As an example, the sensor 140 may comprise a passive infrared (PIR) sensor configured to detect changes in infrared radiation caused by the movement of the recipient 190. As another example, the sensor 140 may comprise an ultrasonic sensor configured to emit high frequency sound waves that reflect off the surface of the recipient 190 and detect motion based on the return time. As yet another example, the sensor 140 may comprise a microwave sensor configured to emit microwave signals and measure the reflection of these signals from nearby objects.

[0067] In one exemplary embodiment, a proximity sensor (not shown) may be attached to the cap of the airbrush 120 to monitor when the airbrush gets too close to an object, particularly the face of the recipient 190. Additionally or alternatively, other sensors external to the machine 100, such as sensors within the device 195, such as cameras, microphones, etc., may be utilized as inputs and sources of signals to the machine 100. As an example, in one embodiment of the machine 100, the device 195 may be mounted at a designated location within the machine 100, and the camera of the device 195 may be utilized to track the user 190 in real time. Additionally or alternatively, the microphone of the device 195 may be utilized to communicate audio or verbal communication between the recipient 190 and the machine 100.

[0068] Additionally or alternatively, the control unit 150 may be configured to adjust set points to control the amount of cosmetic applied by the airbrush 120 and the pressure of the air provided by the air compressor, thereby accommodating customizations based on a user's desired coverage and finish. The control unit 150 may be configured to instruct the air compressor to provide different air pressure levels according to a cosmetic application regimen.

[0069]

[0053] In a representative embodiment, the machine 100 can be coupled to a device 195 of the person 190 or other user to monitor or control the cosmetic application process, such as by using a Wi-Fi connection, Bluetooth connection, etc. The device 195 can have a display means, such as a screen of a computing device, or any user interface UI means, such as through a mobile device, a designated application, etc. The device 195 can be utilized to monitor or review the cosmetic application plan by simulation on a 3D virtual model of the face of the person 190. The device 195 can be utilized to display a preview of the result of spraying each calculated shade by tracing its corresponding trajectory on the virtual 3D model of the face of the person 190, such as by using a graphical simulation, by video, etc.

[0070] Additionally or alternatively, the device 195 can be utilized to communicate with a user managing the makeup application process, such as a cosmetic professional. It can be possible for the cosmetic professional to manually modify the makeup application plan, trajectory instructions in a VR or remote setting, etc.

[0071]

[0055]

[0056] Additionally or alternatively, the patient 190 or any other responsible user may be enabled to use the device 195 to create, upload or select looks from a pre-defined catalogue, provide input to the machine 100, manually update a cosmetic treatment plan, etc.

[0072]

[0057] It should be noted that the person 190 or any other user controlling the cosmetic application process can use the device 195 to stop the cosmetic procedure at any time, such as by directly turning off the power to the machine 100 or by stopping / operating the movement of components of the machine 100.

[0073]

[0058] The machine 100 may also include an emergency stop button (not shown) that the patient 190 can press at any given time while the machine 100 is operating to stop its operation.

[0074]

[0059] Reference is now made to Figure 1B, which illustrates a schematic diagram of an exemplary machine according to certain exemplary embodiments of the disclosed subject matter.

[0060] In one exemplary embodiment, the machine 100 can include a color mixing system 160 (e.g., a color or material mixer) that supplies materials to be applied by the airbrush 120. The color mixing system 160 can be configured to dispense different materials inside the cup of the airbrush 120 to apply makeup to the recipient 190 according to a makeup application plan. The color mixing system 160 can be configured to precisely dispense CMYK or other color tones contained in designated reservoirs within the color mixing system 160. Back bubbling air entering the airbrush cap can then be used to mix the dispensed materials to obtain the corresponding shade on demand for each orbit.

[0075]

[0061] In one exemplary embodiment, the machine 100 can be configured to automatically refill the cup of the airbrush 120 with makeup material from the blending system 160 according to a makeup application plan. Makeup material specific to each area of ​​the face, such as a customized shade of makeup, can be transferred to the airbrush 120 via the body 102 and the robotic arm 110. Additionally or alternatively, the airbrush 120 can be designed to move toward the blending system 160 to fill with makeup material.

[0076]

[0062] In one exemplary embodiment, the machine 100 can be coupled to a stencil making device 105 to make a 4D stencil configured to be applied to the recipient 190 during the application of the makeup according to the makeup application plan. The stencil making device 105 can be integrated within the machine 100, can be separate from the machine 100, connected thereto by wires or wirelessly, etc.

[0077] In one exemplary embodiment, stencil fabricator 105 can be configured to operate offline and independent of machine 100. Stencil fabricator 105 can create and derive a stencil geometry for use with machine 100 at a later time. The stencil geometry can be generated according to a scan of the face of user 190 using device 195, using sensor data from machine 100, or the like.

[0078]

[0064] In one exemplary embodiment, the stencil 102a can be made according to a cosmetic treatment plan using a stencil making device 105. It should be noted that the shape of the stencil 102a before application to the face of the recipient 190 is flat, but may change shape, such as to the shape of stencil 102b, when applied to the face of the recipient 190. It should also be noted that more than one stencil can be applied to the face of the recipient 190 at the same time, such as stencil 102b over the eyebrows and stencil 103b around the mouth of the recipient 190. Additionally or alternatively, different stencils can be applied to the face of the recipient 190 and peeled off separately between application of different trajectories of the cosmetic treatment plan, between use of different cosmetic materials, etc.

[0079]

[0065] Reference is now made to Figure 1C, which illustrates a schematic diagram of an exemplary machine according to certain exemplary embodiments of the disclosed subject matter.

[0066] In one exemplary embodiment, the machine 100 can utilize a chin rest 180 and / or a forehead rest 185 to ensure that the face of the person 190 is still and in the correct position for proper makeup application. The chin rest 180 and the forehead rest 185 can be components of the machine 100, can be separate from the machine 100, can be attachable to the machine 100, etc. Additionally or alternatively, the chin rest 180 and the forehead rest 185 can be controlled by the control unit 150 and adjusted in response to the movement of the person 190 or according to a makeup application plan.

[0080]

[0067] In one exemplary embodiment, the chin rest 180 is a small platform that supports the chin of the person 190, while the forehead rest 185 provides support for the forehead of the person 190. The chin rest 180 and the forehead rest 185 can also be adjustable to accommodate different head sizes and shapes. The chin rest 180 and the forehead rest 185 can be padded to ensure comfort during the makeup application process. The person 190 can be asked to rest her chin on the chin rest 180 and / or her forehead on the forehead rest 185 to help stabilize the head and ensure that the face, especially the eyes and lips, are at the correct distance from the machine 100 or airbrush 120, etc.

[0081]

[0068] In one exemplary embodiment, the chin rest 180 may include a small groove or ridge designed to fit directly under the chin of the person 190. This serves to prevent the chin from sliding forward, and maintains the correct distance between the face of the person 190 and the relevant components of the machine 100. Additionally or alternatively, the chin rest 180 may include a small lip or edge that extends upward and presses against the underside of the chin of the person 190 without interrupting the makeup application process and without covering the face of the person 190. This provides additional support and helps prevent the face of the person 190 from moving forward during the automated makeup application.

[0082] Additionally or alternatively, the forehead rest 185 can be designed to help limit forward movement by providing a contact point against which the recipient 190 can press, helping to keep the head and face in place. The grooves or ridges on the chin rest 180, in combination with the forehead rest 185, can help limit forward movement of the recipient's 190 face while it is resting on the chin rest 180, keeping the face in the correct position to ensure proper makeup results.

[0083] Additionally or alternatively, additional supports such as earrests or side supports may be used to further stabilize the head and prevent movement during the cosmetic application process. The goal of these supports is to ensure that the recipient's head remains stationary during the cosmetic application process, or part thereof, to ensure accurate application.

[0084]

[0071] Reference is now made to Figure 2A, which illustrates a flow chart diagram of a method in accordance with certain exemplary embodiments of the disclosed subject matter.

[0072] In step 210, a cosmetic application plan can be obtained. In an exemplary embodiment, the cosmetic application plan can include instructions for an automated cosmetic application tool to apply cosmetic material to a recipient to achieve a desired appearance for the recipient. In an exemplary embodiment, the cosmetic application tool can be an automated airbrush cosmetic application device or a component thereof, such as the machine illustrated in Figures 1A-1C, 4A-4E, or the like.

[0085]

[0073] In an exemplary embodiment, the cosmetic treatment plan can be developed offline, dynamically updated, developed from scratch, etc. The cosmetic treatment plan can be obtained using one or more of the methods described in Figures 3A-3B, or portions thereof, as an example. Additionally or alternatively, the cosmetic treatment plan can be developed automatically, as described in U.S. Patent No. 20200285835A1, filed March 7, 2019, granted January 31, 2023, and entitled "Systems and methods for automated makeup application." The entire contents of this patent application are incorporated herein by reference without undue citation.

[0086]

[0074] In one exemplary embodiment, the cosmetic planning instructions may include instructions to apply cosmetic materials from a predefined location in space that is a defined distance away from the face surface. Additionally or alternatively, the instructions may include movement instructions that define a 3D trajectory to be followed by the automated cosmetic application tool to achieve the desired look. Each instruction may include material application instructions to be applied to each location in the 3D trajectory. The material application instructions may be configured to indicate application locations, materials to be applied, application attributes to be performed by the automated cosmetic application tool, etc.

[0087]

[0075] In one exemplary embodiment, the cosmetic application plan can be configured to define multiple application trajectories, each of which is configured to be applied separately by the same cosmetic application tool, with a relative ordering therebetween (e.g., one after the other, in layers, etc.), simultaneously by different cosmetic application tools, using different nozzles of an airbrush, by different components of the cosmetic application tool, with materials of different compositions, etc.

[0088]

[0076] At step 220, a portion of the cosmetic treatment plan may be performed by the automated cosmetic treatment tool. In an exemplary embodiment, the portion may include a predetermined number of instructions, such as 1 instruction, 2 instructions, 10 instructions, etc. Additionally or alternatively, the portion may be based on time, such as applying the portion for 1 millisecond, 50 milliseconds, 1 second, 10 seconds, etc. Additionally or alternatively, the portion may not be a predefined portion of the cosmetic treatment plan, but instead may be any portion of the cosmetic treatment plan that is performed until movement of the recipient is detected.

[0089]

[0077] In step 230, sensor readings can be obtained from sensors that monitor movements of the recipient during the implementation of the cosmetic application regimen. In an exemplary embodiment, one or more sensors can be configured to monitor movements of the recipient that may affect the cosmetic application process, such as head movements, shoulder movements, neck movements, movements of specific organs in the face, etc. These sensors can be motion sensors, visual sensors, etc.

[0090]

[0078] In step 240, the movement of the recipient can be identified based on the sensor readings. In one exemplary embodiment, the sensor readings can also include visual readings capturing the recipient. To detect the movement of the recipient, the sensor readings can be analyzed using computer vision techniques, visual analysis techniques, and the like. As an example, the movement of a particular organ or object can be continuously tracked to detect changes in the position, size, or shape of the object in successive frames of a video. As another example, the movement can be identified based on changes in pixel values ​​between successive frames. As yet another example, optical flow can be used to detect movement based on patterns of apparent movement of objects in images, track the movement of image features such as edges or corners, estimate the direction and speed of movement in a scene, and the like.

[0091]

[0079] In step 280, the cosmetic treatment plan can be dynamically updated based on the movement of the recipient to achieve a desired appearance for the recipient despite the movement of the recipient. In an exemplary embodiment, updating the cosmetic plan can include changing instructions based on the movement of the recipient. In an exemplary embodiment, the distance of the cosmetic treatment tool from the recipient's face can be changed to maintain a set distance. Additionally or alternatively, the overall trajectory (position of the cosmetic treatment tool as a function of time) can be updated based on the movement of the recipient. Additionally or alternatively, the relative time of reaching a particular position can be updated based on the movement of the recipient, such as delaying the arrival of the cosmetic treatment tool at a particular position, changing the timing of reaching different target areas, etc. Additionally or alternatively, the composition of the applied material can be updated.

[0092]

[0080] Additionally or alternatively, the machine can be commanded to stop moving and the airbrush can be commanded to stop spraying, such as by turning off the air compressor or releasing the airbrush trigger, or both.

[0093] Additionally or alternatively, the cosmetic treatment plan may be dynamically updated based on the recipient's movements to avoid injury to the recipient, for example to prevent getting too close to the facial surface, particularly in sensitive areas such as around the eyes.

[0094]

[0082] In step 290, a dynamically updated cosmetic treatment plan, or a portion thereof, may be implemented to achieve the desired appearance, taking into account the recipient's movements and the implementation of that portion of the cosmetic treatment plan.

[0095]

[0083] Reference is now made to Figure 2B, which illustrates a flow chart diagram of a method in accordance with certain exemplary embodiments of the disclosed subject matter.

[0084] In step 210b, a cosmetic treatment plan can be obtained. In one exemplary embodiment, the cosmetic treatment plan can include instructions for an automated cosmetic application tool to apply cosmetic materials to a recipient to achieve a desired appearance for the recipient.

[0096]

[0085] In step 260, the results of the process of applying the cosmetic treatment plan to the recipient may be simulated on the 3D model of the recipient. In one exemplary embodiment, the simulated results may depict the recipient made up according to the cosmetic treatment plan.

[0097]

[0086] In step 262, an intermediate simulation result may be generated that depicts a recipient having makeup applied in accordance with partial application of the cosmetic treatment plan.

[0087] In one exemplary embodiment, an intermediate simulation result can be generated based on a 3D model or digital image of the recipient's face. A partial cosmetic treatment plan can be applied onto the digital image or 3D model of the recipient's face using computer software that emulates the action of cosmetic treatment tools, computer vision techniques, etc.

[0098]

[0088] In step 264, the intermediate simulation results may be displayed to the user, such as on a computer screen, a mobile device, or any other display device available to the user or the person being treated. In an exemplary embodiment, the user may be able to zoom in or out or rotate the image of the user's face to view the cosmetic application from different angles. This may help the user make a more informed decision about the final cosmetic application. In an exemplary embodiment, the user may be able to select from a variety of hairstyles to use with the cosmetic application results. This may help the user choose the look they most like along with the cosmetic application.

[0099]

[0089] In step 270, responsive actions may be performed based on the simulation results.

[0090] In step 272, a user's review of the intermediate simulation results may be obtained. In one exemplary embodiment, the intermediate simulation results may be reviewed to allow the user to make any necessary adjustments to the cosmetic application plan before proceeding to the final cosmetic application. Additionally or alternatively,

[0100]

[0091] In step 274, the cosmetic treatment plan may be updated based on the user's considerations.

[0092] In step 290b, a dynamically updated cosmetic treatment plan, or a portion thereof, can be implemented to achieve the desired appearance, taking into account the recipient's movements and the implementation of that portion of the cosmetic treatment plan.

[0101]

[0093] Reference is now made to Figure 3A, which illustrates a flow chart diagram of a method in accordance with certain exemplary embodiments of the disclosed subject matter.

[0094] In step 310, a 3D surface of the recipient's face can be obtained. In an exemplary embodiment, the 3D surface of the recipient's face can be determined based on visual input captured of the recipient. In an exemplary embodiment, the visual input can be obtained from a visual sensor such as a camera, scanner, range-finding imaging sensor, etc. The visual sensor can be configured to scan the surface to which the requested material is to be applied and its attributes, such as the face, neck, other organs of the user, the surface of an object, etc.

[0102]

[0095] In a representative embodiment, the visual input may be analyzed to determine attributes of the 3D surface of the face (or alternatively, any other organ or surface to which makeup is applied or material is applied). The attributes may include the surface color structure (e.g., the user's skin color, background color, etc.), surface texture (e.g., dry skin, acne, pores, wrinkles, etc.), etc. Additionally or alternatively, the visual input may be analyzed to determine the location or positional relationship of surface features such as coordinates of points of interest, boundaries, precise locations of facial features, reference points, etc. As an example, the coordinates of the eyes, nose, eyebrows, lips, etc. may be identified. In a representative embodiment, the analysis may be performed using image analysis techniques, machine learning, etc.

[0103]

[0096] In step 320, user input may be obtained that is indicative of the desired cosmetic application result. In one exemplary embodiment, the user may be allowed to upload a photo of the desired look, such as to select a look from a catalog, to select a look from multiple photos, etc. Additionally or alternatively, the user may be allowed to upload a sketch of the desired result, a combination of different renditions thereof, etc. Additionally or alternatively, the user may be allowed to modify or update the look using a graphical user interface (GUI). In one embodiment, a what-you-see-is-what-you-get interface is used, such as by clicking and dragging on the cosmetic mask that depicts the applied brush on the cheek area, for example. In one embodiment, the user may be allowed to move the mask to a new position on the cheek, expand the mask to cover a smaller or larger area on the cheek, etc. Additionally or alternatively, the user may be allowed to provide any other type of input indicative of the desired result, such as verbal input, text input, etc.

[0104]

[0097] In step 330, a desired appearance may be determined based on the user input and the recipient's face. In an exemplary embodiment, the desired appearance may be obtained directly from the user input, such as based on a previous selection, a selection from dynamic modeling of the makeup on the recipient's face, etc. Additionally or alternatively, the desired appearance may be dynamically generated based on the recipient's face (e.g., a photograph thereof, a 3D model thereof, etc.), based on facial attributes, etc. The desired appearance may be an adaptation of the user input to the recipient's face. As an example, a user may take photographs of the makeup design for different people with different facial attributes, different head structures, different skin colors, etc. The desired appearance may be an adaptation of the makeup design to the recipient's face. As another example, the user input may include keywords or descriptions of the desired appearance, such as heavy makeup, smoked makeup, daily makeup, color matching instructions, etc. The desired appearance may be automatically generated based on the input.

[0105]

[0098] In step 340, a cosmetic treatment plan can be developed based on the 3D surface of the face and the desired appearance. In one exemplary embodiment, the cosmetic treatment plan can include instructions for an automated cosmetic application tool, such as directions, mixtures of materials at each time and position, etc., for the process to give the desired appearance. Additionally or alternatively, the cosmetic treatment plan can include an optimal set of trajectories in 3D space, and the cosmetic application tool is configured to follow the set of trajectories to achieve the desired appearance. Each trajectory can represent the path of the cosmetic application tool while applying cosmetic materials to the recipient, its position as a function of time (such as its X, Y, Z coordinates), etc. Additionally or alternatively, each trajectory of the cosmetic application tool can be configured to emulate the movements of an expert when manually applying materials on a surface to achieve a desired result.

[0106]

[0099] Additionally or alternatively, the cosmetic application plan may include ordered sequences of instructions directed to the cosmetic application tool, indicating the target area to which the makeup is to be applied as a function of time, such as every 0.1 seconds, every 0.5 seconds, every second, etc.

[0107]

[0100] In step 342, the material to be applied, application attributes of the material application (such as strength and consistency), and application distance and orientation of the material application can be determined for each target area on the 3D surface to achieve a desired look in the target area. In an exemplary embodiment, the material to be applied on each target area can be composed of different compositions of cosmetic materials, colors, etc. based on the attributes of the target area, the attributes of the surface, the desired look in the target area, etc. Additionally or alternatively, the distance to apply the material can be calculated for each target area based on the attributes of the target area, such as organ sensitivity (for example, about 3-5 cm from the eyes, 6-10 cm from the neck, etc.), the required material thickness, the required area, etc. As an example, the larger the distance, the larger the application radius. For foundation, contour, blush, bronzer, and highlighter, the application radius may range between 1-5 cm, while for eyeshadow and lipstick, the application radius is often between 0.5-2 cm. Additionally or alternatively, the application attributes may include an application pressure to use when applying the material to the target area, a type of airbrush nozzle to use while applying the material to the target area, etc. For airbrush makeup applications, the air pressure may range between 5-20 psi.

[0108]

[0101] Additionally or alternatively, the material to be applied, the distance over which the material is applied, and other application attributes can be related to each other. As an example, an airbrush makeup tool can be configured to perform a circular or back and forth motion when applying foundation. Additionally or alternatively, the material to be applied, the distance over which the material is applied, and other application attributes can be determined based on attributes of the makeup tool, such as the type of airbrushing, the nozzle, etc.

[0109]

[0102] Additionally or alternatively, the material to be applied, the distance the material is applied, and other application attributes can be calculated based on safety considerations such as the response time of the automated cosmetic application tool to the movement of the recipient, the sensitivity of the organs to which the cosmetic material is applied, etc., to ensure sufficient time to avoid injury to the recipient, etc. It should be noted that in some cases, the range of movement of the recipient is not necessarily 100% free. Instead, the movement of the recipient is limited in some exemplary embodiments by physical constraints that restrict the movement of the recipient, such as the use of a chin rest, a foredream, a strap, a combination of these, etc.

[0110]

[0103] Additionally or alternatively, an estimated amount of material and estimated amount of time required to complete each trajectory can be calculated taking into account consideration parameters such as air pressure, coverage, viscosity of material, etc., and communicated to the user by display, verbal communication, etc.

[0111]

[0104] In step 346, one or more instructions may be generated for the automated cosmetic application tool configured to use the application attributes to apply the material to the target area from an application distance.

[0112]

[0105] Reference is now made to Figure 3B, which illustrates a flow chart diagram of a method in accordance with certain exemplary embodiments of the disclosed subject matter.

[0106] In step 310b, a 3D model of the recipient's face can be obtained. In one exemplary embodiment, the 3D model of the face can be a digital representation of the recipient's face that can be manipulated and viewed from different angles. The 3D model can be generated using specialized software, 3D scanning techniques, modeling techniques, generative AI techniques, etc. As an example, the 3D model can be generated automatically using a 3D scanner. The 3D scanner is configured to capture the contour and surface texture of the recipient's face and then convert this data into a digital format. Additionally or alternatively, the 3D model can be generated based on other types of visual inputs capturing the recipient's face, such as sensor readings obtained in step 230 of FIG. 2A, the 3D surface of the face obtained in step 310 of FIG. 3A, an image of the user obtained in a simulation process, such as shown in FIG. 5, etc.

[0113] In step 330b, a desired appearance can be obtained. Step 330b can be similar to step 330 of Fig. 3A. Additionally or alternatively, the desired appearance can be automatically generated based on user input and the 3D model of the face, such as by applying user input on the 3D model and dynamically adapting the desired appearance for the recipient based on the 3D model.

[0114]

[0108] In step 340b, a cosmetic treatment plan may be developed based on the 3D surface of the face or a 3D model of the recipient.

[0109] In one exemplary embodiment, the cosmetic treatment plan may include instructions for utilizing stencils, such as pre-existing 2D stencils, auto-folding 3D stencils, and the like.

[0115]

[0110] At step 350, a 4D stencil configured to be applied to a recipient can be designed. In one exemplary embodiment, the 4D stencil can be designed based on a 3D model of the recipient. The 4D stencil can be designed to achieve a particular cosmetic result according to a cosmetic treatment plan, such as to allow for the application of thin lines, sharp edges, etc.

[0116]

[0111] In one exemplary embodiment, the 4D stencil can be a custom-fit, auto-folded 3D stencil that can be custom-fit to the recipient's face. The custom-fit, auto-folded 3D stencil can be created based on an analysis of a scanned visual input of the user's face, based on a desired look, based on a 3D model of the recipient, etc. The custom-fit, auto-folded 3D stencil can automatically and individually adapt to the recipient's facial shape to achieve the desired results, such as specific shapes, fine lines, sharp edges, etc., which may not be possible or accurate without the use of a stencil, especially when utilizing airbrush techniques. Additionally or alternatively, the 4D stencil can define an alternative to traditional 2D generic stencils.

[0117]

[0112] In step 352, a 4D stencil can be fabricated. In one exemplary embodiment, the 4D stencil can be fabricated using 4D printing technology. A custom-designed auto-folding 3D stencil can be created using programmable 4D printing. After the fabrication process, the printed stencil reacts to parameters in the environment (humidity, temperature, voltage, etc.) and changes its shape accordingly. The auto-folding stencil can perfectly conform to the user's face and allows for much easier material application without any handling.

[0118]

[0113] In a representative embodiment, 4D printing can be configured to encode self-actuating deformations during the printing process so that an object can be fabricated in a flat surface and then transformed into a target 3D shape. 4D printing can include printing a 3D structure, or printing a 2D structure that can be formed into a 3D form, such as by folding a 2D surface. The fourth dimension involved in 4D printing can be the time dimension, since a 3D object (e.g., a 3D printed object, a 2D printed surface folded into a 3D object, etc.) can change its form over time. The change in form can be caused by heat, contact with other materials, etc. As an example, Fused Deposition Modeling (FDM) printing techniques can be used to injection mold molten thermoplastics through a narrow nozzle, which stretches the material in the printing direction. The pre-stretching causes the material to shrink in the printing direction during heating. In addition to the shrinkage direction, the amount of shrinkage can also be controlled by the printing thickness of each layer. As another example, a printed 2D flatsheet, e.g., a personalized stencil, can be uniformly heated in a hot water bath, such as at about 90° C., and self-transform into a target 3D surface. As yet another example, a printed stencil (2D or 3D) can also automatically transform when brought close to a user's face, or when it comes into contact with a base material applied to the face using an airbrush machine, etc.

[0119]

[0114] In step 353, a 4D (e.g., custom-designed self-folding 3D stencil) stencil can be applied to the recipient's face to obtain an updated 3D surface. In a representative embodiment, the custom-designed self-folding 3D stencil can be automatically applied to the recipient's face at a predetermined position and at a predetermined time according to a calculated trajectory. Additionally or alternatively, the custom-designed self-folding 3D stencil can be mounted on a fixture or manually held and placed in a precise position by a user. In a representative embodiment, a barcode can be printed on the stencil to facilitate positioning the stencil in a scene and guide the user when placing the stencil with respect to the location of points of interest in the scene, such as the edges of the eyes or lips.

[0120]

[0115] In one exemplary embodiment, the custom-designed self-folding 3D stencil can also be capable of transforming into different configurations in response to environmental stimuli, with the fourth dimension resulting in a time-dependent shape change after printing.

[0121]

[0116] In step 390b, a cosmetic treatment plan can be implemented while the 4D stencil is applied to the recipient. After the mixed material is applied, the 4D custom stencil can be automatically removed, adjusted, replaced, etc. An additional layer of material may or may not be applied over the material applied according to the 4D custom stencil.

[0122]

[0117] Reference is now made to Figures 4A-4E, which show schematic diagrams of an exemplary architecture in accordance with certain exemplary embodiments of the disclosed subject matter.

[0118] Figure 4A shows a schematic diagram of an exemplary architecture of system 400. Figure 4B shows a magnified schematic diagram of an exemplary architecture of some components of system 400. Figure 4C shows a schematic diagram of an exemplary architecture of components of system 400. Figure 4D shows the translational motion of components of system 400. Figure 4E shows the rotational motion of components of system 400.

[0123]

[0119] In one exemplary embodiment, the system 400 may be an automated makeup application system. The system 400 may comprise an automated makeup machine that includes a robotic arm 410 mounted on a base module 405. The robotic arm 410 may be movable in three-axis translational motion, as shown in FIG. 4D. The system 400 may include one or more sensors, such as a camera 440, that monitor the movement of a person being treated by the system 400.

[0124] In a representative embodiment, one or more motors 415 may be utilized to perform translational and rotational motion as shown in FIG. 4C. The motors 415 may be of different types, including electric, hydraulic, pneumatic, etc. The motors 415 may comprise linear actuators, each of which acts on one or more axes of motion (X, Y, and Z). Each linear actuator may connect to a particular joint or robot arm 410 to move it along a desired axis. The actuators may be located on the robot arm 410, on the joint itself, on the base module 405, etc. The movement of the robot arm 410 may be controlled by a motion controller (not shown) or by a control unit (not shown). The control unit receives commands or instructions from a computer or other control system, such as from the device 600 shown in FIG. 6. The motion controller may be configured to control the linear actuators to move the arm along a desired path and trajectory. The accuracy of the movement of the robot arm 410 may be improved by incorporating sensors to measure the position and orientation of the robot arm 410. The sensors may be encoders, accelerometers, gyroscopes, etc. The sensors may be configured to provide feedback to the motion controller to adjust the movement of the robotic arm 410 as needed according to the cosmetic treatment plan and the movements of the patient.

[0125] In one exemplary embodiment, the airbrush 420 can be mounted on the robot arm 410 using a Wrist Module 430. The Wrist Module 430 can be configured to enable two-axis (e.g., pitch and yaw) rotational movement of the airbrush 420, as shown in FIG. 4E. The two-axis rotational movement can be performed by an actuator, such as a motor 415. As an example, one actuator can be used for rotation about the Z axis and the other actuator can be used for rotation about the Y axis.

[0126]

[0122] In one exemplary embodiment, the system 400 can include an air compressor 455 configured to drive the application of material by the airbrush 420. The air compressor 455 can be configured to provide different air pressure levels according to a cosmetic treatment regimen.

[0127]

[0123] In one exemplary embodiment, the system 400 can include a color mixer 425 configured to provide materials for application by the airbrush 420. The color mixer 425 can be configured to mix materials for applying makeup on a recipient as prescribed by a makeup application plan.

[0128]

[0124] In one exemplary embodiment, the airbrush 420 may be attachable to multiple alternative nozzles 475 having different sizes and shapes. The nozzles 475 can be automatically selected and attached or detached from the airbrush 420 to allow for different application patterns.

[0129]

[0125] Reference is now made to Figure 5. Figure 5 shows a schematic diagram of an exemplary simulation according to certain exemplary embodiments of the disclosed subject matter.

[0126] Figure 5 shows a simulation of the results of a process of applying a cosmetic treatment plan on a recipient 500 by an automated cosmetic treatment tool, such as the machine shown in Figures 1A-1C or 4A-4E. The intermediate simulations 510-550 may be a simulation of executing the instructions of the cosmetic treatment plan on a 3D model of the recipient 500 to obtain a simulated result depicting the recipient 500 applying makeup according to the cosmetic treatment plan. The intermediate simulations 510-550 may be displayed to a user using a display device, such as one similar to the device 195 shown in Figure 1A or any other display of a computing device available to the user.

[0130] Additionally or alternatively, the intermediate simulations 510-550 may be portions of a video or a sequence of images depicting the process of executing instructions of a cosmetic treatment plan using an automated cosmetic treatment tool. The video or sequence of images may be configured to make the process more predictable for a user or patient by displaying each of the 3D trajectories of the cosmetic treatment plan.

[0131]

[0128] In an exemplary embodiment, each of the intermediate simulations 510-550 may simulate the application of different makeup on the same or different target areas of the recipient 500. Each of the intermediate simulations 510-550 may be an intermediate simulated result depicting the recipient 500 applying makeup according to a partial application of a makeup treatment plan. Additionally or alternatively, each of the intermediate simulations 510-500 may simulate the application of different makeup materials according to different portions of the makeup treatment plan. Each of the intermediate simulations 510-550 may be configured to simulate different or separate trajectories of the makeup treatment plan 454.

[0132]

[0129] In one exemplary embodiment, the intermediate simulation 510 can be configured to simulate an initial appearance of the recipient 500, such as when not wearing makeup, before starting application of the cosmetic treatment plan, etc. The intermediate simulation 520 can be configured to simulate a first portion of the cosmetic treatment plan applied to the intermediate simulation 510 according to a first 3D trajectory. The intermediate simulation 520 can be configured to simulate a first cosmetic layer on the face of the recipient 500, such as a foundation layer 525. The intermediate simulation 510 can also be configured to examine a previous cosmetic layer or portion of the cosmetic treatment plan, such as a contouring layer, a corrective cosmetic layer, etc., that is configured to be applied before the first portion of the cosmetic treatment layer.

[0133]

[0130] Intermediate simulation 530 can be configured to simulate a second layer of makeup applied to recipient 500 according to a second portion of the makeup treatment plan. Intermediate simulation 530 can be configured to simulate a second layer of makeup on the face of recipient 500, such as an eye shadow layer 535. The second applied layer can be applied over the first applied layer, for example, simulated over intermediate simulation 520.

[0134]

[0131] Intermediate simulation 540 can be configured to simulate a third cosmetic layer applied to recipient 500 according to a third portion of the cosmetic treatment plan. Intermediate simulation 540 can be configured to simulate a third cosmetic layer on the face of recipient 500, such as a brushing layer 545. The third applied layer can be applied over the second applied layer, for example, simulated over intermediate simulation 530.

[0135]

[0132] Intermediate simulation 550 can be configured to simulate a fourth cosmetic layer applied to recipient 500 according to a fourth portion of the cosmetic treatment plan. Intermediate simulation 550 can be configured to simulate a fourth cosmetic layer on the face of recipient 500, such as a lipstick layer 555. The fourth applied layer can be applied over the third applied layer, for example, simulated over intermediate simulation 540.

[0136]

[0133] It may be noted that simulating different parts of a cosmetic treatment plan on the same area, such as area 590 on the face of the recipient 500, may result in corresponding changes in intermediate simulation results for the same target area, such as one layer on the previous layer. Thus, instead of simulating only the final result, such as simulation 550, where the pixels in area 590 represent the result of one step, the pixels in area 590 may be simulated for the initial features, color, texture, etc. in intermediate simulation 510. The same pixels in area 590 may then be simulated using foundation layer 525 on top of the initial features, color, texture, etc. in intermediate simulation 520. In intermediate simulation 530, the pixels in area 590 do not need to be updated, but in intermediate simulation 540, because of the addition of brush layer 545, it makes sense to update the pixels in area 590. This results in a more accurate and realistic simulation result that emulates the actual automated treatment, step by step, trajectory by trajectory, layer by layer, etc., according to the treatment plan. Furthermore, the user may be able to review each intermediate simulation and update or instruct the system to modify specific relevant portions of the cosmetic application.

[0137]

[0134] In an exemplary embodiment, the series of images (e.g., intermediate simulations 510-550), videos, etc., can be configured to simulate the use of a 4D stencil as performed during the application of a cosmetic treatment plan. Additionally or alternatively, the series of images (e.g., intermediate simulations 510-550), videos, etc., can be configured to simulate a corrective makeup process performed according to a cosmetic treatment plan. As an example, the intermediate simulation 520 can be configured to simulate corrective makeup performed to blur imperfections on the face of the person 500 using a foundation layer 525. As another example, the intermediate simulation 550 can be configured to simulate corrective makeup to correct an asymmetry in the lips of the person 500 (the upper lip on the right side is higher than the upper lip on the left side) using a lipstick layer 555 that corrects the lips of the person 500 to appear symmetrical. Additionally or alternatively, this asymmetry can be taken into account to create a perfectly symmetrical stencil, resulting in perfectly symmetrical lips.

[0138]

[0135] Reference is now made to Figure 6, which illustrates a block diagram of an apparatus in accordance with certain exemplary embodiments of the disclosed subject matter. Apparatus 600 can be configured to support parallel user interaction with real-world physical systems and their digital representations in accordance with the disclosed subject matter.

[0139]

[0136] In one exemplary embodiment, the device 600 may include one or more processors 602. The processor 602 may be a central processing unit (CPU), a microprocessor, an electronic circuit, an integrated circuit (IC), etc. The processor 602 may be utilized to perform calculations required by the device 600 or any of its subcomponents.

[0140] In certain exemplary embodiments of the disclosed subject matter, the apparatus 600 can include an input / output (I / O) module 605. The I / O module 605 can be utilized to provide output to and receive input from the user or any other device associated therewith, such as, for example, obtaining visual input capturing a user's face, providing a catalog of looks to the user and obtaining a selection of a desired look from the user, obtaining user input indicating a desired look, displaying cosmetic results to the user, providing instructions to other devices, etc.

[0141]

[0138] In an exemplary embodiment, the I / O module 605 can be utilized to obtain sensor readings from one or more sensors 610. The sensors 610 can be configured to monitor the movements of a person to whom the automated makeup application tool 680 is configured to apply makeup, or a person to whom makeup is already being applied. In an exemplary embodiment, the sensors 610 can be connected to the automated makeup application tool 680, can be a component thereof, etc. Additionally or alternatively, the sensors 610 can be detached and not-directly associated with the automated makeup application tool 680.

[0142]

[0139] In one exemplary embodiment, the device 600 may include a memory 607. The memory 607 may be a hard disk drive, a flash disk, a random-access memory (RAM), a memory chip, etc. In one exemplary embodiment, the memory 607 may hold program code operable to cause the processor 602 to perform acts associated with any of the subcomponents of the device 600.

[0143] Additionally or alternatively, the device 600 may be configured to control the automated cosmetic application tool 680, provide instructions thereto, manage the automated cosmetic application process, etc. The memory 607 may hold program code that operates to cause the processor 602 to execute a computer program product or software that controls the automated cosmetic application tool 680, any other automated cosmetic machine, an airbrushing robot, etc. The automated cosmetic application tool 680 may be configured to execute a predefined cosmetic application plan, or a portion thereof, to achieve a desired appearance for the recipient, taking into account the recipient's movements.

[0144]

[0141] In one exemplary embodiment, a cosmetic treatment planner 620 can be configured to develop a cosmetic treatment plan based on visual input captured of the recipient, based on a 3D surface of the face, based on user input indicating a desired appearance, etc. The cosmetic treatment plan can comprise instructions directed to an automated cosmetic application tool to apply cosmetic materials to the recipient to achieve a desired appearance for the recipient. The cosmetic treatment planner 620 can be configured to obtain the 3D surface of the recipient's face, such as based on input from the sensor 610. Additionally, the cosmetic treatment planner 620 can be configured to obtain the desired look from a desired look generator 625, directly from a user, etc. In one exemplary embodiment, the desired look generator 625 can be configured to determine the desired look based on user input indicating the outcome of the requested cosmetic treatment on the recipient's face, such as based on a user selection from the appearance database 615, based on adaptation of the user input to the recipient's facial surface, etc.

[0145]

[0142] In one exemplary embodiment, the cosmetic application planner 620 can be configured to generate instructions for the automated cosmetic application tool 680 to apply cosmetic material from a predefined position in space a defined distance away from the facial surface. The cosmetic application planner 620 can be configured to utilize a 3D trajectory generator 622 to calculate movement instructions that define a 3D trajectory for the automated cosmetic application tool 680 to follow to achieve a desired look. Additionally or alternatively, the 3D trajectory generator 622 can be configured to define different application trajectories for different nozzles or components of the automated cosmetic application tool 680.

[0146]

[0143] Additionally or alternatively, the cosmetic application planner 620 can be configured to utilize a material module 624 to determine material application instructions, each instruction indicating a material to be applied to each application location on the recipient's face. The material module 624 can be configured to calculate the composition of cosmetic material required for a particular location, such as based on the recipient's facial propertied and the desired appearance. Additionally or alternatively, the cosmetic application planner 620 can be configured to utilize a treatment attribute module 626 to calculate the attributes of the treatment to be performed by the automated cosmetic application tool 680. When applying the material on the target area, the treatment attributes are required to achieve the desired appearance, such as the application distance to achieve the desired appearance in the target area. Additionally or alternatively, the treatment attribute module 626 can be configured to determine the pressure to be used when applying the material on the target area, the type or size of the nozzle to be used when applying the material, the type and duration of use of the pod to be used when applying the material, etc. The cosmetic application planner 620 can be configured to generate one or more instructions. The one or more instructions are configured to cause the automated makeup application tool 680 to apply a material determined by the material module 624 from an application distance determined by the makeup attribute module 626, according to a pressure attribute or any other application attribute determined by the application attribute module 626, onto a target area determined with respect to a position on the 3D trajectory determined by the 3D trajectory generator 622.

[0147]

[0144] It may be noted that the treatment attribute module 626 may be configured to determine different treatment attributes, for example, different distances, different pressures, etc., for different target areas on the recipient's face.

[0148]

[0145] Additionally or alternatively, the cosmetic treatment planner 620 may be configured to develop a cosmetic treatment plan based on safety considerations, or to ensure sufficient time to avoid injury to the recipient, such as the response time of the automated cosmetic treatment tool 680 to movements of the recipient, to avoid injury to the recipient, etc.

[0149]

[0146] Additionally or alternatively, the cosmetic treatment planner 620 may be configured to automatically and continuously update the cosmetic treatment plan during the application by the automated cosmetic treatment tool 680, such as after each predetermined portion of the cosmetic treatment is performed, at predetermined times, etc., based on the identification of the movement of the recipient by the motion monitor 635. In one exemplary embodiment, the motion monitor 635 may be configured to analyze sensor readings obtained from the sensor 610, or other sensors associated with the automated cosmetic treatment tool, during the application of the cosmetic treatment plan. The cosmetic treatment planner 620 may be configured to dynamically update or replan the cosmetic treatment plan in response to the identification of the movement of the recipient by the motion monitor 635. The cosmetic treatment planner 620 may be configured to dynamically update the cosmetic treatment plan by modifying instructions based on the movement of the recipient to maintain the defined distance and other treatment attributes.

[0150]

[0147] Additionally or alternatively, the cosmetic treatment planner 620 may be configured to instruct the 4D stencil module 665 to fabricate a 4D stencil configured to be applied to the recipient during application of the cosmetic treatment plan or a portion thereof. The automated cosmetic treatment tool 680 may be instructed to execute the cosmetic treatment plan while the 4D stencil is applied to the recipient to enable creation of specific shapes, lines, etc. The 4D stencil module 665 may be configured to design the 4D stencil based on a 3D model of the recipient, a 3D model generator based on input from the sensor 610, etc. The 4D stencil module 665 may be configured to instruct a designated device, such as a 4D printer 690, a designated component of the automated cosmetic treatment tool 680, etc., to fabricate the 4D stencil. Additionally or alternatively, the 4D stencil module 665 may be configured to instruct the user or the automated cosmetic treatment tool 680 to apply the 4D stencil to a predetermined location on the recipient, remove the 4D stencil, etc.

[0151]

[0148] Additionally or alternatively, the cosmetic treatment planner 620 can be configured to correct the cosmetic treatment plan based on the corrective cosmetic instructions or measurements determined by the corrective cosmetic module 645. The corrective cosmetic module 645 can be configured to identify facial defects that may make it impossible to reach the desired appearance, such as facial dissimilarity, skin imperfections, pigmentation, scars, and inappropriate sizes of certain organs, and determine instructions to perform corrective cosmetic treatment to overcome such defects. Additionally or alternatively, the corrective cosmetic module 645 can be configured to identify such facial imperfections and determine corrective cosmetic treatment to enhance the desired appearance without user input indicating the same. The corrective cosmetic module 645 can be configured to determine instructions to enhance the appearance of facial features by creating the illusion of balance and symmetry using cosmetic materials. The corrective cosmetic module 645 can be configured to identify areas of the face that require correction to reach the desired appearance, and determine shades of cosmetic materials to utilize to highlight or contour the particular areas of the face. The corrective makeup module 645 can be configured to provide instructions to the materials module 624 to generate precise material compositions to achieve light and dark shades and colors to highlight and contour features, create effects of balance and proportion, etc. Additionally or alternatively, the corrective makeup module 645 can be configured to provide instructions to the treatment attributes module to set treatment attributes that enable the illusion of shape, precise highlighting, precise contouring, etc.

[0152]

[0149] In one exemplary embodiment, the simulation module 650 can be configured to simulate the results of the process of applying a cosmetic treatment plan to a recipient. The simulation module 650 can be configured to simulate the execution of instructions of the cosmetic treatment plan generated by the cosmetic treatment planner 620 on the 3D model of the recipient generated by the 3D model generator 640. The simulated results can depict the recipient, layer by layer, with makeup applied according to the cosmetic treatment plan. As an example, the simulated results can include a simulation of a particular cosmetic treatment on a target area of ​​the recipient, and a simulation of a second cosmetic treatment on the target area independent of or on top of the first particular cosmetic treatment. Additionally or alternatively, the simulation module 650 can be configured to generate a series of intermediate simulation results. The series of intermediate simulation results depict the recipient with makeup applied according to separate portions of the application of the cosmetic treatment plan. The intermediate simulation results can be displayed to a user on the display device 670 to enable the user to review the cosmetic plan.

[0153]

[0150] The present invention may be a system, a method, and / or a computer program product that may include a computer-readable storage medium (or a plurality of media) having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0154]

[0151] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage media includes the following: portable computer diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, punch cards, or mechanically encoded devices such as raised structures in the grooves in which the instructions are recorded, and any suitable combination of the above. Computer-readable storage medium, as used herein, should not be construed as a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating in a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted down wires.

[0155]

[0152] The computer-readable program instructions described herein may be downloaded to each computing / processing device from a computer-readable storage medium or through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or storage device. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on the computer-readable storage medium within the respective computing / processing device.

[0156]

[0153] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages. Programming languages ​​include object oriented programming languages ​​such as Smalltalk, C++, etc., and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer as a stand alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network. The network may include a local area network (LAN) or a wide area network (WAN), or may provide a connection to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer readable program instructions to personalize the electronic circuitry by utilizing state information of the computer readable program instructions to perform aspects of the invention.

[0157] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0158]

[0155] These computer-readable program instructions, when provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device, can constitute a machine, and when executed by the processor of the computer or other programmable data processing device, the instructions form a means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions can also be stored on a computer-readable storage medium, and can direct a computer, programmable data processing device, and / or other device to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored constitutes an article of manufacture including instructions for implementing aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0159]

[0156] Furthermore, the computer readable program instructions, when loaded onto a computer, other programmable data processing apparatus, or other device, may cause the computer, other programmable apparatus, or other device to perform a series of operational steps, and may also generate a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0160]

[0157] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which constitute one or more executable instructions that implement the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, and these blocks may sometimes be executed in reverse order depending on the functions involved. It is also noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the blocks and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that executes the specified functions or acts, or executes a combination of special-purpose hardware and computer instructions.

[0161]

[0158] The terms used herein are merely for the purpose of describing particular embodiments and are not intended to limit the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, it will be understood that the terms "comprises" and / or "comprising", as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or aggregates thereof.

[0162]

[0159] The corresponding structure, material, act, or equivalent of any means or step-plus-function element in the following claims is intended to include any structure, material, or act that performs a function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description only, and is not intended to be exhaustive or to limit the invention to the disclosed form. Many changes and modifications that do not depart from the scope and spirit of the invention will be apparent to those skilled in the art. The above embodiments have been selected and described in order to best explain the principles and practical applications of the invention, and to enable others skilled in the art to understand the invention in various embodiments with various modifications as appropriate for the particular use envisaged.

Claims

1. 1. A method comprising: obtaining a cosmetic treatment plan, the cosmetic treatment plan including instructions for an automated cosmetic application tool, the instructions being configured, when executed by the automated cosmetic application tool, to apply cosmetic material onto a recipient to achieve a desired appearance for the recipient; executing a portion of the cosmetic treatment plan by the automated cosmetic treatment tool; obtaining sensor readings from a sensor during said performance, said sensor being configured to monitor movement of said patient; updating the cosmetic treatment plan in response to identifying a movement of the recipient based on the sensor readings to obtain a dynamically updated cosmetic treatment plan; executing the dynamically updated cosmetic treatment plan, or a portion thereof, to achieve a desired appearance for the recipient, while taking into account the recipient's movements and the execution of the portion of the cosmetic treatment plan; A method comprising:

2. 2. The method of claim 1, the cosmetic regimen instructions include instructions to apply the cosmetic material from a predefined location in space a defined distance from a facial surface; The method, wherein updating the cosmetic plan includes modifying the instructions based on movement of the recipient to maintain the defined distance.

3. 2. The method of claim 1, the instructions include movement instructions defining a 3D trajectory to be followed by the automated cosmetic application tool to achieve the desired look; The method, wherein the instructions include material application instructions, each material application instruction indicating an application location, a material to be applied, and attributes of the makeup to be applied by the automated makeup application tool.

4. 2. The method of claim 1, wherein obtaining the cosmetic treatment plan comprises: obtaining a three-dimensional (3D) surface of the recipient's face; obtaining the desired appearance, the desired appearance being determined based on user input indicating a desired cosmetic result on the recipient's face; planning the cosmetic treatment based on the user input and further based on the 3D surface of the face; A method comprising:

5. 5. The method of claim 4, wherein the planning step comprises: determining a material to be applied to a target area on the 3D surface of the face, application attributes for application of the material, and an application distance and orientation at which the material is to be applied to the target area to achieve the desired appearance in the target area; generating one or more instructions to the automated cosmetic application tool configured to apply the material using the application attributes from the application distance and orientation onto the target area; A method comprising:

6. The method of claim 5 , wherein the application attributes include an application pressure used when applying the material onto the target area.

7. 6. The method of claim 5, wherein the planning step is performed for a first target area and a second target area, and a coating distance for the first target area is different than a coating distance for the second target area.

8. 6. The method of claim 5, wherein the planning step is performed for a first target area and a second target area, the application attribute is a pressure used by the automated cosmetic application tool, an application distance in the first target area is equal to an application distance in the second target area, and the pressure used by the automated cosmetic application tool in the first target area is different from the pressure in the second target area.

9. 5. The method of claim 4, wherein the planning step is performed based on safety considerations, the safety considerations including a response time of the automated cosmetic application tool to movements of the recipient, thereby ensuring sufficient time to avoid injury to the recipient.

10. 10. The method of claim 1, wherein the updating step is performed to avoid injury to the recipient.

11. 2. The method of claim 1, further comprising: simulating an outcome of a process of applying the cosmetic treatment plan on the recipient, the simulating step including simulating execution of instructions of the cosmetic treatment plan on a 3D model of the recipient, thereby obtaining a simulated outcome depicting the recipient made up in accordance with the cosmetic treatment plan, the simulating execution of instructions including simulating a first application of makeup on a target area of ​​the recipient, and simulating a second application of makeup on the target area; displaying the simulation results; A method comprising:

12. 12. The method of claim 11, wherein the simulating step includes generating an intermediate simulation result depicting the recipient made up according to partial application of the cosmetic treatment regimen; The method, comprising displaying the intermediate simulation results.

13. 2. The method of claim 1, wherein the cosmetic treatment plan includes instructions for creating a four-dimensional (4D) stencil configured to be applied to the recipient, the method further including the step of creating the 4D stencil, and the step of executing is performed while the 4D stencil is being applied to the recipient.

14. 14. The method of claim 13, wherein the 4D stencil is fabricated based on a three-dimensional (3D) model of the recipient.

15. 2. The method of claim 1, wherein the automated makeup application tool comprises an airbrush movable with five degrees of freedom, the airbrush capable of translational movement along three axes and rotational movement about two axes.

16. 16. The method of claim 15, wherein the airbrush has a plurality of nozzles having varying sizes and shapes.

17. 17. The method of claim 16, wherein the cosmetic application plan defines a first application trajectory for a first nozzle and a second application trajectory for a second nozzle, the cosmetic application plan defining a relative application order between the first nozzle and the second nozzle.

18. A machine, a robot arm movable with five degrees of freedom, said five degrees of freedom including translational movement along three axes and rotational movement about two axes; an airbrush mounted on the robotic arm; A sensor for monitoring the movement of the patient; a control unit for controlling movement of the robotic arm in accordance with a cosmetic application plan, the control unit further configured to control application of the airbrush in accordance with the cosmetic application plan, the control unit configured to modify the cosmetic application plan based on sensor readings from the sensor; A machine equipped with:

19. 20. The machine of claim 18, further comprising an air compressor configured to effect application of material through the airbrush, and wherein the control unit is configured to instruct the air compressor to provide different air pressure levels according to the cosmetic treatment regimen.

20. 20. The machine of claim 18, further comprising a material mixer that supplies materials to be applied by the airbrush, and wherein the cosmetic application plan defines different materials to be mixed to apply makeup on the recipient.

21. 20. The machine of claim 18, wherein said airbrush is mountable with a plurality of alternative nozzles having different sizes and shapes, thereby allowing different application patterns by said airbrush.

22. 22. The machine of claim 21, wherein the machine is configured to automatically attach and detach a nozzle from the airbrush.

23. 20. The machine of claim 18, wherein the machine is coupled to a stencil making device that makes a four-dimensional (4D) stencil, the 4D stencil being configured to be applied to the recipient during application of makeup according to the cosmetic application plan.

24. 20. The machine of claim 18, further comprising a chin rest and a forehead rest.

25. 20. The machine of claim 18, further comprising a proximity sensor that monitors the distance of the airbrush from a physical object.

26. 1. A computerized apparatus having a processor, the processor comprising: obtaining a cosmetic treatment plan, the cosmetic treatment plan including instructions for an automated cosmetic application tool, the instructions being configured, when executed by the automated cosmetic application tool, to apply cosmetic material onto a recipient to achieve a desired appearance for the recipient; executing a portion of the cosmetic treatment plan by the automated cosmetic treatment tool; obtaining sensor readings from a sensor during said performance, said sensor being configured to monitor movement of said patient; updating the cosmetic treatment plan in response to identifying a movement of the recipient based on the sensor readings to obtain a dynamically updated cosmetic treatment plan; executing the dynamically updated cosmetic treatment plan, or a portion thereof, to achieve a desired appearance for the recipient, while taking into account the recipient's movements and the execution of the portion of the cosmetic treatment plan; 23. A computerized apparatus configured to:

Images