MICROROBOT PLATFORM AND USER INTERFACE FOR EYELASH ENHANCEMENT

Abstract

MICROROBOT PLATFORM AND USER INTERFACE FOR EYELASH LIFTING A method for controlling one or more microrobots to apply eyelash extensions includes obtaining digital source image data of a subject; defining an eyelash region of the subject within the source image data; generating an eyelash map based at least in part on the analysis of the defined eyelash region; and generating microrobot control instructions based at least in part on the eyelash map. The control instructions are configured to cause the microrobot(s) to apply eyelashes to the subject based on the eyelash map. Defining the eyelash region may include extracting facial landmarks (e.g., points or contours) from the source image data, identifying the eyelash region based on the facial landmarks, and applying an image mask corresponding to the eyelash region.A modified image or 3D model can be generated based on the source image data and a lash recommendation. Command instructions can also be based on the lash recommendation. Figure for the abbreviation: none.

Description

Title of the invention: MICROROBOT PLATFORM AND USER INTERFACE FOR EYELASH ENHANCEMENT SUMMARY

[0001] In one aspect, a computer-implemented method for controlling one or more microrobots to apply eyelash enhancements includes obtaining digital source image data of a subject; defining an eyelash region of the subject in the digital source image data; generating an eyelash map based at least in part on the analysis of the defined eyelash region; and generating microrobot control instructions based at least in part on the eyelash map, wherein the microrobot control instructions are configured to cause one or more microrobots to apply one or more artificial eyelashes to the subject based on the eyelash map.

[0002] In some embodiments, the definition of the eyelash region includes obtaining facial landmarks (e.g., eye points or contours or eyebrow points or contours) from the digital source image data, identifying the location and shape of the eyelash region based on the facial landmarks, and applying an image mask corresponding to the eyelash region to the digital source image data.

[0003] In some embodiments, the method further includes providing the eyelash map to an eyelash recommendation engine; and by the eyelash recommendation engine, generating an eyelash recommendation based at least in part on the eyelash map, wherein the eyelash recommendation includes a position on an eyelid or an existing eyelash of the subject for an artificial eyelash to be applied by one or more microrobots.

[0004] In certain embodiments, the method further includes performing an attribute analysis on the digital source image data to identify one or more subject attributes (for example, a face shape attribute, an age attribute, an eye attribute, an eyebrow attribute, a complexion attribute, a skin texture attribute, a skin condition attribute, a hair attribute), wherein the eyelash recommendation is further based on the one or more facial attributes.

[0005] In some embodiments, the method further includes providing digital source image data and eyelash recommendation to an image generation module; and generating a modified image or 3D model based on the digital source image data and eyelash recommendation.

[0006] In some embodiments, the method further includes displaying the modified image or 3D model in a lash enhancement user interface.

[0007] In certain embodiments, the eyelash enhancement user interface further includes a virtual try-on feature that allows the eyelash recommendation to be modified through user interaction with the modified image or 3D model.

[0008] In some embodiments, the microrobot control instructions are further based on the recommendation of eyelashes.

[0009] In some embodiments, the method further includes receiving user input from an eyelash enhancement user interface, wherein the microrobot control instructions are further based on the user input.

[0010] In another aspect, a system includes circuitry configured to perform any one of the process steps or steps identified herein, including circuitry configured to obtain digital source image data of a subject; circuitry configured to define a lash region of the subject in the digital source image data; circuitry configured to generate a lash map based at least in part on the analysis of the defined lash region; and circuitry configured to generate microrobot control instructions based at least in part on the lash map, wherein the microrobot control instructions are configured to cause one or more microrobots to apply one or more artificial lashes to the subject based on the lash map.

[0011] In some embodiments, the system further includes one or more cameras configured to capture digital source image data of a subject.

[0012] In some embodiments, the system further comprises one or more microrobots.

[0013] In another aspect, non-transient computer-readable media include instructions stored on them, configured to cause one or more computer devices to perform any of the process steps or steps identified herein.

[0014] This summary is provided to present a selection of concepts in a simplified form, which are described in greater detail below in the detailed description. This summary is not intended to identify key features of the claimed subject matter, nor to be used as an aid in determining the scope of the claimed subject matter. Brief description of the drawings

[0015] The foregoing aspects and many related advantages of the present invention will be more readily appreciated as they are better understood with reference to the following detailed description, when taken in conjunction with the accompanying drawings, in which:

[0016] Fig. 1A is a schematic illustration of a non-limiting example embodiment of an automated eyelash enhancement system using microrobots, according to various aspects of this disclosure;

[0017] Fig. 1B is a schematic diagram that illustrates examples of non-limiting embodiments of a client computer device that can be used to implement aspects of this disclosure;

[0018] Figures 2A-2E are examples of eyelash application systems, in accordance with aspects of this disclosure;

[0019] Figures 3A-3F are examples of microrobots, in accordance with aspects of this disclosure;

[0020] Figures 4A-4B show another example of an eyelash application system, in accordance with aspects of this disclosure;

[0021] Figure 5 is a flowchart of a method for controlling one or more microrobots to apply eyelash enhancements, in accordance with aspects of this disclosure; and

[0022] Figure 6 is a schematic diagram of an example of the sequence of eyelash application operations, in accordance with aspects of this disclosure. Detailed description

[0023] An automated and robotic eyelash extension system is disclosed herein. In some embodiments, the system includes a computer vision system comprising one or more cameras that identify and determine the positions of existing eyelashes; a recommendation engine or an eyelash placement engine that determines where to place the artificial eyelashes; and a microrobot control module that determines, for example, the two-dimensional (2D) or three-dimensional (3D) coordinates and motion patterns for motion-controlled eyelash positioning microrobots and plans the movements / trajectories of these microrobots. In some embodiments, the computer vision system includes multiple cameras (for example, in a stereoscopic camera system).In some embodiments, the computer vision system and the eyelash recommendation / placement engine work together to identify existing natural eyelashes and their location on the user's eyelids, to determine eyelash characteristics such as length and density, and to determine an eyelash extension that is appropriate to be applied to the . The user's eyelids are used to complement natural eyelashes (for example, using machine learning (ML) face or object recognition techniques and / or product recommendation techniques). In some embodiments, the output of the eyelash placement / recommendation engine is presented to a user in a client application that provides functionality for assessing / diagnosing the condition of existing eyelashes and a user interface for selecting eyelash placement strategies to achieve a desired appearance. In some embodiments, options for possible appearances are offered by an eyelash recommendation engine, and a modified version of the user's image may be presented with eyelash recommendations incorporated into the modified image.In some embodiments, a digital twin or virtual 3D model of the user's eyelash or face region(s) is presented in the user interface to enable virtual eyelash placement (e.g., based on a user-selected appearance or an appearance recommended by a recommendation engine), which may allow users to virtually try out different eyelash appearances before performing microrobot operations.

[0024] Figure 1A is a schematic illustration of a non-limiting embodiment of an automated eyelash enhancement system 1 using microrobots, according to various aspects of this disclosure. In some embodiments, components of the system 1 are implemented by a client computer device, a server computer system, or a combination thereof. In the example shown in Figure 1A, digital source image data in the form of one or more digital source images 90 are provided to a face detection module 10, which detects a face in the source image. The face detection module 10 provides facial feature information to an image masking module 20, which calculates a region in the source image 90 in which corresponding image information (e.g., pixel information) is to be masked or removed (e.g., by cropping).In an illustrative implementation, the face detection module 10 includes machine learning (ML) face detection, such as the face detection application programming interface (API) for the necessary ML, available from Google LLC. In the example shown in [Fig. 1A], the facial feature information includes a set of facial landmarks (e.g., points or contours) 14 (e.g., points or contours corresponding to eyelids, eyelashes, eyebrows, or the like). The image mask module 120 uses facial landmarks 14 to compute an image mask region 92, which corresponds to the boundaries of one or more eyelash regions for an eye or eyes. For example, the . Image mask module 120 can identify a region between the lower edge of an eyebrow and an upper eyelid as a lash region. In the example shown in [Fig. 1A], image mask region 92 corresponds to an upper left lash region and an upper right lash region as depicted in source image 90. Once the boundaries of the lash region(s) are determined, system 1 detects and maps individual lashes in the lash map module 34, which generates a lash map. The lash map module can then provide the lash map to the microrobot control module 80 to guide the application of artificial lashes to appropriate locations, as described in more detail here.

[0025] In some embodiments, the lash map module 34 creates skeletons of individual lashes in the defined lash region(s) by determining the proximal and distal endpoints of the lashes as well as their lengths. In an illustrative scenario, the lash map module 34 assumes that the shape of the lashes is an arc of a circle rather than a straight line, and determines the length of individual lashes on this basis. The lash map module 34 can also create boundaries or windows of lash portions, identify the number of lashes present in the set of lashes or in its segments, and calculate characteristics such as lash density (for example, the number of lashes per 5 mm segment or another segment size), average lash length, individual lash lengths, average lash thickness, individual lash thicknesses, etc.All this information, or portions thereof, or additional information, can be included in the lash map in various embodiments. In some embodiments, the lash map module 34 uses machine learning-based face or object recognition techniques to detect and measure eyelashes, or to detect and identify anomalies in lash regions within a lash map (e.g., missing eyelashes, short eyelashes, damaged eyelashes, gaps in eyelashes, etc.). The identified anomalies can be useful for generating corresponding lash enhancement recommendations.

[0026] In some embodiments, the system 1 performs a facial attribute analysis of facial features to determine the application options for artificial eyelashes (for example, to treat a particular condition, such as sparse, irregular, short, or damaged eyelashes, or to achieve a desired appearance). In the example shown in [Fig. 1A], the facial analysis module 30 identifies one or more facial attributes of the subject in the image and proposes the attribute(s) to the eyelash recommendation module 40, which generates an eyelash enhancement recommendation based at least in part on the identified facial attributes. In some embodiments, the facial attributes considered by the eyelash recommendation module in generating a recommendation include one or more shape attributes. facial attributes, age attributes, eye attributes (e.g., shape, size, color), eyebrow attributes (e.g., shape, size, color), a complexion attribute, a skin texture attribute (e.g., wrinkles, firmness), a skin condition attribute (e.g., blemishes, dryness, oiliness, redness), and a hair attribute (e.g., color, texture, length) are considered. In some embodiments, machine learning recommendations (e.g., using an artificial neural network approach) can be used to identify desired eyelash enhancements based on a combination of a user's attributes. In some embodiments, the eyelash enhancement recommendation includes identifying locations on the eyelids where artificial eyelashes should be applied to treat a condition or achieve a desired appearance.

[0027] In some embodiments, the system 1 presents lash recommendations in a user interface, for example for approval by a user or to offer options for further modifications. In the example shown in [Fig. 1A], the system 1 offers a lash enhancement recommendation and information on existing lashes (for example, a lash map generated by the lash map module 34) to the image generation module 42, which generates a modified image 94 based on the source image 90, the information on existing lashes, and the lash enhancement recommendation.In this example, in the modified image 94, the upper lash area in each eye has been supplemented with representations of additional lashes in areas of space between existing lashes on the upper eyelids, resulting in fuller upper lashes, and the modified image 94 is displayed in the lash enhancement user interface 76 (e.g., on a smartphone display or a display of another client computing device).

[0028] In certain embodiments, the system 1 allows a user to provide an additional input to modify or control an eyelash lifting process. In the example shown in [Fig. 1A], a user (for example, a subject in the source image 90 or a salon professional) provides an input via the user interface 76 to select, modify, or confirm an eyelash lifting and / or initiate an eyelash lifting process by sending control signals to the microrobot control module 80, which performs the eyelash application according to one or more techniques described herein.

[0029] Figure 1B is a schematic diagram illustrating non-limiting examples of embodiments of a client computing device 4 according to various aspects of this disclosure. The client computing device 4 can be used to implement one or more aspects of system 1, including all or part of the modules and functionalities shown in Figure 1A, in any combination. In an illustrative scenario, the client computing device 4 captures one or more digital source images of a user, transmits the image data to another computer system (such as a server computer system) for processing, receives modified image data, and implements the eyelash enhancement user interface 76. The client computing device 4 can communicate with other computers or another system using any appropriate communication technology, such as wireless communication technologies, including but not limited to Wi-Fi, WiMAX, Bluetooth, 3G, 4G, 5G, and LTE; or wired communication technologies, including but not limited to Ethernet, FireWire, and USB.

[0030] In the illustrated embodiment, the client computer device 4 includes a camera 50 and a client application 60, which includes an image preprocessing engine 70, a lash enhancement user interface 76, and a communication module 78. The user interface 76 can present various types of features to a user, such as guides, tutorials, or a virtual "try-on" feature to explore new products or appearances. This technology can, in some embodiments, allow users to virtually try on different appearances or products (e.g., eyelashes of different lengths, colors, thicknesses, finishes, etc., or related cosmetics such as mascara or eyeshadow) by applying virtual eyelashes or cosmetics to 2D face images or a virtual 3D model of a user's face.This technology can use source images or user-modified images, which can be generated according to embodiments described herein. In some embodiments, the user interface includes a graphical user interface to help a user obtain high-quality source images on which the modified images can be based.

[0031] In some embodiments, the image preprocessing engine 70 is configured to preprocess images, for example, before they are transmitted to an image processing computer system. In some embodiments, the image preprocessing engine 70 performs image normalization, which may include, for example, color correction, noise reduction or filtering; orientation adjustment; cropping; brightness / exposure adjustment; or contrast adjustment. In an illustrative scenario, an image includes an off-center face where an area of ​​interest, such as the user's eyes, occupies only a small portion of the overall image. To allow for more precise or photorealistic image editing, it may be desirable to reduce the area in the image that is of no interest.This can be accomplished, for example, by using a face detection algorithm to detect the portion of the image that represents the eyes, by centering. The eyes within the image are zoomed in to make them occupy a larger portion of the image. Other possible normalization actions include cropping the image, reducing or increasing the bit depth, downsampling or upsampling image pixels, or similar operations. The product image data can then be sent (potentially along with other information, such as a user ID, device ID, or similar) to a communication module 78 for further formatting and transmission to an image processing system. (Other features of the client computing device 4 are not shown in [Fig. 1B] for ease of illustration.)

[0032] Numerous variations of the arrangements and usage scenarios shown in Figures IA and IB are possible. For example, although the descriptions in Figures IA and IB illustrate various components as being provided by the client computing device 4 or a server computer system, in some embodiments the arrangement or functionality of the components may be different. For example, functionality described as being performed by the client computing device 4 may instead be performed by a server computer system, or vice versa, or such functionality may be performed by different devices or systems.As another example, functionality described as being performed by a particular module or component may instead be performed by a combination of such modules or components, or by a different module or component, or functionality described as being performed by individual modules or components may be combined into a single module or component.

[0033] Figures 2A-2E are examples of 100 eyelash application systems, in accordance with the present technology.

[0034] As shown in [Fig. 2A], the system 100 can include a flexible printed circuit board (PCB) substrate 105, a motor base 110, and one or more linear actuators 115A, 115B, 115C... 115N. In some embodiments, the flexible PCB substrate 105 is configured to bend and / or curve in response to one or more linear actuators 115A, 115B, 115C... 115N (which can here be referred to as "adjustment" of the PCB substrate 105).

[0035] In some embodiments, the motor base 110 is arranged under the flexible PCB substrate 105. The motor base 100 is coupled to one or more linear actuators 115A, 115B, 115C... 115N and is configured to drive and command the linear actuators 115A, 115B, 115C... 115N to move up and down in order to adjust the PCB substrate 105.

[0036] Figure 2B is a top-down view of system 100. The flexible PCB substrate 105 can be arranged above a plurality of linear actuators. 115A, 115B, 115C... 115N. The flexible PCB substrate 105 is shown as dashed lines in Figure 6B to better show the position of the plurality of linear actuators 115A, 115B, 115C... 115N.

[0037] In certain embodiments, the linear actuators 115A, 115B, 115C... 115N are arranged in a network. Each of the linear actuators 115A, 115B, 115C... 115N can move independently, which allows for numerous adjustments to be made to the flexible PCB substrate 105.

[0038] Figure 2C shows the position where the linear actuators 115A, 115B, 115C... 115N have adjusted the flexible PCB substrate 105. During operation, each of the linear actuators 115A, 115B, 115C... 115N moves independently to bend, curve, and otherwise manipulate the flexible PCB substrate 105.

[0039] As shown in [Fig. 2D], one or more microrobots 200A, 200B can slide or levitate above a flexible PCB substrate 105. In some embodiments, the flexible PCB substrate 105 is adjusted before the one or more microrobots 200A, 200B move above it. In other embodiments, the flexible PCB substrate 105 can be adjusted dynamically, that is, while the one or more microrobots 200A, 200B are in motion. In some embodiments, the flexible PCB substrate 105 is configured to adjust the pitch, yaw, roll, or any combination thereof of one or more microrobots 200A, 200B.

[0040] Figure 2E shows microrobots 200A and 200B with applicators. As the microrobots 200A and 200B move across the flexible PCB substrate 105, an angle A, B between the applicator and the flexible PCB substrate 105 changes. In some embodiments, the applicators are arranged at an angle. These angles A, B are called "angles of attack," meaning that the angles A, B allow the applicators to apply an eyelash to an eyelid or an existing eyelash of a user, as shown in Figures 4A-4B. In such embodiments, the flexible PCB substrate is configured to adjust the angle of attack of the applicators.

[0041] Figures 3A-3F are examples of microrobots, in accordance with the present technology.

[0042] Figure 3A shows an example of a microrobot 200 including four magnets 205A, 205B, 205C... 205N. Figure 3B shows an example of a microrobot 200 having magnets 205A, 205B, 205C... 205N, an applicator 210, and an artificial eyelash (or a cluster of eyelashes) L. The applicator 210 is configured to hold an eyelash or a cluster of eyelashes for possible application to an eyelid. As used here, the term "cluster of eyelashes" refers to two or more eyelashes that have been grouped together, either by being manufactured together or by being fixed together, for example with an adhesive. Individual lashes within a cluster of lashes may have identical or different lengths, thicknesses, colors, finishes or similar characteristics.

[0043] Figures 3C-3D show examples of microrobots 200 positioned on or levitating above a printed circuit board (PCB) substrate 105. In some embodiments, the checkerboard arrangement of a plurality of magnets (such as magnets 205A, 205B, 205C... 205N) together with a graphite layer of the substrate 105 confines the microrobot 200 to a specific location at (x, y, z) coordinates in a 3D coordinate space. A magnetic potential well can be generated to locate the microrobot 200. In some embodiments, a magnetic force is generated by four PCB current tracks located inside the substrate 105. [Fig. 3C] shows a sliding substrate system. In such systems, the microrobot(s) 200 is / are configured to glide on the substrate. Figure 3D shows a levitating substrate system.In such embodiments, the microrobot(s) 200 can levitate from an elevation E relative to the substrate 105.

[0044] Figures 3E-3F show various magnet topologies for the microrobots 200. It should be understood that any number of magnets can be included. In some embodiments, the magnets 205A, 205B, 205C... 205N are arranged in an alternating orientation, with the magnetization alternating between adjacent magnets.

[0045] In some embodiments, the microrobot(s) 200 is / are controlled by the local track pattern and currents. That is, the control of the microrobot is based on a region or zone, rather than on something that moves with the microrobot (as would be the case for conventional motorized robots). Zone control has both advantages and disadvantages for multi-agent control. The disadvantage of zone control is that two microrobots in close proximity may not be controlled independently unless they are in different, independent zones. The advantage of zone control is that a large number of microrobots can be controlled to perform the same movement in parallel using only a few control channels.The control zone approach generally reduces the number of control channels required, since microrobots do not need to carry additional control channels in areas that require, for example, only one degree of freedom for transport.

[0046] In certain embodiments, as described herein, the microrobots can be configured to "cooperate" with each other by performing different steps in the eyelash application process on a single eye or a single user with two eyes. For example, one or more microrobots can be configured to separate the eyelashes, another microrobot can be configured to apply the eyelash, and yet another microrobot can be configured to apply eyelash glue or adhesive.

[0047] Figures 4A-4B show another example of a 1000 eyelash application system according to the present technology. In some embodiments, the 1000 system includes a flexible PCB substrate 105, a plurality of linear actuators 115A, 115B, 115C... 115N, a motor base 120, one or more microrobots 200, a chin rest 720, one or more cameras 1005A, 1005B, 1005C (which can be collectively described as a camera system), and a processor 1010. A user 300 can operate the 1000 system shown in Figures 4A-4B.

[0048] Although three cameras are shown in Figures 4A-4B, it should be understood that any number of cameras can be used as a camera system, including a single camera. In some embodiments, the cameras 1005A, 1005B, 1005C are configured to capture the user 300, one or more microrobots 200, a flexible PCB substrate 105, or other components or subjects within their field of view. In some embodiments, the cameras 1005A, 1005B, 1005C are coupled in communication to a processor 1010. The processor 1010 can be configured to determine a desired position (e.g., a height) of the linear actuators 115A, 115B, 115C... 115N, a new position of one or more microrobots, or both.System 1000 can use cameras 1005A, 1005B, and 1005C to determine the locations of components such as the flexible PCB substrate 105, one or more microrobots 200, the applicator 210, or any combination thereof with the processor 1010. Although the processor 1010 is shown to be local to other components of System 1000, it should be understood that the processor 1010 can be located anywhere, including at a remote location connected to other components via a network, or incorporated directly into the system. In some embodiments, the processor 1010 can be embedded in a remote device, such as a smartphone, desktop computer, laptop, or tablet. In some embodiments, the processor 1010 is configured to communicate with one or more cameras 1005A, 1005B, 1005C, the motor base 120, one or more linear actuators 115A, 115B, 115C...115N, and / or one or more 200 microrobots.

[0049] In some embodiments, the positioning of one or more microrobots 200 on the flexible PCB substrate 105 includes instructing one or more microrobots 200 to slide or levitate above the flexible PCB substrate 105. In some embodiments, the positioning includes pitch, yaw, roll, or a combination thereof adjustment of one or more microrobots 200 with one or more linear actuators 115A, 115B, 115C... 115N under the flexible PCB substrate 105. Adjusting the position of one or more microrobots 200 one or more of the methods described here allows the 1000 system to apply an eyelash with one or more microrobots.

[0050] In operation, as shown in [Fig. 4A], the user 300 can rest their chin on the chin rest 720. In some embodiments, the chin rest 720 may be omitted. In some embodiments, the chin rest 720 is adjustable to position the eyes of the user 300 so that one or more microrobots 200 can come into contact with the user's eyelash line, eyelid, or similar features. One or more cameras 1005A, 1005B, 1105C can monitor the position, orientation and / or angle of one or more microrobots 200. One or more cameras 1005A, 1005B, 1005C can also determine a location of user features 300 such as eyelash lines, eyelashes and eyelids, for example, using a face detection technique.

[0051] In some embodiments, a first camera 1005A is positioned to view the flexible PCB substrate 105 from top to bottom (or from a "bird's-eye" view), a second camera 1005B is positioned to view the flexible PCB substrate 105 from an angle, and a third camera 1005C is positioned to view the flexible PCB substrate 105 from the side. However, those skilled in the art will recognize that the camera system can be arranged in other configurations.

[0052] In some embodiments, one or more cameras 1005A, 1005B, 1005C transmit image data from one or more microrobots 200, a flexible PCB substrate 105 and / or a user 300 to the processor 1010, which can then analyze this image data and adjust a position (e.g., height) of at least one actuator 115A, 115B, 115C... 115N and / or adjust a position of one or more microrobots 200. In some embodiments, adjusting the position of one or more microrobots 200 includes adjusting an angle of attack of the applicator 210 with the linear actuators 115A, 115B, 115C... 115N.

[0053] In certain embodiments, as shown in [Fig. 4B], the processor 1010, together with the flexible PCB substrate 105, can position a first microrobot 200A to apply a first eyelash L1 and position a second microrobot 200B to apply a second eyelash L2. In such embodiments, the first microrobot 200A and the second microrobot 200B can move along the flexible PCB substrate in direction D, applying eyelashes to different locations on the left eye of the user 300. In some embodiments, the microrobots 200A and 200B can apply eyelashes simultaneously. For example, the first microrobot 200A can apply eyelashes to an eyelid of a first eye and the second microrobot 200B can apply eyelashes to an eyelid of a second eye. In some embodiments, the flexible PCB substrate 105 can adjust the position of one or more microrobots 200A, 200B so that the applicators 21 OA, 210B are able to apply eyelashes or tufts of eyelashes to an upper lash line, a lower lash line, or both.

[0054] During operation, the user 300 or a salon professional can select a lash style, such as with the lash lift user interface 76 of the application 60 running on the client computer device 4 (see [Fig. 1B]). In some embodiments, the client computer device 4 houses the processor 1010. The user 300 or a salon professional can modify their selection, view their selection in a photo or live feed, or receive a selection based on a questionnaire, personal preferences, or trending styles. After selecting the style and / or providing other user input, the user 300 can then sit or stand at the system 1000. The processor 1010 can then instruct one or more microrobots 200 to apply one or more lashes or one or more clusters of lashes to achieve the selected style.One or more cameras 1005A, 1005B, 1005C monitor one or more microrobots 200 and the user 300 to ensure that the microrobots are directed to a location to apply one or more eyelashes or one or more tufts of eyelashes to achieve the selected style. In some embodiments, the processor 1010 further instructs one or more linear actuators 115A, 115B, 115C... 115N to adjust the flexible PCB substrate 105 to position one or more microrobots 200.

[0055] Figure 5 is a flowchart of a method 500 for controlling one or more microrobots to apply eyelash extensions. In some embodiments, the method 500 is carried out by the system 1 shown in Figure 1A, or by another device or system.

[0056] In block 502, the system obtains digital source image data of a subject (for example, one or more digital images of user 300 shown in [Fig. 4A]-4B). In block 504, the system defines a lash region of the subject in the digital source image data. In block 506, the system generates a lash map based at least in part on an analysis of the defined lash region. In block 508, the system generates microrobot control instructions based at least in part on the lash map, wherein the microrobot control instructions are configured to cause one or more microrobots to apply one or more artificial eyelashes to the human subject based on the lash map.

[0057] Figure 6 is a schematic diagram of an example of a sequence of 600 eyelash application operations, according to the present technology. The sequence of 600 operations can be carried out by a single device or system or a combination of systems, such as system 1000 (see Figures 4A-4B).

[0058] At block 605, the system is calibrated. In some embodiments, this is done only once. In some embodiments, calibration includes determining the initial position of a flexible PCB substrate (such as the flexible PCB substrate 105), one or more microrobots (such as the microrobots 200A, 200B), and / or a subject (such as the user 300). In some embodiments, calibration includes calibrating a camera system, which may include, for example, correcting optical distortion, lens aberrations, and the like.

[0059] At block 610, stereo image data is captured. In some embodiments, this is done with a camera system including one or more cameras. In some embodiments, the stereo image data allows for the determination of depth information and can be used to create a snapshot of the lash region. In some embodiments, the stereo image data is captured in a controlled lighting environment to avoid problems related to capturing accurate representations of the lash region due to poor lighting conditions (for example, conditions that are too dark or that involve inconsistent light sources).

[0060] The captured image data can be used to detect microrobots and their corresponding positions, as well as to detect regions of cilia and create maps of existing cilia. In the example shown in [Fig. 6], at block 615, one or more microrobots are detected in the stereo image data. This allows the system to determine and receive position data for the microrobot (e.g., in the form of 3D coordinates) (block 620). At block 625, a dense disparity map of the stereo images is generated to match pixels in the stereo images. At block 630, 3D region pixels are acquired.

[0061] Referring again to block 610, after obtaining the stereo images, image data can be used to detect eyelash regions and individual eyelashes and to create maps of existing eyelashes. In the example shown in [Fig. 6], in block 635, face detection is performed. For example, facial landmarks (e.g., the contours of the eyelids) can be detected in the stereo images. In block 640, an image mask is applied to define the boundary of one or more eyelash regions (e.g., an eyelash region of one eye or eyelash regions for both eyes of a subject). An eyelash map is created.

[0062] In block 650, a lash region (or a portion thereof) is selected. For example, the eyelid within a lash region is divided into 5 mm segments, and an initial segment for lash application is selected.

[0063] In block 655, two-dimensional (2D) pixel regions of the eyelid are acquired. In conjunction with the 3D pixel region of the desired position of the microrobot(s) (block 630), microrobot control instructions can be generated, and instructions can be executed to make the microrobot(s) move to a correct region in block 660.

[0064] In block 665, the cilia can be separated by the microrobot(s). In some embodiments, the cilia are then visualized again by returning to block 610 for further processing. The images of the separated cilia can thus undergo the same processing steps in blocks 610 to 655.

[0065] At block 670, the length of the eyelashes can be estimated (for example, on the basis of detected proximal and distal endpoints of individual eyelashes). At block 675, the extension (eyelashes and / or eyelash tuft) is applied (for example, by executing corresponding microrobot control instructions generated by system 1 ([Fig. 1A]).

[0066] Although illustrative embodiments have been shown and described, it will be appreciated that various changes can be made to them without departing from the spirit and scope of the invention.

[0067] This application may refer to quantities and numbers. Unless otherwise specified, these quantities and numbers are not to be considered restrictive, but rather representative of the possible quantities or numbers associated with this application. Similarly, in this regard, this application may use the term "plurality" to refer to a quantity or number. In this regard, the term "plurality" is understood to mean any number greater than one, for example, two, three, four, five, etc. The terms "about," "approximately," "near," etc., mean plus or minus 5% of the stated value. For the purposes of this disclosure, the expression "at least one of A, B, and C," for example, means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C), including all other possible permutations when more than three items are listed.

[0068] The embodiments disclosed herein may use circuitry to implement the technologies and methodologies described herein, functionally connect two or more components, generate information, determine operating conditions, control an apparatus, device, or process, and / or the like. Any type of circuitry may be used. In one embodiment, the circuitry includes, among other things, one or more computing devices such as a processor (e.g., a microprocessor), a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like, or any combination thereof, and may include separate digital or analog circuit elements or electronics, or combinations thereof.

[0069] An embodiment includes one or more data stores that, for example, store instructions or data. Non-limiting examples of one or Multiple data stores include volatile memory (e.g., random access memory (RAM), dynamic random access memory (DRAM), or the like), non-volatile memory (e.g., read-only memory (ROM), electrically erasable read-only memory (EEPROM), compact disc read-only memory (CD-ROM), or the like), persistent memory, or the like. Other, non-limiting examples of one or more data stores include erasable read-only memory (EPROM), flash memory, or the like. One or more data stores can be connected, for example, to one or more computing devices by one or more instruction, data, or power buses.

[0070] In one embodiment, the circuitry includes a computer-readable media player or a memory slot configured to accept a signal-carrying medium (for example, computer-readable memory storage, computer-readable recording storage, or the like). In one embodiment, a program intended to cause a system to perform any of the disclosed processes may be stored, for example, on computer-readable recording storage (CRMM), a signal-carrying medium, or the like.Non-limiting examples of signal-carrying media include recordable media such as any form of flash memory, magnetic tape, floppy disk, hard drive, compact disc (CD), digital multipurpose disc (DVD), Blu-ray disc, computer memory, or the like, as well as transmission media such as digital and / or analog communication media (e.g., fiber optic cable, waveguide, wired communication link, wireless communication link) (e.g., transmitter, receiver, transceiver, transmission logic, receiving logic, etc.). Other non-limiting examples of signal-carrying media include, but are not limited to, flash memory, magnetic tape, magneto-optical disc, non-volatile memory card, EEPROM, optical disc, optical storage, RAM, ROM, system memory, or the like.

[0071] The detailed description presented above in relation to the accompanying drawings, where similar numbers refer to similar elements, is intended to be a description of various embodiments of this disclosure and is not intended to represent the only embodiments. Each embodiment described in this disclosure is offered solely by way of example or illustration and should not be construed as being preferred or advantageous over other embodiments. The illustrative examples provided herein are not intended to be exhaustive or to limit the disclosure to the specific forms disclosed. Similarly, all the steps described herein may be interchangeable with other steps, or combinations of steps, to achieve the same or substantially similar result. In general, the embodiments disclosed herein are not exhaustive, and the The inventors anticipate that other embodiments within the scope of this disclosure may include structures and features from more than one specific embodiment shown in the figures and described in the patent memorandum.

[0072] In the preceding description, specific details are presented to allow for a thorough understanding of examples of embodiments of this disclosure. However, it will be apparent to those skilled in the art that the embodiments described herein can be implemented without incorporating all the specific details. In some cases, well-known process steps have not been described in detail so as not to unnecessarily obscure various aspects of this disclosure. Furthermore, it should be noted that the embodiments of this disclosure may employ any combination of features described herein.

[0073] This application may include references to directions, such as "vertical", "horizontal", "front", "back", "left", "right", "up" and "down", etc. These references, and other similar references in this application, are intended to help describe and understand the particular embodiment (such as when the embodiment is positioned for use) and are not intended to limit this disclosure to those directions or locations.

[0074] This application may also refer to quantities and numbers. Unless otherwise specified, these quantities and numbers are not to be considered restrictive, but rather as examples of the possible quantities or numbers associated with this application. Similarly, in this respect, this application may use the term "plurality" to refer to a quantity or number. In this respect, the term "plurality" is understood to mean any number greater than one, for example, two, three, four, five, etc. The terms "about," "approximately," etc., mean to within 5% of the stated value. The term "based on" means "based at least partially on."

[0075] The principles, representative embodiments, and modes of operation of this disclosure have been described in the preceding description. However, aspects of this disclosure that are intended to be protected should not be interpreted as being limited to the particular embodiments disclosed. Furthermore, the embodiments described herein should be considered illustrative rather than restrictive. It should be understood that variations and changes may be made by other means, and equivalents may be employed, without departing from the spirit of this disclosure. Accordingly, it is expressly intended that all such variations, changes, and equivalents fall within the scope of this disclosure.

Claims

Demands

1. A computer-implemented method for controlling one or more microrobots to apply eyelash enhancements, the method comprising: obtaining digital source image data of a subject; defining an eyelash region of the subject in the digital source image data; generating an eyelash map based at least in part on the analysis of the defined eyelash region; and generating microrobot control instructions based at least in part on the eyelash map, wherein the microrobot control instructions are configured to cause one or more microrobots to apply one or more artificial eyelashes to the subject based on the eyelash map.

2. A method according to claim 1, wherein the definition of the eyelash region includes obtaining facial landmarks from the digital source image data, identifying the location and shape of the eyelash region based on the facial landmark points, and applying an image mask corresponding to the eyelash region to the digital source image data.

3. A method according to claim 2, wherein the facial markers include eye dots or contours or eyebrow dots or contours.

4. A method according to claim 1, further comprising: supplying the eyelash map to an eyelash recommendation engine; and by the eyelash recommendation engine, generating an eyelash recommendation based at least in part on the eyelash map, wherein the eyelash recommendation includes a position on an eyelid or an existing eyelash of the subject for an artificial eyelash to be applied by one or more microrobots.

5. A method according to claim 4 further comprising performing an attribute analysis on the digital source image data to identify one or more subject attributes, wherein the eyelash recommendation is further based on the one or more attributes.

6. A method according to claim 5, wherein the one or more attributes include one or more of a face shape attribute, an age attribute, an eye attribute, an eyebrow attribute, a complexion attribute, a skin texture attribute, a skin condition attribute, a hair attribute.

7. Method according to claim 4, wherein the microrobot control instructions are further based on the recommendation of eyelashes.

8. A method according to claim 4, further comprising: providing digital source image data and eyelash recommendation to an image generation module; and generating a modified image or 3D model based on the digital source image data and eyelash recommendation.

9. A method according to claim 8, further comprising: displaying the modified image or 3D model in a lash enhancement user interface; wherein the lash enhancement user interface further includes a virtual try-on feature that allows the lash recommendation to be modified via user interaction with the modified image or 3D model.

10. A method according to claim 1, further comprising receiving user input from an eyelash enhancement user interface, wherein the microrobot control instructions are further based on the user input.

Images