System and method for automated application of a cosmetic product to a user

Abstract

The present patent of invention pertains to the technical field of portable devices or items for cosmetic treatments involving the application of fluid substances to the lips, including the identification and processing of image data, and relates more specifically to a system and method for automated application of a cosmetic product to a user using artificial intelligence, which recognises the lips by means of an image and automatically applies lipstick to the user's lips with precision, essentially by means of a robotic applicator device (2) operating along a Z-axis via a CNC system, a computer vision camera (3) for capturing an image and a corresponding depth map in order to obtain 3D mapping of the lips and convert this to the 3D perspective of said robotic applicator device (2), and multisensory devices for interaction with the user for controlling and actuating said robotic applicator device (2).

Description

“SYSTEM AND METHOD FOR AUTOMATIC APPLICATION OF COSMETIC PRODUCTS TO THE USER”

[0001] The present invention patent belongs to the technical field of portable devices or articles for cosmetic treatments with the application of fluent materials to the lip surface, including the identification and processing of image data, more specifically it is a system and method for the automatic application of a cosmetic product to a user with artificial intelligence, which recognizes the lip through an image and automatically applies lipstick to the user's lips with precision. STATE OF THE TECHNIQUE

[0002] Various devices and methods for automatically applying cosmetic products to users are known in the prior art. For example, patent documents BR102020022162-0 and BR102022004560-7, both owned by the applicant of the present invention, disclose, respectively, a method for detecting and segmenting the lip region, and a device and method for automatically applying cosmetic products to users.

[0003] Among other documents in the patent literature, publication number IN202111050964 describes an automatic makeup application device, including an imaging unit that captures the image of the user's lips, providing selection suggestions regarding the color of the lip pigment and the designs stored in the database,

[0004] Patent document CN112643691 describes an intelligent automatic makeup and skincare robot device comprising a touchscreen, a storage module, an automated robot module, a high-definition camera, a 3D scanner, a light sensor, and an infrared scanner. The publication also describes the presence of a calculation module, a communication module, a storage module, a control module, a voice recognition module, and a sound system integrated into the touchscreen.

[0005] Finally, patent document US11568675 presents systems and methods for automated makeup application that allows a user to select and apply desired makeup styles to the user's face. The systems and methods include a computer application with a graphical user interface that allows the selection of a look from a plurality of pre-configured looks. A camera attached to a robotic arm records a facial map and color coding and sends this data to be stored in a virtual server database. Technical problem

[0006] Thus, although the state of the art provides for methods of image processing of the lip region, as well as a device for moving and automatically applying lipstick to a user using CNC-guided mechanisms, it lacks a more improved solution that uses a system for more precise automatic application of cosmetic product to the user, essentially comprising a lipstick applicator with resilience characteristics, a continuous liquid lipstick dispensing mechanism adjacent to said applicator, the pressure of which applied to the user's lip is measured and controlled, in addition to other multisensory interaction components. TECHNICAL SOLUTION

[0007] The present invention presents a system for automatically applying cosmetic products to the user, ensuring more precise application, with easier lip contouring, in addition to a surprising and pleasant user experience.

[0008] The present invention provides an improvement in the cosmetic product applicator with greater precision on the Z axis through computer vision, in addition to a dispenser that ensures a constant flow of raw material at the applicator tip.

[0009] The present invention provides an improvement in the cosmetic product applicator by means of a resilient applicator tip and whose applicator device has a controlled pressure for applying lipstick to a user's lip.

[0010] The present invention provides an improvement in 3D lip mapping with AI by combining 2D lip segmentation with captured depth map by the computer vision camera.

[0011] Additionally, the present invention provides greater possibilities for inclusion for people with special needs due to the multisensory feedback made possible by visual (LEDs), sensory (embossed buttons), and auditory components present in the automatic lipstick application device. BRIEF DESCRIPTION OF THE FIGURES

[0012] The following description is not limited to the drawings or components cited, but is based on the illustrations referenced below.

[0013] Figures IA and 1B show views of the system for automatically applying cosmetic products to the user, according to the present invention.

[0014] Figure 2 shows the system for automatically applying cosmetic products to the user, according to the present invention, highlighting the computer vision camera.

[0015] Figure 3 shows images of a three-dimensional representation of the lips, composed of a 3D point cloud of the lips obtained in one of the application tests of the present invention.

[0016] Figure 4 shows a diagram of the lip touch detection system, as specifically applied to a mannequin.

[0017] Figure 5 shows test images of a setup for calculating the spring constant.

[0018] Figure 6 shows the applicator device and the lipstick wear area on the applicator tip.

[0019] Figure 7A shows an image of some samples exemplified in testing for choosing FSR sensors.

[0020] Figure 7B shows an image of the samples exemplified in testing, including 3D cases encapsulating the test FSR sensors.

[0021] Figure 8 shows a simplified image of a test setup to verify the sensitivity of the application FSR sensor.

[0022] Figure 9 shows a setup for carrying out tests with a barometric pressure sensor.

[0023] Figure 10 shows the system for automatically applying a cosmetic product to a user, highlighting an applicator tip present in the robotic applicator device, and a lipstick dispensing device connected to said applicator tip, with the purpose of automatically and continuously moistening said lipstick applicator tip.

[0024] Figure 11 shows an image of an extruder component present in the automatic cosmetic product application system for the user.

[0025] Figure 12 shows the system for automatically applying a cosmetic product to a user, highlighting the presence of a touch sensor component adjacent to the front surface of the housing structure for the components of said system for automatically applying a cosmetic product to a user. DETAILED DESCRIPTION

[0026] As seen in figures IA and IB, the present invention presents a system for automatically applying a cosmetic product to a user (1), comprising a new robotic application device (2), application score, multisensory devices for interacting with the user and computer vision using a computer vision camera (3) that uses 3D mapping with image capture of the lips by AI.

[0027] More specifically, by means of the present invention it is possible to apply lipstick automatically, by determining the positioning of the lip in relation to the robotic application device (2).

[0028] Furthermore, the present invention uses the computer vision camera (3), as seen in figure 2, to capture a color image and a corresponding depth map in order to obtain a 3D mapping of the lip, in which, using Artificial Intelligence (AI), the 2D segmentation of the lip is combined, which is detected in the color image, with a depth map captured by the computer vision camera (3), and once the lip position is obtained, a conversion to the 3D perspective of the robotic application device is performed.

[0029] To determine the depth map, initially the vision camera computational (3), for example, Intel RealSense D405 model, is capable of capturing both the color image and the depth map. For this purpose, a library such as pyrealsense2 is used, for example, to perform essential operations.

[0030] First, a color image of the lip is captured, and the depth map is aligned to it. Subsequently, spatial and temporal filters are applied to smooth and stabilize the depth map, improving the quality of the data obtained.

[0031] In turn, the 3D mapping of the lip is obtained by combining the 2D segmentation with the depth map. Initially, the lip's X and Y pixel coordinates are obtained from this color image. Then, for each of these pixels, the corresponding Z coordinate in millimeters is extracted from the depth map.

[0032] The conversion of X and Y coordinates in pixels to millimeters is performed using the intrinsic depth parameters of the computer vision camera, for example, the Intel RealSense D405, parameters that are obtained using the library, for example, pyrealsense2. This allows us to obtain the X, Y and Z coordinates in millimeters, referring to the coordinate system of the depth camera.

[0033] This approach provides a three-dimensional representation of the lips, as illustrated in the images in Figure 3. In this figure, the 3D point cloud of the lips obtained in one of the application tests is presented, demonstrating the effectiveness of the method used.

[0034] Additionally and in an unusual way, the present invention also presents a lip touch detection system, as specifically applied to a mannequin, shown in the diagram in figure 4, essentially comprising a fixed mannequin (4), a computer vision camera (3) for mapping the X, Y and Z axes and capturing 3D images, a robotic application device (2) using a convolutional neural network (CNC), a Z-axis sensor (5) that reads application parameters, and a mannequin with lipstick applied (6).

[0035] The structure and essential electromechanical functioning of the robotic applicator device (2) by convolutional neural network (CNC) for executing the present invention can be understood, for example, from the understandings arising from a person skilled in the art, provided by reading the teachings disclosed in patent document BR102022004560-7, of the same ownership as the applicant of the present invention.

[0036] To achieve the surprising effect of the present invention, it was necessary to choose the type of sensor to use and the range of values ​​that they should work with to make it possible to have a lipstick application responsive to the pressure applied to the user's lip, with the variable of interest being the pressure on the lip when applying lipstick.

[0037] To determine the variable of interest, lip pressure when applying lipstick, several scientific articles were analyzed to define the necessary operating range, involving parameters of Lip pressure, Lipstick application, Lip cosmetics, Lipstick pressure, Lip force, Lip make-up, Lipstick technique, Lipstick deposition, Lipstick wear, and Lipstick adherence.

[0038] For example, the article "A pressure-distribution sensor (PDS) for evaluation of lip functions, 1996" focuses on the development and application of a pressure sensor to analyze the forces and pressure patterns applied to the lips. In the experiments described in this article, the pressures used ranged from 1 to 2 g / mm. 2 , and with this result we have that pressures in this range are not inconvenient for the user.

[0039] In turn, the study presented in "Lip pressure at rest and during function in 2 groups of patients with different occlusions, 2011" analyzed the influence of perioral muscles on tooth position, specifically the forces exerted by soft tissues at rest and during swallowing. Lip pressure was measured using a force-sensitive device placed midline between the upper incisors. Measurements revealed that the mean lip pressure at rest was 24.59 g / cm³. 2and during swallowing it was 24.87 g / crn 2 .

[0040] The development of technologies to measure lip pressure during The performance of wind instruments such as trumpet and cornet is addressed in the article "Sensor and Software Technologies for Lip Pressure Measurements in Trumpet and Cornet Playing - from Lab to Classroom, 2015." The calibration method used for the sensor system demonstrated measurement accuracy with a deviation of less than 5% and a measurement range from 0.6 N to a maximum load of 70 N.

[0041] A study aimed at testing a new sensor to assess lip function is presented in the article “Development of a novel composite sensor for evaluating lip function, 2018.” The sensor, called a composite sensor (CS), was used to measure electromyographic (EMG) activity of the superior orbicularis oris muscle (OOM), lip closure pressure (LP), and intraoral baro-pressure (IP) in 20 healthy individuals (10 women and 10 men). Participants performed lip closure, blowing, and sucking tasks while EMG signals from the upper and lower OOM were recorded with conventional electrodes to assess the accuracy of the CS. Mean lip closure pressures during maximal lip closure, blowing, and sucking ranged from 2 to 6 kPa for women and from 5 to 7 kPa for men. The corresponding intraoral pressures were evaluated as 0.0 ± 0.5, 3.2 ± 1.4, and -4.4 ± 2.6 kPa for women, and -0.5 ± 1.4, 4.9 ± 1.8, and -5.6 ± 2.8 kPa for men.

[0042] Furthermore, in the article “Lip pressure and resistance in subjects with and without occlusal alterations, 2019”, participants were classified according to malocclusion as Class I, Class II, and Class III. A general decrease in lip pressure values ​​was observed in all groups, including the group with normal occlusion, when compared to the reference values ​​of the Iowa Oral Performance Instrument (IOPI) manual, presenting the following mean lip pressure values: Class I group (women): Mean of 20.75 kPa, minimum of 10 kPa, and maximum of 29 kPa; Class I group (men): Mean of 29.50 kPa; minimum of 16 kPa, and maximum of 48 kPa; Class II group (women): Mean of 16.80 kPa, minimum of 2 kPa, and maximum of 28 kPa; Class III Group (women): Average of 15.55 Kpa, Minimum of 4 Kpa, and Maximum of 36 Kpa; and Class III Group (men): Average of 16.29 Kpa, Minimum of 2 Kpa, and Maximum of 51 Kpa.

[0043] Thus, from the results of this study, it is possible to identify that the sensor's operating range must operate at a minimum of 50 kPa, which refers to a lip pressure that causes injuries in the experiments.

[0044] There is a wide variety of sensors available on the market today, which can be broadly categorized according to the type of measurement (physical quantity), the detection principle employed and the output signal.

[0045] However, this invention provides technology that can extract the pressure value from a system composed of a device with an applicator tip pressed against a human lip.

[0046] In the present invention, two methods are used to measure a physical quantity: direct measurement and indirect measurement. Direct measurement occurs when the sensing element is sensitive to the quantity of interest, while indirect measurement occurs when the sensing element is sensitive to another quantity, but due to some specific arrangement, this other quantity and the quantity of interest are correlated in a known way.

[0047] In the direct methodology, sensors of three types of measurements are applied: - Absolute: measured in relation to a perfect vacuum, that is, it is the difference in pressure at a given measurement point by the vacuum pressure (absolute zero); - Gaussian: measured in relation to the ambient pressure, that is, in relation to the atmosphere, that is, it is the difference between the absolute pressure measured at any point and the atmospheric pressure; and - Differential: difference in pressure measured between two points, in which when any point other than vacuum or atmosphere is taken as a reference it is said to measure differential pressure.

[0048] As an indirect measurement, a mass-spring system is envisaged, and for this arrangement a force sensor is used that performs: - the measurement of the force that the mass exerts against the spring; and - the measurement of the force that the spring exerts against the wall.

[0049] For this arrangement, it is also possible to use a displacement sensor and measure the compression and extension of the spring.

[0050] For the system and method of automatically applying cosmetic products to a user according to the present invention, a direct and absolute pressure sensor was chosen. This choice is due to the fact that the pressure of interest refers to the difference between the moment an applicator tip is in contact with the lips and the moment the applicator tip is not in contact with the lips. Additionally, and as an aid, an indirect force sensor was also chosen.

[0051] After understanding the technologies available on the market, it is necessary to estimate the sensitivity required for the automatic application of cosmetic products to the user. To this end, several testing efforts were carried out in the laboratory to achieve the desired technical effect.

[0052] In this sense, it is necessary to calculate the maximum and minimum pressure for applying lipstick to the chosen sensor, whose formula for calculating pressure is given by Equation 1, in which P = pressure, F = Force and A = Area: P = F / A (Equation 1)

[0053] To estimate the spring's spring constant, the force calculation required finding the spring constant (k) of the spring attached to the smart lipstick applicator. To do this, loads (m) of known values ​​were applied, and the spring's displacement (x) was verified in each situation, as shown in the images in Figure 5, which shows a setup for calculating the spring constant.

[0054] Thus, according to the formula for the elastic force Fe = kx, with m representing the known masses mg = kx, g the acceleration of gravity is 9.8 m / s 2 , 3 elastic constants were calculated and from an average the estimated value was k = 21.3 kg / s 2 .

[0055] With the spring constant calculated, the next step is to calculate the minimum and maximum force when applying lipstick. To do this, for example, four people applied lipstick to the mannequin with the lipstick attached to a spring. Through the maximum and minimum compression of the spring, it was possible to calculate the respective forces (Fe = kx): Fmin = 21.3 • 0.001 = 21.3 nN (Equation 2); and Fmax = 21.3 • 0.005 = 106.5 nN (Equation 3)

[0056] Finally, from these obtained force values, where P = F / A, it is possible to finally calculate the maximum and minimum pressure that the sensor should measure. To do this, the contact area of ​​the lipstick when applying it to the mannequin is used, as shown in figure 6, which shows the applicator device (2) and the lipstick wear area (7) on the applicator tip (9): Pmin = 21.3 / 16- IO -6 m 2 = 1.33kPa (Equation 4); and Pmax = 106.5- 10' 3 N / 16- 10' 6 m2 = 6.65 kPa (Equation 5)

[0057] In addition to the sensor range and sensitivity criteria, a third selection criterion was added to the research: the nature of the output signal. Priority is given to sensors with digital signal output to facilitate implementation and because this signal will be used as feedback for the system, which requires a high response speed. However, sensors with analog output signals are not completely ruled out and are also considered for testing.

[0058] Regarding the tests with the chosen sensors, the FSR (Force-Sensing Resistor) sensor is a device used to measure the force or pressure applied to a surface. It consists of a pressure-sensitive material that varies its electrical resistance according to the force exerted on it. Tests begin with this FSR sensor, as shown in some samples shown in Figure 7A.

[0059] Additionally, in order to achieve a uniform pressure reading on the sensors, 3D cases were designed, which are designed and manufactured to encapsulate the test FSR sensors, as shown in Figure 7B.

[0060] However, this method of pressure measurement can be inefficient, since the cover area, by occupying the entire surface of the pressure sensor, does not provide sufficient signal outputs to be adequately interpreted, in which, only under extremely high manual pressures can the output values ​​be read satisfactorily.

[0061] Additionally, it was found that the larger model sensor, represented by Table 2, does not fit into the extrusion Z applicator arm, and consequently, this geometry is discarded.

[0062] After analyzing the behavior of FSR sensors of different sizes, including the medium-sized model (represented by Table 3) and the smaller-sized model (represented by Table 4), the medium-sized FSR sensor model was chosen for use in the automatic lipstick application system, according to the present invention.

[0063] Thus, a test setup was built to verify the sensitivity of the FSR application sensor. Figure 8 shows the planned setup, which comprises the FSR sensor (10), a hot glue piece point (11), a mobile plunger (8), a flexible elastic component (12) (such as a spring), preceding the applicator device (2) with an applicator tip (9).

[0064] To enable the testing process, modifications to the Z-axis system were necessary to allow for the integration of the FSR sensor. Additionally, a plunger that exerts pressure on the FSR sensor, allowing the applied force to be measured, was also necessary.

[0065] Initially, it is noted that the plunger applies a very low pressure, insufficient for it to be detected by the sensor. To solve this problem, the strategy adopted was to reduce the contact area between the plunger and the sensor, using the hot glue piece (11), for example, with an area of ​​approximately 25 mm 2 .

[0066] The test was conducted as follows: the Z axis of the CNC system is coupled and integrated with the functionality of a repetier host. Through this control, the applicator tip (9) is gradually brought closer to the lips of the test dummy.

[0067] During testing, a reference state was established for "No Pressure," when there is no contact with the lips. When the applicator tip makes gentle contact with the mannequin's lips, "Light Touch" was defined. From this point on, when the pressure applied is increased, the "Soft Squeeze" setting is used. Values ​​that should be avoided were categorized as "Moderate Squeeze." The pressure values ​​found are presented in Table 5.

[0068] A second sensor option is considered for the present invention comprising: a barometric pressure sensor, such as with I2C communication, with the ELVH-B002A-HRNJ-C-N2A4 and WSEN-PDUS sensors being acquired, whose technical characteristics are presented in Table 6.

[0069] To perform the tests, a test setup (13) was constructed with a barometric pressure sensor of greater complexity than the FSR sensor, as shown in figure 9, which comprises a pressure sensor (14), hose (15), syringe (16) preceding the application device (2).

[0070] To carry out the tests, the following methodology is adopted, in which the characterization of the two sensors is carried out, whose test setup comprises: - connect the system to the syringe with the plunger in the 2.5ml position and note the value read by the sensor; - press the plunger to go to the 2.0ml position and note the value read by the sensor; - press the plunger to go to the 1.0ml position and note the value read by the sensor; - press the plunger to go to the 0.5ml position and note the value read by the sensor; - relax the plunger so that it returns to the original position of 2.5ml, in this case it is necessary to pull it all the way out so that it removes some of the internal pressure, as in the 0.5ml position some of the pressure escapes, do this until the relaxed system is at 2.5ml; - pull the plunger to go to the 3.0ml position and note the value read by the sensor; - pull the plunger to go to the 4.0ml position and note the value read by the sensor; and - pull the plunger to go to the 5.0ml position and note the value read by the sensor.

[0071] Based on the test results, it can be concluded that the setup can operate within an operating range of 0.6 bar to 1.8 bar. Exceeding these limits can cause seal failure, resulting in the system losing pressure when it reaches a value above 1.8 bar or gaining pressure when it falls below 0.6 bar. With the syringe used and the system starting from latm, the plunger travels 1.5 cm.

[0072] According to the present invention, and as observed in figure 10, the system for automatically applying cosmetic product to the user (1), which in addition to allowing for improved precision of the Z axis through computer vision provided by the computer vision camera (3), also comprises a resistive sensor capable of measuring and providing control over the degree of pressure exerted by the applicator tip (9) during the application of lipstick to the lip, performed by the robotic applicator device (2), which is responsive to touch on the lip.

[0073] The surprising robotic applicator device (2) presented by the present invention further comprises the applicator tip (9) as a resilient component, and a spring structure that allows the resilient movement of said applicator tip (9), while signaling to the control module of the automatic cosmetic product application system in the user (1), the variation in the application pressure that occurs when the applicator tip (9) touches the lip of a user, and whether or not it is necessary to correct this pressure application value.

[0074] In order for the automatic cosmetic product application system to the user (1) to provide a more precise application of the lipstick to the user's lip, the applicator tip (9) may also comprise a flocked cushion tip.

[0075] In addition to the applicator tip (9) of the automatic cosmetic product application system for the user (1), a lipstick dispensing device (17) is also provided, connected to said applicator tip (9), for the purpose of automatically and continuously moistening said lipstick applicator tip (9).

[0076] Thus, as seen by the various and exhaustive tests carried out to achieve the automatic user application system for cosmetic products (1) of the present invention, it resulted from meticulous and well-structured methodologies, integrating various stages of research, development and evaluation, starting with comprehensive preliminary studies to explore the variety of lipstick applicators available on the market, identifying trends, innovative features and points of improvement to be solved by the present invention.

[0077] Based on the insights gained prior to the development of the present invention, prototypes were developed and structured within the mechanical applicator arm, containing the applicator tip, compliant Z, spring, plunger, slot for the pressure sensor (FSR) and the liquid lipstick extrusion duct.

[0078] Subsequently, the complete structure of the object of the present invention underwent rigorous performance and usability tests, including a plurality of usage simulations with the participation of real users, so as to ensure the selection of the best solution to meet consumer expectations, guaranteeing precise application, facilitated lip contouring and a surprising and pleasant user experience.

[0079] To determine the ideal application thresholds for the new lipstick applicator, a systematic procedure involving human users was performed. This process of determining the thresholds was divided into two stages, the first aimed to define the safe pressure zone for the user.

[0080] In a first step, the mechanical applicator arm was removed from the case. The structure housing the components of the automatic cosmetic product application system, which is manually applied to the user's lips. This data was collected, statistically processed, and a safe pressure threshold for the user's lips was defined.

[0081] In a second stage, these thresholds were refined through testing with several users. Users applied the lipstick following a specific protocol under controlled conditions, and the quantitative data collected allowed the establishment of consistent application standards. These thresholds are essential to ensure that the new applicator provides adequate lipstick application, offering comfort, safety, and uniformity.

[0082] According to the present invention, for the automatic application system of cosmetic product to the user (1), a lipstick extruder component (18) was created and developed, as shown in the image in figure 11, with the initial idea being to use a peristaltic pump to pressurize the liquid lipstick in the ducts up to the applicator tip.

[0083] The peristaltic pump is an efficient device for this purpose, as it uses a system of rollers that compress the hose and propel the liquid precisely. However, during the development of this invention, it was discovered that this option takes up considerable space and makes it difficult to integrate the extruder into a compact format.

[0084] Thus, for the extrusion of liquid lipstick through the applicator tip (9), a custom pump was created and developed that pressurizes a syringe (19), for example of 10 ml, using a stepper motor (20), both positioned on a support structure (21) of said lipstick extruder component (18), which actuates the plunger of said syringe (19). The support structure (21) of said lipstick extruder (18), seen in the image of figure 11, for example, can be printed through 3D printing (FDM) with PLA material. This structural arrangement of the lipstick extruder component (18) converts the rotary movement of the stepper motor (20) into linear movement, pushing the plunger of the syringe (19) filled with liquid lipstick, and once the liquid lipstick leaves the syringe, it travels a hose (22), for example made of silicone, until it reaches the mechanical arm of the robotic application device (2).

[0085] The connection between the hose (22) and the mechanical arm of the robotic applicator device (2) is made using small plastic tubes with a thickness greater than the thickness of the hose (22), in order to ensure a good seal. Inside the mechanical arm of the robotic applicator device (2), the liquid lipstick travels through capillary action until it reaches its end at the applicator tip (9), to be applied to the user's lips.

[0086] To ensure safety and proper sealing, several tests were performed. These tests consisted of pressurizing the liquid lipstick at higher pressures than those typically used during normal application. The goal was to prevent any lipstick spillage within the structure of the automatic user-facing cosmetic product application system (1) of the present invention and to ensure a controlled and safe extrusion process.

[0087] Furthermore, experiments were conducted to evaluate the performance of the lipstick extruder component (18). In this phase, the amounts of lipstick released per unit time, the uniformity of the lipstick output and the consistency of the dosages during use were measured, such data being essential to optimize the operation of the automatic user cosmetic product application system (1) of the present invention, and to ensure a precise and homogeneous application of the lipstick on the lips.

[0088] Thus, the change to a custom pump using a stepper motor proved to be advantageous in terms of efficiency and size, making the lipstick extruder component (18) more practical and suitable for the purpose of the present invention.

[0089] Furthermore, with the aim of ensuring that all users could interact efficiently with the solution proposed by the automatic cosmetic product application system (1), regardless of their disabilities, multisensory feedback components were created and developed to integrate into the hardware system of said automatic cosmetic product application system. cosmetic product in user (1), comprising components involving sound aspects (such as speakers and / or microphones, information indicators and / or operating commands), visual aspects (such as LEDs indicating operating information) and tactile aspects (such as buttons or touch sensors and / or in high relief), aiming for the maximum possible inclusion with the user.

[0090] For example, as seen in figure 12, the system for automatically applying a cosmetic product to a user (1) is presented, highlighting the presence of a touch sensor component (23) located adjacent to the front surface (24) of the housing structure for the components of said system for automatically applying a cosmetic product to a user (1).

[0091] Although the present invention illustrates several multisensory feedback components, such as sound, visual and tactile, obviously, other types of deployment of the present solution can be implemented in the present system for automatic application of cosmetic product to the user (1), without straying from the scope claimed herein, for example: - it is possible that the solution must adapt to the possible heights of the user, so that he or she can move to any side and lose the idea of ​​a totem or rigid and non-adjustable structure; - it is important that the solution presents a way to ensure that the user can know perfectly where to position their head comfortably and safely; and - it is possible for the user to choose the lipstick colors, so that the chosen lipstick color option adapts to the lipstick extruder component.

[0092] Specifically, to define the trigger button for this system for automatically dispensing cosmetic products to users, commercially available options were identified that meet the needs of the target audience, such as people with physical disabilities and those with visual impairments. Based on this analysis, the options were evaluated based on their compliance with the prototype's hardware and software requirements, the required development effort, acquisition cost, and alignment with the user profile.

[0093] Considering that the system proposed by the present invention aims to provide a product that promotes the inclusion of all people, without discriminating against any disability, favoring usability and non-adaptability of the product. It is important that for people with the target audience profile, solutions that require physical activation must have a size, color, and texture that aid in the identification and location of the button, and must provide sensory feedback to the user. Furthermore, considering people with physical disabilities, this button must be easy to activate, such as a joystick or a tactile surface, which does not require gripping movements or specific gestures. Thus, it was concluded that the options for component forms for activation by button (push button), joystick, or touch sensor are capable of being implemented in execution modes of the present invention, and can satisfactorily serve the user.

[0094] In order to provide the various functionalities achieved by the present invention, the system for automatically applying cosmetic products to the user also comprises control and user interface circuits interconnected to the sound, visual and tactile components; and a feedback sensor circuit.

[0095] Regarding the user interface circuit, for audio feedback, for example, a speaker is used, and for visual feedback, for example, an LED strip is used. In other possible embodiments of the present invention, an RGB LED strip can be implemented, without departing from the scope claimed herein.

[0096] Regarding the control circuit, the LED strip, speaker, pressure sensor reading, and especially computer vision model processing are all performed by a single multiplatform board, such as a Raspberry Pi. The control circuit may also comprise, for example, an Arduino Mega, to control the motors present in the automatic user cosmetic product application system according to the present invention.

[0097] Finally, referring to the pressure sensor circuit, this includes a circuit, for example, for the analog FSR sensor. As described previously, two sensors are considered in the tests of the present invention. In one embodiment of the present invention, the pressure sensor circuit may also comprise a circuit for an I2C digital sensor, if its use is necessary.

Claims

CLAIMS 1.) “AUTOMATIC COSMETIC PRODUCT APPLICATION SYSTEM TO USER”, characterized by said automatic cosmetic product application system to user (1), comprising: a robotic application device (2) acting on a Z axis by a CNC system; computer vision camera (3) for capturing images and corresponding depth maps aimed at obtaining 3D mapping of the lip and converting it to the 3D perspective of said robotic application device (2); and multisensory devices for interacting with the user for controlling and activating said robotic application device (2); in which the multisensory devices for interacting with the user of said automatic cosmetic product application system to user (1), comprise components involving sound aspects, such as speakers and / or microphones, indicating information and / or operating commands; components involving visual aspects, such as LEDs indicating operating information;and components involving tactile aspects, such as buttons or touch and / or embossed sensors; in which said robotic application device (2) comprises an application tip (9) being a resilient component with a cushion flocked tip; a resistive FSR sensor (10) for measuring the pressure exerted by the application tip (9) during application of lipstick to the lip; a flexible elastic component (12), such as a spring, for resilient movement of said application tip (9), positioned adjacent to the application device (2) with said application tip (9); a control module of the automatic application system of cosmetic product to a user (1), for controlling and acting on the variation in application pressure occurring when the application tip (9) touches a user's lip; a lipstick dispensing device (17) connected to said application tip (9) for automatically and continuously moistening lipstick on said lipstick application tip (9); a lipstick extruder component (18) acting as a peristaltic pump for extruding liquid lipstick through the applicator tip (9), comprising a syringe (19) with a plunger and a stepper motor (20), both positioned on a support structure (21) of said lipstick extruder component (18); and a hose (22) and a mechanical arm of the robotic applicator device (2) connected by a tube for conducting the liquid lipstick by capillarity to one end of the applicator tip (9). 2.) “METHOD FOR AUTOMATIC APPLICATION OF COSMETIC PRODUCT TO USER”, characterized by: the computer vision camera (3) capturing a color image and capturing a depth map of the lip aligned with this image; combining the 2D segmentation of the lip, detecting the color image with a depth map; applying spatial and temporal filters in order to smooth and stabilize the depth map; obtaining the 3D mapping of the lip from the combination of the 2D segmentation with the depth map; in which initially, the X and Y coordinates in pixels of the lip are obtained from this color image; then, for each of these pixels, the corresponding Z coordinate in millimeters is extracted from the depth map; converting the X and Y coordinates in pixels to millimeters using the depth parameters of the computer vision camera;obtain the X, Y and Z coordinates in millimeters, referring to the coordinate system of the depth camera; and once the lip position has been obtained, perform a conversion to the 3D perspective of the robotic application device (2) acting on a Z axis by means of a CNC system; a resistive FSR sensor (10), acting within an operating range, measure and provide control over the degree of pressure exerted by the application tip (9) during the application of lipstick to the lip, performed by the robotic application device (2), which is responsive to touch on the lip and signals the control module of the automatic cosmetic product application system to the user (1), a; variation in application pressure occurring when the applicator tip (9) touches a user's lip, and whether or not this pressure application value needs to be corrected; a lipstick dispensing device (17) connected to said applicator tip (9), automatically and continuously wetting said lipstick applicator tip (9); a lipstick extruder component (18), acting as a peristaltic pump, pressurizing a syringe (19), using a stepper motor (20), both positioned on a support structure (21) of said lipstick extruder component (18); converting the rotary movement of the stepper motor (20) into linear movement, pushing the plunger of the syringe (19) filled with liquid lipstick, and once the liquid lipstick leaves the syringe (19), it travels through a hose (22) until it reaches the mechanical arm of the robotic applicator device (2); and within the mechanical arm of the robotic applicator device (2), the liquid lipstick travels the path by capillarity until it reaches its end in the applicator tip (9), to be applied to the user's lips;and multisensory devices comprising components involving sound aspects, components involving visual aspects, and components involving tactile aspects, interact with the user through control and user interface circuits and feedback sensor circuitry.; 3.) “METHOD FOR AUTOMATIC APPLICATION OF COSMETIC PRODUCT TO USER”, according to claim 2, and characterized by the FSR sensor (10) resistive pressure measurement device exerted by the applicator tip (9) during the application of lipstick to the lip, operating under a minimum measured pressure of 1.33kPa and a maximum measured pressure of 6.65kPa.

Images