Cosmetic applicator configured for users with limited mobility

Abstract

A motion stabilization device is provided for stabilization of a cosmetic applicator. The motion stabilization device includes a motion stabilizer handle configured to receive an adapter that holds a cosmetic applicator for a cosmetic application; and processing circuitry configured to detect a transition from a set up phase to an application phase of the cosmetic application process, and hold the cosmetic application in a set position when the transition is detected.

Description

COSMETIC APPLICATOR CONFIGURED FOR USERS WITH LIMITED MOBILITYCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application is related to U.S. Non-Provisional Application No. 18 / 228,336 July 31, 2023, U.S. Non-Provisional Application No. 18 / 228,347 filed July 31, 2023, FR Application No. 2311354 filed October 20, 2023, and FR Application No. 231 1367 filed October 20, 2023, the disclosures of which are incorporated herein by reference in their entirety.TECHNICAL FIELD

[0002] The present disclosure describes a system and features related to a device for modifying, mitigating, altering, reducing, compensating for or the like, the movement of a cosmetic applicator caused by unintentional movements, tremors, limited mobility, or the like of a user.BACKGROUND

[0003] Unintentional movements of the human body, or human tremors, can occur in individuals suffering from motion disorders or even healthy individuals. Due to these unintentional movements, a person may have difficulty in performing a task that requires care and precision, such as applying a cosmetic composition to a part of the body, such as the face, hands, or feet.

[0004] Therefore, there is a need for a solution that allows application of a cosmetic composition that is compatible with the diverse and disposable nature of cosmetic applicators.SUMMARY

[0005] In an embodiment, an accessibility diagnosis system is provided for a motion stabilization device for stabilization of a cosmetic applicator, comprising: a mobile user device that includes a touch sensitive display and a motion sensor, the mobile user device including processing circuitry configured to: display a request to a user for an input of a specific drawingImotion or a specific movement of the mobile user device, and collect information indicating a movement pattern of the user's hands based on the results of the user's input in response to the request; and a server device that is connected to the mobile user device via a network, the server device including processing circuitry configured to receive the collected information from the mobile user device, determine an amount of motion stabilization required to be set for the motion stabilization device based on the collected information, and output the determined amount of motion stabilization.

[0006] In an embodiment, the input of a specific drawing motion includes at least one of drawing a straight line and drawing a circular pattern.

[0007] In an embodiment, the input of a specific movement of the mobile user device includes at least one of moving the mobile user device close to the user's face and rotating the mobile user device in a predetermined direction.

[0008] In an embodiment, the mobile user device is configured to provide a questionnaire to the user and the amount of motion stabilization required to be set for the motion stabilization device is further based on the user's responses to the questionnaire.

[0009] In an embodiment, the processing circuitry of the server device is configured to combine the collected information from the mobile user device and determine if motion stabilization is active or static based on comparing a value associated with the collected information to a threshold.

[0010] In an embodiment, the processing circuitry of the server device is configured to determine if motion stabilization is active or static for at least one particular cosmetic application based on comparing a value associated with the collected information to a threshold.

[0011] In an embodiment, the processing circuitry server device determines the amount of motion stabilization by inputting the collected information into a neural network that is trained to output a motion stabilization profile based on pre-existing combinations collected information the motion stabilization profiles.

[0012] In another embodiment, a motion stabilization device is provided for stabilization of a cosmetic applicator, comprising: a motion stabilizer handle configured to receive an adapter that holds a cosmetic applicator for a cosmetic application; and processing circuitry configured to detect a transition from a set up phase to an application phase of the cosmeticapplication process, and hold the cosmetic application in a set position when the transition is detected.

[0013] In an embodiment, the processing circuitry is configured to detect the transition based on a predetermined change in movement patterns of the cosmetic applicator.

[0014] In an embodiment, the motion stabilization device further includes a force sensor, wherein the processing circuitry is configured to detect the transition based on a detected threshold amount of force placed upon the cosmetic applicator.

[0015] In an embodiment, the motion stabilization device further includes a torque sensor, wherein the processing circuitry is configured to detect the transition based on a detected threshold amount of torque placed upon the cosmetic applicator.

[0016] In an embodiment, the adapter further includes a hall sensor, and the processing circuitry is configured to detect the transition based on the hall sensor detecting that a cosmetic applicator has been inserted into the adapter.

[0017] In an embodiment, the motion stabilization device further includes an embedded camera and / or proximity sensor, and the processing circuitry is configured to detect the transition based on the camera and / or proximity sensor detecting that the cosmetic applicator has moved towards a target area of the cosmetic application.

[0018] In an embodiment, the motion stabilization device further includes a microphone, and the processing circuitry is configured to detect the transition based on an audible input from the user indicating that the application phase has started.

[0019] In an embodiment, the motion stabilization device further includes a switch, and the processing circuitry is configured to detect the transition based on an activation of the switch by the user indicating that the application phase has started.

[0020] In an embodiment, the processing circuitry is configured to detect the transition based on detecting that the motion stabilization device is held in a singular position for a threshold amount of time.

[0021] In an embodiment, the processing circuitry is configured to provide a learning mode to learn the predetermined change in movement patterns specific to the user, wherein in the learning mode the processing circuitry is configured to: (i) record movement patterns before the application phase has started (ii) receive an input from the user directly indicating when the application phase has started and record movement patterns during the application.5phase; (iii) receive an input from the user directly indicating that the application phase has ended and record movement patterns for after the application phase has endedBRIEF DESCRIPTION OF THE DRAWINGS

[0022] A more complete appreciation of the embodiments and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:

[0023] Fig. 1 shows a motion stabilizing device.

[0024] Fig. 2 shows how the motion stabilizing device couples with an adaptor and a make-up applicator.

[0025] Fig. 3A shows a diagram of the internal components of motion stabilizing device.

[0026] Fig. 3B shows a diagram of an alternative embodiment of the motion stabilizing device in which the receiver portion includes an electromagnetic positioner instead of the motive elements shown in Fig. 3A.

[0027] Fig. 4 shows an overview of a universal adapter handle connection system.

[0028] Fig. 5A depicts a situation in which the motion stabilizer is in a setup phase of using the motion stabilizer device.

[0029] Fig. 5B depicts a situation in the motion stabilizer is an application phase

[0030] Fig. 5C shows different phases of the process which are applicable to the abovedescribed device.

[0031] Fig. 6 shows a flowchart of a process in which the motion stabilizer detects when the user has transitioned from the setup phase to the application phase.

[0032] Fig. 7A shows one option in which an actual movement pattern of handling the motion stabilizer is analyzed and when the user's movements match expected movements for the application phase.

[0033] Fig. 7B shows a scenario in which at a time region indicates a particular change in the measurements of the X, Y, and Z sensors in comparison to the application phase.

[0034] Fig. 8A depicts a situation in which force is applied by the user to the flexible portion of the motion stabilizer handle to cause the cosmetic applicator to orient in the direction desired for cosmetic application.

[0035] Fig. 8B shows a situation when the cosmetic adapter depicted in Fig. 4 is being used.

[0036] Fig. 9A shows different components of the motion stabilizer and adapter.

[0037] Fig. 9B shows components included within the receiver portion of the motion stabilizer.

[0038] Fig. 9C-9D show how the proximity sensor is used to determine a phase.

[0039] Fig. 10 shows a flowchart according to a method of learning user's movement patterns.

[0040] Fig. 11 shows a system according to an embodiment that includes a mobile user device, a motion stabilizer, and a server device.

[0041] Figs. 12A and 12B show two challenges involving the user attempting to draw objects on the smartphone itself.

[0042] Figs. 12C and 12D show examples of how results are detected based on the challenges presented in Fig . 12A and 12B.

[0043] Figs. 13A-13D show four challenges related to moving the smartphone itself in space.

[0044] Figs. 13E and 13F show how the results are received for any of the challenges shown on Figs. 13A-13D.

[0045] Fig. 14 shows an additional information of collecting information about the user with a questionnaire

[0046] Fig. 15 shows that at the server device the results of the various challenges are input to a diagnostic engine.

[0047] Fig. 16 illustrates that one method of applying data collection to the motion stabilizer is to make binary decision on whether to use the motion stabilizer or not during certain types of cosmetic applications.

[0048] Fig. 17 shows a process that is specifically based on the results of the challenge shown in Fig. 12A.

[0049] Fig. 18 shows a process that is specifically based on the results of the challenge shown in Fig. 12B, where a user is asked to draw a circle.

[0050] Fig. 19 shows an example of a display to a user based on the results of the diagnosis in the form of a "fit graph.”

[0051] Fig. 20A is a diagram shown how machine learning can help determine movement patterns which indicate a particular phase of use.

[0052] Fig. 20B is a diagram shown how machine learning can help detelmine motion stabilization settings.

[0053] Fig. 21A and Fig. 21B show the usage of the deep learning model after training has reached an adequate level.

[0054] Fig. 22 is a more detailed block diagram illustrating a mobile user device according to certain embodiments of the present disclosure.

[0055] Fig. 23 shows a hardware description of the server device.DETAILED DESCRIPTION

[0056] The present disclosure describes a cosmetic applicator system that minimizes modifies, mitigates, alters, reduces, compensates for, or the like unintentional movements by stabilizing, orienting, operating, controlling, etc. an applicator for a user and is also designed to be flexible to accommodate different types of commercially available cosmetic applications. The present disclosure further describes a system and features to enhance the functionality of such a cosmetic applicator system.

[0057] The basic features and operation of a motion stabilizing device for a cosmetic applicator is described in U.S. Patent No. 11,458,062, which is incorporated herein by reference.

[0058] Fig. 1 shows a conventional motion stabilizing device 1100, which serves as a base unit for receiving a cosmetic applicator according to an embodiment. The device 1100 includes a handle portion 1101, a receiver portion 1102 and a strap 1103. The receiver portion 1102 includes an interface 1104, shown as a male connector that couples with a cosmetic applicator, which will be discussed in detail below. The receiver portion could be utilized for communication between the base unit and the applicator. The connection to an adaptor and / or an applicator could be accomplished with a mechanical coupling, such as screw-in or snap-fit, or it could be accomplished with magnets.

[0059] Fig. 2 shows how the device 1100 couples with an adaptor 1105 and a make-up applicator 1106. It can be seen that the adaptor fits over the exposed end of the receiver portion 1102. The adaptor includes electrical mating connectors (a female connector- not shown) in a recessed portion to make contact with the electric interface of the receiver portion 1101.

[0060] As shown in Fig. 2, the receiver portion 1102 is configured to contort, articulate, reposition, etc., between an upright posture (as shown in Fig. 1) and an angled posture (as shown in Fig. 2). This is accomplished with a hinge mechanism contained inside the receiver portion 1102. Fig. 2 shows that the hinge mechanism is a self-leveling / motion stabilizing hinge.

[0061] Fig. 3A shows a diagram of the internal components of device 1100 according to one embodiment. In the handle portion, the device includes a power source 1301, which may be a battery or the like. The device includes a printed circuit board assembly (PCBA) 1302, which may include positional sensor circuitry 1307, reader circuitry 1308, control circuitry 1309, and communication interface 1310, as understood in the art.

[0062] For instance, as the sensor circuitry 1307, the PCBA may include at least one inertial sensor and at least one distributed motion sensor to detect unintentional muscle movements and measure signals related to these unintentional muscle movements that are created when a user adversely affects motion of the applicator. These sensors also detect the motion of the stabilized output relative to device. The control circuitry sends voltage commands in response to the signals to the motion generating elements (described below) to cancel the user's tremors or unintentional muscle movements. This cancellation maintains and stabilizes a position of the applicator, keeping it stable.

[0063] One of ordinary skill in the art readily recognizes that a system and method in accordance with the present invention may utilize various implementations of the control circuitry and the sensor circuitry and that would be within the spirit and scope of the present invention. In one embodiment, the control circuitry 1309 comprises an electrical system capable of producing an electrical response from sensor inputs such as a programmable microcontroller or a field programmable gate array (FPGA). In one embodiment, the control circuitry comprises an 8-bit ATMEGA8A programmable microcontroller manufactured by Atmel due to its overall low-cost, low-power consumption and ability to be utilized in high- volume applications.

[0064] In one embodiment, the at least one inertial sensor in the sensor circuitry is a sensor including but not limited to an accelerometer, gyroscope, or combination of the two. In one embodiment, the at least one distributed motion sensor in the sensor circuitry is a contactless position sensor including but not limited to a hall-effect magnetic sensor.

[0065] The system created by the combination of the sensor circuitry, the control circuitry, and the motion generating elements may be a closed-loop control system that senses motion and acceleration at various points in the system and feeds detailed information into a control algorithm that moves the motion-generating elements appropriately to cancel the net effect of a user's unintentional muscle movements and thus stabilize the position of the applicator. The operation and details of the elements of the control system and control algorithm are understood in the art, as described in U.S. PG Publication 2014 / 0052275A1, incorporated herein by reference.

[0066] The communication interface 1310 may include a network controller such as BCM43342 Wi-i, Frequency Modulation, and Bluetooth combo chip from Broadcom, for interfacing with a network.

[0067] In the receiver portion of the device, there may be two motive elements to allow 3-dimensional movement of the receiver as anti-shaking movement. The two motive elements include a y-axis motive element 1303 and an x-axis motive element 1304, each being connected to and controlled by the PCBA 1302. Each of the motive elements may be servo motors as understood in the art. The device further includes end effector coupling 1305, which is configured to couple with the adaptor 1105. The end effector coupling 1305 may include a radiofrequency identification (RFID) reader 1306, configured to read an RFID tag, which may be included with the applicator, as will be discussed below.

[0068] Fig. 3B shows a diagram of an alternative embodiment of the device 1100 in which the receiver portion includes an electromagnetic positioner 1311 instead of the motive elements shown in Fig. 3A. The electromagnetic positioner 1311 may include U-shaped magnetic cores 1312 arrayed around a non-magnetic tube 1313, which is filled with a magnetic fluid 1314. Each of the magnetic cores bas arm portions that are surrounded by windings 1315. The magnetic cores may be controlled by the control circuitry in the PCBA 1302 to act as a controllable active magnetic field-generating structure which is used to generate a variable magnetic field that acts upon the magnetic fluid, causing it to be displaced, thereby enabling the armature to be moved to a desired coordinate position and / or orientation. The details of implementing the electromagnetic positioner 1311 may be found in U.S. Patent No. 6,553,161, which is incorporated herein by reference.

[0069] In the above-described conventional motion stabilizing device, there is a problem thatthe interface 1104 that receives the adaptor 1105 requires a specific point of attachment to align properly with the interface.

[0070] Therefore, the below embodiments provide a universal adapter connection between the handle of the motion stabilizing device in order to improve user experience and reduce the struggle and time taken to set up the system for use.

[0071] In one embodiment, the present disclosure is directed towards a cosmetic applicator. The cosmetic applicator can be used for a variety of cosmetics and cosmetic applications, including, but not limited to, mascara, eyeliner, eyebrow products, lip products (lipstick, lip gloss, lip liner, etc.), skin products, and / or hair products. In one embodiment, the cosmetic applicator can include an adapter, wherein the adapter can connect the cosmetic applicator to a motion stabilizer. The motion stabilizer can be, for example, a handle that can counteract unintentional motions such as tremors or spasms. These motions can interfere with the application of cosmetics and can also make it difficult to generally interact with cosmetic applicators or tools. For example, the many cosmetic products require a twisting motion or force to be applied to open or extrude the product. It can be difficult for users to achieve the range of motion or the precision necessary to apply these forces to the cosmetic. In one embodiment, the cosmetic applicator can hold a cosmetic and can enable the proper force to be applied to the cosmetic to open, close, mix, stir, blend, extrude, or achieve other similar functions necessary for application.

[0072] FIG. 4 shows an overview of a universal adapter handle connection system 400. The system includes a motion stabilizer device 150 that includes a handle portion 151 and a hinge portion 152 (receiver portion) that is functionally similar to the device 1100 shown in Fig. 1. It further includes a universal adapter 100 that attaches to the device 150 and also holds different types of cosmetic applicators. The basic features and operation of universal adapter handle connection system 400 are described in co-pending U.S. Applications 18 / 091,882; 18 / 091,920; 18 / 091,843; 18 / 091,925; 18 / 148,957; 18 / 148,880; and 18 / 148,930, which are incorporated herein by reference.

[0073] Currently, the above-described device does not know when the user is applying makeup or setting up positioning. This can cause difficulty of controlling the device position once they start applying makeup since the device will sometimes continue to move when they don't intend for it to move (i.e. want it to stay still in space once they start applying).The device should understand when the user is transitioning from setup of device orientation to actually applying the makeup.

[0074] This below features provide a solution to detect (either automated or by user input) when the user is transitioning from setup to application. When application is detected, the device will pause / hold its orientation in space to allow for accurate and controlled application by the user.

[0075] This doesn't mean that the device will be held static in a fixed state, but rather that the position of the applicator will be attempting to hold still in space while the motion stabilizer makes constant adjustments to account for the unintentional movements of the user's hands.

[0076] Fig. 5A depicts a situation in which the motion stabilizer is in a setup phase of using the motion stabilizer device. In this phase, the user is possibly attempting to attach the cosmetic applicator to the motion stabilizer and does not yet want the motion stabilizer to be oriented and held in the position that is required for application of the cosmetic product.

[0077] Fig. 5B depicts a situation in the motion stabilizer is an application phase and the user is ready for the motion stabilizer to be oriented and held in the position required for application of the cosmetic product.

[0078] Fig. 5C shows different phases of the process which are applicable to the abovedescribed device. Phase 501 involves the insertion of a cosmetic applicator (such as lipstick) into an adapter on the device. Phase 502 involves the user gripping the device to start to use the device. Phase 503 involves bringing the device to the target area of application (such as the lips of a user). Phase 504 involves the actual application of the cosmetic to the target area of the application. Finally phase 505 involves the removal of the applicator from the target area and the completion of the process.

[0079] The below embodiments provide a solution to detect (either automated or by user input) when the user is transitioning from setup to application. When application is detected, the device will pause / hold its orientation to allow for accurate and controlled application by the user. The device should understand when the user is transitioning from setup of device orientation to actually applying the makeup. Once this is detected, the device will hold its current orientation until application is complete or directed otherwise from the user.

[0080] The features described below achieve: (i) stable device positioning once applicationis started; (ii) improved confidence from user that device will do what they intend it to do; and (iii) potentially less user interaction, which improves accessibility for users that have trouble with fine motor skills or pain.

[0081] Fig. 6 shows a flowchart of a process in which the motion stabilizer detects when the user has transitioned from the setup phase to the application phase. Upon such detection, the motion stabilizer will hold the orientation of the cosmetic applicator in the proper position for cosmetic application (such as the position shown on Fig. 5B). In step 601, the motion stabilizer is in the setup phase, which may include the user lifting the motion stabilizer from a base unit and / or inserting the cosmetic applicator into the motion stabilizer. In step 602 the motion stabilizer detects that there is a transition from the setup phase to the application phase. Upon such detection, in step 603, the motion stabilizer will hold he general orientation of the cosmetic applicator in the user's desired position for make-up application. Different options for performing the detection step at step 602 will be described below.

[0082] Fig. 7A shows one option in which an actual movement pattern of handling the motion stabilizer is analyzed and when the user's movements match expected movements for the application phase, then the transition to the application phase is detected. Fig. 7A shows example measurements taken by the accelerometer in the X, Y, and Z directions during movement of the motion stabilizer during a setup phase.

[0083] Fig. 7B shows a scenario in which at a time region indicated by 3401 indicates a particular change in the measurements of the X, Y, and Z sensors in comparison to the application phase. Based on type of change, the motion stabilizer detects that the application phase has started.

[0084] Fig. 8A depicts a situation in which force is applied by the user to the flexible portion of the motion stabilizer handle to cause the cosmetic applicator to orient in the direction desired for cosmetic application. A force or torque sensor may be included in the motion stabilizer which detects a threshold amount of force or torque which will indicate that the transition to the application phase is detected.

[0085] Fig. 8B shows a situation when the cosmetic adapter 100 depicted in Fig. 4 is being used. In this situation. There might not be any force which bends the flexible portion of the motion stabilizer a significant amount, but instead the applicator may be twisted to be in the desired position. In this situation, a torque sensor is used to determine if a threshold amountI lof torques has been applied, which will indicate that the transition to the application phase is detected.

[0086] Fig. 9A shows a situation in which the cosmetic adapter 100 includes a hall effect sensor 910 which can detect the presence of a cosmetic applicator that has been inserted into the adapter. With this type of sensor, the application phase may be detected based solely on insertion of the applicator if normally no other adjustments are made after insertion.

[0087] Fig. 9A further shows that the cosmetic adapter 100 includes an embedded camera and / or proximity sensor 920. With this type of sensor, detection of the application phase may be detected based on a combination of insertion of the applicator into the adapter and movement of the adapter towards the user.

[0088] Fig. 9A further shows that the motion stabilizer may include means to allow the user to directly input that the application phase has started. For instance, the motion stabilizer includes a microphone 930, that will allow the user to enter voice commands to “hold” the applicator at the current position for the application phase. Alternatively, the motion stabilizer includes a slider switch 940, that will allow the user to directly toggle the motion stabilizer to hold its position for the application phase.

[0089] Fig. 9A further shows that the motion stabilizer includes timer circuitry 950. The timer can be used to enter a “hold” mode if the device is held in a singular position for a threshold amount of time (i.e., 3 seconds).

[0090] Fig. 9B shows additional details regarding the receiver portion 152. The receiver portion 152 includes flex sensor 960 , as understood in the art, to detect forced flexing of the receiver portion when the user in the setup phase. The receiver portion further includes a torque / force sensor 970, as understood in the art, that can detect a force to twist or turn the adapter that is disposed on the receiver portion 152.

[0091] Figures 9C and 9D further illustrate how the embedded camera and / or proximity sensor 920 is used to detect the start of the application and the end of the application phase. Fig. 9C shows (for the lipstick example) that the sensor 920 detects when the device is brought within X inches of a user, which can be used to detect the start of the application phase. Fig. 9D shows that the sensor further detects (after the application phase started) that the device is moved outside of the range of X inches from the user, which can be used to detect the end of the application phase.1 '2

[0092] Additionally, it was noted above for Figs. 7 A and 7B, that a change in the movement patterns may be detected to determine when an application phase is started. Related to this feature, the device can learn from a diagnosis process to determine a user's profile (i.e. types of movements made for their disability) and automatically recognize by means of their movements that application is being attempted / started.

[0093] For instance, Fig. 10 shows a flowchart according to this method of learning. In Step 1001, the motion stabilizer may be set to "learning mode", in which the user utilizes voice commands or the slider switch 940 to indicate when the application process has started. During a predetermined number of uses, the motion stabilizer will record the movement patterns before the application phase has started in the setup phase (step 1002) and during the application phase after the user inputs the switch or voice command to hold the applicator in the proper configuration (step 1003). The user can also enter a command of “hold off (or the like) or disable the slider switch to indicate that the application has ended, and the device will record the movement patterns indicating the application phase has ended (step 1004).

[0094] While the above example describes a situation in which the user enters a learning phase to have their own movement patterns learned, the user may also avoid the learning phase and utilize a "crowd-sourcing" technique that leverages the movement patterns of other users that have a similar profile or characteristics.

[0095] For the device described herein, due to different limitations caused by different disabilities, it can be challenging to understand what level of assistance is needed to provide a helpful / accurate assisted makeup application experience (i.e. how many degrees of freedom & how much automation from device). The below embodiments provide a simple self-guided diagnostic tool for at home use. Users use an online or app-based tool to run through simple exercises that will help diagnose how much assistance they need from the device, and can make a recommendation on which options of device best suit their needs (if any).

[0096] Fig. 11 shows a system 500 according to an embodiment includes a mobile user deice 510 and the motion stabilizer 520 described above. The system further includes a cloud server device 530 that connects to both the smartphone and the motion stabilizer. The smartphone will provide a user with multiple challenges and possibly a questionnaire todiagnose how much assistance they will need in terms of motion stabilization when using the device.

[0097] The system measures: shaking, duration to accomplish challenge, pattern accuracy, detects phone fall, asks about pain / discomfort, and records sound. Specifically, the smartphone is utilized to ask users to perform motions such as: draw a line (tremor check), draw a circle (prediction check), bring smartphone to face (elbow mobility check), rotation from portrait to landscape (wrist mobility check), detect fall (finger grasping check). The smartphone records position in real time utilizing accelerometer / gyroscope / camera.

[0098] Figs. 12A and 12B show two challenges involving the user attempting to draw objects on the smartphone itself. Fig. 12A requests the user to draw straight lines, and Fig. 12B requests the user to draw circular patterns. A target shape may be displayed which the user will attempt to trace within a shaped region.

[0099] Figs. 12C and 12D show examples of how results are detected based on the challenges presented in Figs. 12A and 12B. As shown in Figs. 12A and 12B, the target shape has a certain thickness that the user is essentially drawing within. The centroid of the user's contact on the touchscreen is detected, as is understood in the art. Fig. 12C shows an example when the user drawings a fairly straight line that stays with the displayed target region. When the user keeps the line within the region, it may be considered to have 0% deviation. On the contrary, Fig. 12D shows an example when the user is unable to keep the line straight and within the displayed region. In this situation, half of the user's contact is outside the region and thus this represents 50% deviation. The same method of detecting deviation applies to the circle example shown in Fig. 12B.

[0100] While the above method describes one way to detect if a user has successfully drawn a line or a circle, other methods may be used as is understood in the art. For instance, a method of recognizing graphical objects, such as lines and circles, is described in U.S. Patent No. 10,372,321, which is incorporated herein by reference. Additionally, image recognition techniques may be used to determine if the user has successfully drawn a recognizable line or circle.

[0101] Figs. 13A-13D show four challenges related to moving the smartphone itself in space. Fig. 13 A shows a challenge in which the user is asked to bring the phone to close to a target area, such as the lips or eyes. This challenge aims to show a user's ability to control a devicewhile bringing it towards their face.

[0102] Fig. 13B shows a challenge where the user is asked to attempt to rotate the smartphone forward or backwards in landscape mode. Fig. 13C shows a challenge where the user is asked to rotate the smartphone clockwise or counter-clockwise. These challenges aims to show a user's ability to make wrist movements while operating the device during application of the make-up.

[0103] Fig. 13D shows a challenge where the user is asked to hold the smartphone still. This challenge aims to determine the level of unintentional movements of the user's hands when just holding the device in space.

[0104] Figs. 13E and 13F show how the results are received for any of the challenges shown on Figs. 13A-13D. Fig. 13E shows the amplitude of vibration for a user that does not show signs of unintentional movements of the hands when performing any of the challenges. However, Fig. 13F shows the amplitude of vibration for a user that does show signs of unintentional movements of the hands when performing any of the challenges. As can be seen, in it. 13F, the amplitude of vibration is above a threshold amount X.

[0105] Fig. 14 shows an additional information of collecting information about the user with a questionnaire. Such questions may request age, gender, and health conditions. Additionally, if will be asked if the user has a particular health condition that affects the movement of their hands.

[0106] The collection of data from the various tests and questionnaire may come in different forms. For instance, a percentage accuracy of staying on the lines when tracing lines or circles may be collected. Additionally, the nature of the tracing (such as a "shaky1tracing pattern) will be detected as well.

[0107] Similarly, the shakiness of the smartphone while the user is attempting to bring the smartphone to a target area on the face, or when rotational exercises are being performed, will be detected.

[0108] The data that is collected during the challenges is transmitted to the server device 530. The server device will analyze the data to determine an amount and type of support needed to be provided by the motion stabilizer device.

[0109] Fig. 15 shows that at the server device 530 the results of the various challenges are input to a diagnostic engine 901, which is implemented by processing circuitry at the serverdevice 530.

[0110] Fig. 16 illustrates that one method of applying data collection to the motion stabilizer is to make binary decision on whether to use the motion stabilizer or not during certain types of cosmetic applications. In Step 1005, the results that are input to the diagnostic engine are normalized and combined. For instance, all results can be transformed into an integer value.

[0111] When the user is attempting to trace the line or circle, each time the user deviated from the path will increment a counter to give an integer value. For the challenges when the smartphone is being moved closer to the face or rotated in landscape mode, then each time the user shakes the phone in the process will increment a counter. Also, certain questions to the questionnaire will cause a counter to be incremented. For example, if the user answers "yes" to having tremors, then that will increment a counter.

[0112] In Step 1006, the combined integer value from all of the counters is compared to a threshold number. This number can be predetermined based on combined data from multiple users. In Step 1007, if the number is greater than or equal to the threshold number, then it is determined that motion stabilization should be activated. In Step 1008, if the number is not greater than or equal to the threshold number, then motion stabilization should not be activated (i.e., the device should be static).

[0113] While the above process of Fig. 16 represents a simplified method of determining a binary setting for the motion stabilization function on the device, a customized approach may also be used. For instance, certain results are indicative that the user may need motion stabilization for one application but not another. Fig. 17 shows a process that is specifically based on the results of the challenge shown in Fig 12A. In step 1101, the results for the straight line challenge are nolmalized. In step 1102, the results are compared to a threshold value. Then in step 1103 or 1104, motion stabilization will be active or deactivated for lipstick, eyeshadow, or mascara applications based on the results.

[0114] Alternatively, Fig. 18 shows a process that is specifically based on the results of the challenge shown in Fig. 12B, where a user is asked to draw a circle. In step 1201, the results for the circle line challenge are normalized. In step 1202, the results are compared to a threshold value. Then in step 1203 or 1204, motion stabilization will be active or deactivated for skincare applications.

[0115] The above processes show considerations by the diagnostic engine in setting themotion stabilizer. However, if the results show severe shakiness from either of the challenges in Figs. 13A or 13B, then the diagnostic engine may override the results of the processes in Figs. 17 or 18, and still activate motion stabilization.

[0116] Alternatively, certain parameters within the motion stabilizer may be set based on the results obtained from the diagnostic engine. For instance, the results could be used to turn on / off different variables, or weight variables, related to rotation or flexion. Since a specific user profile may be complex to customize, machine learning may also be used to determine the optimum settings for a user.

[0117] Fig. 19 shows an example of a display to a user based on the results of the diagnosis in the form of a "fit graph.” As shown, different types of cosmetic applications may populate the points along a perimeter of the graph, and color coding can be used to indicate which of the cosmetic applications will have motion stabilization, and which will not.

[0118] The above-described embodiments allows for characterization of a user's level of mobility to check whether or not a motion stabilizer will be useful, which will allow the consumer to determine if the device will be helpful for them or not even prior to purchase.

[0119] Furthermore, while the above-described embodiments describe the functionality being performed by a server device 530, it will be appreciated that the functionality can be performed by the user device 510 or a personal computer (not shown) of the user.

[0120] Fig. 20A is a diagram shown how machine learning can help determine movement patterns which indicate a particular phase of use. During training, previous users data is collected that has sensed data as described above with a label corresponding to the phase of use. For instance, a sensed movement pattern representing the transition to the application phase may be inputted.

[0121] However, any of the data sensed from the above-noted sensing mechanisms may be used. These inputs are provided at stage 1107, with the label of the phase (application phase in this example). While the application phase is used in this example, data corresponding to any of the phases (such as setup or post-application) may be uses and labeled appropriately.

[0122] The inputs are provided to a deep learning algorithm in step 1 108. The deep learning algorithm used may be based on available software as known in the art, such as Tensorflow, Keras, Mxnet, Caffe, or Pytorch. The result of the labeled training will be a neural network at step 1109. The neural network created includes nodes of each layer are clustered, theclusters overlap, and each cluster feeds data to multiple nodes of the next layer.

[0123] FIG. 20B is a diagram shown how machine learning can help determine motion stabilization settings. During training, previous users data is collected that has a combination of results of the diagnostic challenges described above and the optimal motion stabilization settings profile that was used for a respective user. These inputs are provided at stage 1110, where normalized and combined results similar to those described above are input along with the motion stabilization profile that essentially acts as a label.

[0124] The inputs are provided to a deep learning algorithm in step 1111. The deep learning algorithm used may be based on available software as known in the art, such as Tensorflow, Keras, Mxnet, Caffe, or Pytorch. The result of the labeled training will be a neural network at step 1112. The neural network created includes nodes of each layer are clustered, the clusters overlap, and each cluster feeds data to multiple nodes of the next layer.

[0125] Fig. 21 A shows the usage of the deep learning model after training has reached an adequate level. This is referred to as "inference time" since recommendation will be inferred from input data without a label. It can be seen that the input stage does not include a label of phase itself. The inputs of data sensed from a user are fed to the trained neural network at step 1206, which will provide an output at step 1207 of the phase that best fits the inputted data.

[0126] Fig. 21B shows the usage of the deep learning model after training has reached an adequate level. This is referred to as "inference time" since recommendation will be inferred from input data without a label. It can be seen that the input stage does not include a label of a motion stabilization profile. The inputs are fed to the trained neural network at step 1209, which will provide an output at step 1210 of the motion stabilization profile that best fits the inputted normalized and combined results of the user.

[0127] With the neural network models described herein, complex diagnosis results can be utilized to determine a custom motion stabilization profile for a user without having to come with up predetermined manual settings.

[0128] Fig. 22 is a more detailed block diagram illustrating a mobile user device 510 according to certain embodiments of the present disclosure. In certain embodiments, user device 510 is a smartphone. However, the skilled artisan will appreciate that the features described herein may be adapted to be implemented on other devices (e.g., a laptop, a tablet,a server, an e-reader, a camera, a navigation device, etc.). The exemplary user device 510 includes a controller 3110 and a wireless communication processor 3102 connected to an antenna 3101. A speaker 3104 and a microphone 3105 are connected to a voice proces sor 3103.

[0129] The controller 3110 may include one or more Central Processing Units (CPUs), and may control each element in the user device 510 to perform functions related to communication control, audio signal processing, control for the audio signal processing, still and moving image processing and control, and other kinds of signal processing. The controller 3110 may perform these functions by executing instructions stored in a memory 3150. Alternatively or in addition to the local storage of the memory 150, the functions may be executed using instructions stored on an external device accessed on a network or on a non-transitory computer readable medium. As described above in relation to Fig. 16, the controller 3110 may execute instructions allowing the controller 3110 to function as the display control unit 3211, operation management unit 3212 and game management unit 3213 depicted in Fig. 22.

[0130] Next, a hardware description of the server device 530 according to exemplary embodiments is described with reference to Fig. 23. In Fig. 23, the server device 530 includes a CPU 17 which performs the processes described above / below. The process data and instructions may be stored in memory 1702. These processes and instructions may also be stored on a storage medium disk 1704 such as a hard drive (HDD) or portable storage medium or may be stored remotely. Further the claimed advancements are not limited by the form of the computer-readable media on which the instructions of the inventive process are stored. For example, the instructions may be stored on CDs, DVDs, in FLASH memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk or any other information processing device with which the device communicates, such as a server or computer.

[0131] Further, the claimed advancements may be provided as a utility application, background daemon, or component of an operating system, or combination thereof, executing in conjunction with CPU 1700 and an operating system such as Microsoft Windows 7, UNIX, Solaris, LINUX, Apple MAC-OS and other systems known to those skilled in the art.

[0132] The hardware elements in order to achieve the device may be realized by variouscircuitry elements, known to those skilled in the art. For example, CPU 17 may be a Xenon or Core processor from Intel of America or an Opteron processor from AMD of America, or may be other processor types that would be recognized by one of ordinary skill in the art.

[0133] Alternatively, the CPU 1700 may be implemented on an FPGA, ASIC, PLD or using discrete logic circuits, as one of ordinary skill in the art would recognize. Further, CPU 1700 may be implemented as multiple processors cooperatively working in parallel to perform the instructions of the inventive processes described above.

[0134] The device in Figure 23 also includes a network controller 1706, such as an Intel Ethernet PRO network interface card from Intel Corporation of America, for interfacing with network 1717. As can be appreciated, the network 1717 can be a public network, such as the Internet, or a private network such as an LAN or WAN network, or any combination thereof and can also include PSTN or ISDN sub-networks. The network 1717 can also be wired, such as an Ethernet network, or can be wireless such as a cellular network including EDGE, 3G and 4G wireless cellular systems. The wireless network can also be WiFi, Bluetooth, or any other wireless form of communication that is known.

[0135] The device further includes a display controller 1708, such as a NVIDIA GeForce GTX or Quadro graphics adaptor from NVIDIA Corporation of America for interfacing with display 1710, such as a Hewlett Packard HPL2445w LCD monitor. A general purpose 1 / 0 interface 1712 interfaces with a keyboard and / or mouse 1714 as well as a touch screen panel 171 6 on or separate from display 1710. General purpose 1 / 0 interface also connects to a variety of peripherals 1718 including printers and scanners, such as an OfficeJet or DeskJet from Hewlett Packard.

[0136] A sound controller 1720 is also provided in the device, such as Sound Blaster X-Fi Titanium from Creative, to interface with speakers / mi crophone 1722 thereby providing sounds and / or music.

[0137] The general purpose storage controller 1724 connects the storage medium disk 1704 with communication bus 1726, which may be an ISA, EISA, VESA, PCI, or similar, for interconnecting all of the components of the device. A description of the general features and functionality of the display 1710, keyboard and / or mouse 1714, as well as the display controller 1708, storage controller 1724, network controller 1706, sound controller 1720, and general purpose 1 / 0 interface 1712 is omitted herein for brevity as these features areknown.

[0138] Obviously, numerous modifications and variations of the present invention are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein.

[0139] In a preferred embodiment, an accessibility diagnosis system for a motion stabilization device for stabilization of a cosmetic applicator may include a mobile user device that includes a touch sensitive display and a motion sensor, the mobile user device including processing circuitry configured to: display a request to a user for an input of a specific drawing motion or a specific movement of the mobile user device and collect information indicating a movement pattern of the user's hands based on the results of the user's input in response to the request; and include a server device that is connected to the mobile user device via a network, the server device including processing circuitry configured to receive the collected information from the mobile user device, determine an amount of motion stabilization required to be set for the motion stabilization device based on the collected information and output the determined amount of motion stabilization.

[0140] In another embodiment, the input of a specific drawing motion may include at least one of drawing a straight line and drawing a circular pattern; the input of a specific movement of the mobile user device includes at least one of moving the mobile user device close to the user's face and rotating the mobile user device in a predetermined direction; the mobile user device may be configured to provide a questionnaire to the user and the amount of motion stabilization required to be set for the motion stabilization device may be further based on the user's responses to the questionnaire; the processing circuitry of the server device may be configured to combine the collected information from the mobile user device and determine if motion stabilization is active or static based on comparing a value associated with the collected information to a threshold; the processing circuitry of the server device may be configured to determine if motion stabilization is active or static for at least one particular cosmetic application based on comparing a value associated with the collected information to a threshold; and / or the processing circuitry server device may determines the amount of motion stabilization by inputting the collected information into a neural network that is trained to output a motion stabilization profile based on pre-existingcombinations collected information the motion stabilization profiles.

[0141] Another preferred embodiment includes a motion stabilization device for stabilization of a cosmetic applicator, the motion stabilization device including a motion stabilizer handle configured to receive an adapter that holds a cosmetic applicator for a cosmetic application; and processing circuitry configured to detect a transition from a set up phase to an application phase of the cosmetic application process, and hold the cosmetic application in a set position when the transition is detected.

[0142] In another embodiment, the processing circuitry may be configured to detect the transition based on a predetermined change in movement patterns of the cosmetic applicator; the motion stabilization device may include a force sensor, such that the processing circuitry is configured to detect the transition based on a detected threshold amount of force placed upon the cosmetic applicator; the motion stabilization device may include a torque sensor, such that the processing circuitry is configured to detect the transition based on a detected threshold amount of torque placed upon the cosmetic applicator; the adapter may further include one or more of a hall sensor, an embedded camera, or a proximity sensor, and the processing circuitry is configured to one or more of detect the transition based on the hall sensor detecting that a cosmetic applicator has been inserted into the adapter or detect the transition based on the camera and / or proximity sensor detecting that the cosmetic applicator has moved towards a target area of the cosmetic application; the motion stabilization device may include one or more of a microphone or switch, such that the processing circuitry is further configured to one or more of detect the transition based on an audible input from the user indicating that the application phase has started, detect the transition based on an activation of the switch by the user indicating that the application phase has started, or detect the transition based on detecting that the motion stabilization device is held in a singular position for a threshold amount of time; and / or the processing circuitry may be further configured to provide a learning mode to learn the predetermined change in movement patterns specific to the user, such that in the learning mode the processing circuitry is configured to record movement patterns before the application phase has started, receive an input from the user directly indicating when the application phase has started and record movement patterns during the application phase, receive an input from the user directly indicating that the application phase has ended andrecord movement patterns for after the application phase has ended.

[0143] Another preferred embodiment includes a method, implemented by processing circuitry of a motion stabilization device for stabilization of a cosmetic applicator, the motion stabilization device including a motion stabilizer handle configured to receive an adapter that holds a cosmetic applicator for a cosmetic application. The method may include detecting a transition from a set up phase to an application phase of the cosmetic application process, and holding the cosmetic application in a set position when the transition is detected.

Claims

CLAIMS1. An accessibility diagnosis system for a motion stabilization device for stabilization of a cosmetic applicator, comprising: a mobile user device that includes a touch sensitive display and a motion sensor, the mobile user device including processing circuitry configured to: display a request to a user for an input of a specific drawing motion or a specific movement of the mobile user device, and collect information indicating a movement pattern of the user's hands based on the results of the user's input in response to the request; and a server device that is connected to the mobile user device via a network, the server device including processing circuitry configured to receive the collected information from the mobile user device, determine an amount of motion stabilization required to be set for the motion stabilization device based on the collected information, and output the determined amount of motion stabilization.

2. The accessibility diagnosis system according to claim 1, wherein the input of a specific drawing motion includes at least one of drawing a straight line and drawing a circular pattern.

3. The accessibility diagnosis system according to any of claims 1-2,wherein the input of a specific movement of the mobile user device includes at least one of moving the mobile user device close to the user's face and rotating the mobile user device in a predetermined direction.

4. The accessibility diagnosis system according to any of claims 1-3, wherein the mobile user device is configured to provide a questionnaire to the user and the amount of motion stabilization required to be set for the motion stabilization device is further based on the user's responses to the questionnaire.

5. The accessibility diagnosis system according to any of claims 1-4, wherein the processing circuitry of the server device is configured to combine the collected information from the mobile user device and determine if motion stabilization is active or static based on comparing a value associated with the collected information to a threshold.

6. The accessibility diagnosis system according to any of claims 1-5, wherein the processing circuitry of the server device is configured to determine if motion stabilization is active or static for at least one particular cosmetic application based on comparing a value associated with the collected information to a threshold.

7. The accessibility diagnosis system according to any of claims 1-7 , wherein the processing circuitry server device determines the amount of motionstabilization by inputting the collected information into a neural network that is trained to output a motion stabilization profile based on pre-existing combinations collected information the motion stabilization profiles.

8. A motion stabilization device for stabilization of a cosmetic applicator, comprising: a motion stabilizer handle configured to receive an adapter that holds a cosmetic applicator for a cosmetic application; and processing circuitry configured to detect a transition from a set up phase to an application phase of the cosmetic application process, and hold the cosmetic application in a set position when the transition is detected.

9. The motion stabilization device according to claim 8, wherein the processing circuitry is configured to detect the transition based on a predetermined change in movement patterns of the cosmetic applicator.

10. The motion stabilization device according to any of claims 8-9, further comprising a force sensor, wherein the processing circuitry is configured to detect the transition based on a detected threshold amount of force placed upon the cosmetic applicator.11 . The motion stabilization device according to any of claims 8-10, further comprising a torque sensor, wherein the processing circuitry is configured to detect the transition based on a detected threshold amount of torque placed upon the cosmetic applicator.

12. The motion stabilization device according to any of claims 8-11, wherein the adapter further includes one or more of a hall sensor, an embedded camera, or a proximity sensor, and the processing circuitry is configured to one or more of detect the transition based on the hall sensor detecting that a cosmetic applicator has been inserted into the adapter; or detect the transition based on the camera and / or proximity sensor detecting that the cosmetic applicator has moved towards a target area of the cosmetic application.

13. The motion stabilization device according to any of claims 8-12, further comprising one or more of a microphone or switch, and the processing circuitry is further configured to one or more of: detect the transition based on an audible input from the user indicating that the application phase has started; detect the transition based on an activation of the switch by the user indicating that the application phase has started; or detect the transition based on detecting that the motion stabilizationdevice is held in a singular position for a threshold amount of time.

14. The motion stabilization device according to any of claims 8-13, wherein the processing circuitry is configured to provide a learning mode to learn the predetermined change in movement patterns specific to the user, wherein in the learning mode the processing circuitry is configured to:(i) record movement patterns before the application phase has started(ii)receive an input from the user directly indicating when the application phase has started and record movement patterns during the application phase;(iii)receive an input from the user directly indicating that the application phase has ended and record movement patterns for after the application phase has ended.

15. A method, implemented by processing circuitry of a motion stabilization device for stabilization of a cosmetic applicator, the motion stabilization device including a motion stabilizer handle configured to receive an adapter that holds a cosmetic applicator for a cosmetic application, the method comprising: detecting a transition from a set up phase to an application phase of the cosmetic application process, and holding the cosmetic application in a set position when the transition is detected.

Images