Digital contact lens insertion and removal aid

Abstract

Disclosed herein are methods, systems, and user interfaces for inserting and removing contact lenses. Exemplary methods may include receiving one or more images of at least a portion of a user. The one or more images may include a representation of the user's eye and a representation of a contact lens on the user's eye. Exemplary methods may include analyzing the placement of the contact lens on the user's eye based on the one or more images. Exemplary methods may include outputting an indication of proper or improper placement of the contact lens based on the analysis of the placement of the contact lens.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Patent Application No. 17 / 891,736, filed August 19, 2022, which is incorporated herein by reference in its entirety. [Background technology]

[0002] Patients new to soft contact lenses often have difficulty inserting and removing them. This frustration can lead to lens wear failure, discomfort, and / or discontinuation. Often, patients can be successful when instructed to insert and remove contacts in an eye-care professional's (ECP's) office, but are unable to replicate the process once the patient returns home. While in the office, patients have the benefit of receiving feedback from the office staff, patients do not have this advantage when attempting to insert or remove lenses at home.

[0003] Improvement is needed. Summary of the Invention [Means for solving the problem]

[0004] Disclosed herein are methods, systems, and user interfaces for contact lens insertion and removal that assist novice soft contact lens patients in inserting and removing their contact lenses and help alert them when the contact lens is placed correctly on the eye. Contact lens dislodgement may be reduced when patients have tools that provide them with feedback regarding lens insertion and removal outside of an eye care provider's office.

[0005] The present disclosure relates to a digital (e.g., app) solution that can be thought of as a virtual eye technician, assisting novice soft contact lens patients in inserting and removing contact lenses and helping alert patients when the contact lenses are placed correctly on the eye. Accurate control of the upper and lower eyelids is key to insertion and removal, and therefore the methods, systems, and user interfaces disclosed herein can utilize a camera in a device, such as a mobile phone or tablet, to project visual cues onto the upper and lower eyelids to assist patients in precisely controlling the eyelids with their fingers during insertion and removal. When the fingers are in the correct position, the visual cues may change to a confirming color, such as green. When the fingers are not in the correct position, the visual cues may change to, for example, red. The methods, systems, and user interfaces disclosed herein may also provide step-by-step guidance to patients to successfully insert and remove contact lenses via voice or audio control.

[0006] One aspect of successful insertion and removal can be knowing that the contact lens is in the proper position on the eye. Therefore, it can be useful for the methods, systems, and user interfaces disclosed herein to be able to detect when the contact lens is centered on the eye. To determine if the contact lens is inserted correctly, the methods, systems, and user interfaces disclosed herein may utilize edge detection techniques to identify correct placement. The methods, systems, and user interfaces disclosed herein may be used in a home environment to enhance in-office instruction of contact lens insertion and removal when office staff are not available. Eye care professionals and staff may also use the methods, systems, and user interfaces disclosed herein to assist in-office training.

[0007] One or more computer systems may be configured to perform particular operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that, during operation, causes the system to perform the actions. One or more computer programs may be configured to perform particular operations or actions by including instructions that, when executed by a data processing device, cause the device to perform the actions. One general aspect includes a method for contact lens assistance. The method also includes generating, via a user interface, an output of one or more lens type options, receiving, via the user interface, an indication of a lens type to select from the one or more lens type options, generating, via the user interface, an output of one or more lens action options, receiving, via the user interface, an indication of a lens action to select from the one or more lens action options, generating, via the user interface, an output of one or more ophthalmic options, receiving, via the user interface, an indication of an eye to select from the one or more ophthalmic options, and generating, via the user interface, and based on one or more of the selected lens type, the selected lens action, or the selected eye, an output of one or more user instructions for inserting or removing a contact lens, wherein the one or more user instructions incorporate user feedback based on images of the user captured in real time. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0008] One general aspect includes a method for contact lens assistance, the method also including receiving one or more first images of at least a portion of a user, where the one or more first images may include a representation of the user's eye, causing, via a user interface and based on the one or more first images, output of visual cues indicative of a user state of the user with respect to the user's eye, the visual cues having a first state indicative of a rejected user state and a second state indicative of an accepted user state, causing output of one or more user instructions for contact lens insertion in response to the accepted user state, receiving one or more second images of at least a portion of the user, where the one or more second images may include a representation of the user's eye and a representation of the contact lens on the user's eye, analyzing placement of the contact lens on the user's eye based on the one or more second images, and outputting an indication of proper or improper placement of the contact lens based on the analyzing of the contact lens placement. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0009] One general aspect includes a method for contact lens assistance. The method also includes receiving one or more images of at least a portion of a user, where the one or more images may include a representation of the user's eye and a representation of a contact lens on the user's eye, analyzing the placement of the contact lens on the user's eye based on the one or more images, and outputting instructions for proper or improper placement of the contact lens based on the analysis of the placement of the contact lens. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0010] One general aspect includes a method for contact lens assistance. The method also includes receiving real-time video of at least a portion of a user, where the video may include a representation of the user's eyes; analyzing a user state of the user based at least on the video to determine whether the user state is an accepted user state or a rejected user state; generating, via a user interface and based on the video, output of visual cues indicative of the user's user state related to the user's eyes, the visual cues having a first state indicative of a rejected user state and a second state indicative of an accepted user state; generating, via a user interface and based on the video and based on the user state, output of one or more real-time prompts to be output to the user; and repeating steps a-d until the user state is determined to be an accepted user state. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method. [Brief explanation of the drawings]

[0011] The following drawings illustrate generally, by way of example, but not by way of limitation, various examples contemplated in this disclosure. [Figure 1] 1 shows an exemplary flowchart for a method according to the present disclosure. [Figure 2] 1 illustrates an exemplary user interface screen according to the present disclosure. [Figure 3] 1 shows an exemplary flowchart for a method according to the present disclosure. [Figure 4] 1 shows an exemplary flowchart for a method according to the present disclosure. [Figure 5] 1 shows an exemplary flowchart for a method according to the present disclosure. [Figure 6]1 illustrates an exemplary image of a user's eye, according to the present disclosure. [Figure 7] The example image of FIG. 6 is shown with the calculated boundary zones overlaid. [Figure 8] 7 shows an exemplary masked image after performing an intelligent masking technique on the image of FIG. 6 to remove image data outside a defined image window, according to the present disclosure. [Figure 9] 9 illustrates the masked image of FIG. 8 showing possible boundary arcs according to the present disclosure. [Figure 10] 9 illustrates the masked image of FIG. 8 showing delineation circles for identifying good and bad delineation circles according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present disclosure relates to methods, systems, and user interfaces for inserting and removing contact lenses. A user may use a camera on a smart device to record video of the user attempting to insert and / or remove a contact lens. The smart device screen may provide the user with immediate feedback. The immediate feedback may include approval and / or disapproval of the user's finger placement, eyelid positioning, contact lens positioning, etc. The methods, systems, and user interfaces described herein may reduce the time, effort, and frustration associated with learning to insert and / or remove contact lenses.

[0013] FIG. 1 shows an exemplary flowchart for a method for contact lens assistance, such as inserting or removing a contact lens into or from a user's eye(s). One or more computer systems may be configured to perform specific operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform the action during operation. As an example, a smartphone may be equipped to capture one or more images (e.g., video, real-time video) of a user. As a further example, a user may perform the method on a device configured to display information to the user while capturing a "selfie" view image (e.g., video, real-time video) of the user.

[0014] In step 102, one or more lens type options may be output via a user interface. The user interface may be on any device or display. As an example, the user interface may be on a device configured to capture an image of a user's face while displaying the user interface. The lens type options may include one or more indications of reusable or daily disposable.

[0015] At step 104, an indication of a lens type to select from the one or more lens type options may be received via a user interface. The user interface may be on any input device or touch-enabled display. As an example, the user interface may be on a device configured to display options and receive an indication of selection of one of the displayed options. Receiving an indication of a lens type to select from the one or more lens type options may include receiving an indication of engagement of a clickable element, such as a button, associated with the selected lens type.

[0016] In step 106, one or more lens operation options may be output via a user interface. The user interface may be on any device or display. As an example, the user interface may be on a device configured to capture an image of the user's face while displaying the user interface. The one or more lens operation options may include one or more indications of inserting or removing contact lenses.

[0017] At step 108, an indication of a lens action to select from the one or more lens action options is received via a user interface. The user interface may be on any input device or touch-enabled display. As an example, the user interface may be on a device configured to display options and receive an indication of a selection of one of the displayed options. Receiving an indication of a lens action to select from the one or more lens action options may include receiving an indication of engagement of a clickable element, such as a button, associated with the selected lens action.

[0018] In step 110, one or more eye options may be output via a user interface. The user interface may be on any device or display. As an example, the user interface may be on a device configured to capture an image of the user's face while displaying the user interface. The one or more eye options may include one or more representations of a left eye or a right eye.

[0019] At step 112, an indication of an eye to select from the one or more eye options is received via a user interface. The user interface may be on any input device or touch-enabled display. As an example, the user interface may be on a device configured to display options and receive an indication of selection of one of the displayed options. Receiving an indication of an eye to select from the one or more eye options may include receiving an indication of engagement of a clickable element, such as a button, associated with the selected eye.

[0020] In step 114, one or more user instructions for inserting or removing a contact lens may be output via a user interface and may be based on one or more of a selected lens type, a selected lens action, or a selected eye. The one or more user instructions may incorporate user feedback based on images of the user captured in real time. The user interface may be on any device or display. As an example, the user interface may be on a device configured to capture images of the user's face while displaying the user interface. Causing the output of the one or more user instructions for inserting or removing a contact lens may be performed for the selected eye. One or more user instructions for inserting or removing a contact lens for an unselected eye of the one or more eye options may be automatically output via the user interface based on one or more of a selected lens type or a selected lens action. The one or more user instructions may include cleaning and drying instructions. The one or more user instructions may include contact lens handling instructions. The one or more user instructions may include contact lens position instructions. The one or more user instructions may include contact lens orientation instructions. The one or more user commands may include a finger position command, a camera configuration command, or a user feedback query.

[0021] FIG. 2 illustrates an exemplary method flow for contact lens assistance, such as inserting or removing a contact lens into or from a user's eye(s). One or more computer systems may be configured to perform specific operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform the action during operation. As an example, a smartphone may be equipped to capture one or more images (e.g., video, real-time video) of a user. As a further example, a user may perform the method on a device configured to display information to the user while capturing a "selfie" view image (e.g., video, real-time video) of the user.

[0022] An exemplary method flow includes an introductory (e.g., start, landing, etc.) page. The introductory page may include a button for launching one or more methods described herein. The introductory page may include a button for initiating a registration procedure. Activating a button for launching one or more methods described herein may cause a first screen with lens type options to be displayed. The lens type options may include a reusable option and a daily disposable option. Each lens type option may be associated with a button on the first screen. Activating a button on the first screen may cause the associated lens type option to be selected and cause a second screen with lens action options to be displayed. The lens action options may include an insert option and a remove option. Each lens action option may be associated with a button on the second screen. Activating a button on the second screen may cause the associated lens action option to be selected and cause a third screen with eye options to be displayed. The eye options may include a left option and a right option. Each eye option may be associated with a button on the third screen. Engaging a button on the third screen may cause an associated eye option to be selected and may launch a set of instruction screens based on the selected option. The set of instruction screens may include written instructions. The set of instruction screens may include a video of the user. The user video may include an overlay. The overlay may include instructions to approve or disapprove of the positioning of the finger, eyelid, contact lens, etc. The set of instruction screens may include start instructions. The set of instruction screens may include end instructions. Once the instructions for the selected eye are completed, the set of instructions may include instructions related to the non-selected eye.

[0023] 3 shows an exemplary flowchart for a method for contact lens assistance, such as inserting or removing a contact lens into or from a user's eye(s). One or more computer systems may be configured to perform specific operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform the action during operation. As an example, a smartphone may be equipped to capture one or more images (e.g., video, real-time video) of a user. As a further example, a user may perform the method on a device configured to display information to the user while capturing a "selfie" view image (e.g., video, real-time video) of the user.

[0024] In step 302, one or more first images of at least a portion of a user may be received. The one or more first images may include a representation of the user's eye. The one or more first images may include a representation of the iris of the user's eye. The one or more first images may be captured from video. The one or more first images may be captured from real-time video.

[0025] In step 304, visual cues indicating a user state of the user for the user's eyes may be output via a user interface based on the one or more first images. The visual cues may have a first state indicating a rejected user state and a second state indicating an accepted user state. The user state may include one or more of the user's eye openness, a characteristic of the user's other eye, and the user's finger placement relative to the eye. Output prompts may be provided to the user via the user interface and may be based on the one or more first images until an accepted user state is detected. The user interface may be on any device or display. As an example, the user interface may be on a device configured to capture images of the user's face while displaying the user interface.

[0026] One or more user instructions for contact lens insertion may be output in response to the allowed user state in step 306. The one or more user instructions for contact lens insertion may include audio instructions, visual instructions, or both.

[0027] In step 308, one or more second images of at least a portion of the user may be received. The one or more second images may include a representation of the user's eye and a representation of a contact lens on the user's eye. The one or more second images may be captured from video. The one or more second images may be captured from real-time video.

[0028] In step 310, the placement of the contact lens on the user's eye may be analyzed based on the one or more second images. Analyzing the placement of the contact lens on the user's eye based on the one or more second images may include detecting edges of the contact lens using an edge detection model. Analyzing the placement of the contact lens on the user's eye based on the one or more second images may include preprocessing the one or more second images using one or more of cropping, masking, removal, or a combination thereof. Analyzing the placement of the contact lens on the user's eye based on the one or more second images may include preprocessing the one or more second images by digitally removing artifacts from the one or more second images. The artifacts may include the user's eyelashes. Analyzing the placement of the contact lens on the user's eye based on the one or more second images may include detecting edges of the contact lens using a mask for edge detection. As a non-limiting example, analyzing the placement of the contact lens on the eye may include one or more processes described herein, for example, implementing one or more processes described with reference to FIGS. 6-10. Other processes may also be used.

[0029] In step 312, an indication of proper or improper placement of the contact lens may be output based on analyzing the placement of the contact lens.

[0030] 4 shows an exemplary flowchart for a method for contact lens assistance, such as inserting or removing a contact lens into or from a user's eye(s). One or more computer systems may be configured to perform specific operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform the action during operation. As an example, a smartphone may be equipped to capture one or more images (e.g., video, real-time video) of a user. As a further example, a user may perform the method on a device configured to display information to the user while capturing a "selfie" view image (e.g., video, real-time video) of the user.

[0031] In step 402, one or more images of at least a portion of a user may be received. The one or more images may include a representation of the user's eye and a representation of a contact lens on the user's eye. The one or more images may be captured from video. The one or more images may be captured from real-time video.

[0032] In step 404, the placement of the contact lens on the user's eye may be analyzed based on the one or more images. Analyzing the placement of the contact lens on the user's eye based on the one or more images may include detecting edges of the contact lens using an edge detection model. Analyzing the placement of the contact lens on the user's eye based on the one or more images may include preprocessing the one or more images using one or more of cropping, masking, removal, or a combination thereof. Analyzing the placement of the contact lens on the user's eye based on the one or more images may include preprocessing the one or more images by digitally removing artifacts from the one or more second images. The artifacts may include the user's eyelashes. Analyzing the placement of the contact lens on the user's eye based on the one or more images may include detecting edges of the contact lens using a mask for edge detection. As a non-limiting example, analyzing the placement of the contact lens on the eye may include one or more processes described herein, for example, implementing one or more processes described with reference to FIGS. 6-10. Other processes may also be used.

[0033] In step 406, an indication of proper or improper placement of the contact lens may be output based on analyzing the placement of the contact lens.

[0034] FIG. 5 shows an exemplary flowchart for a method for contact lens assistance, such as inserting or removing a contact lens into or from a user's eye(s). One or more computer systems may be configured to perform specific operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform the action during operation. As an example, a smartphone may be equipped to capture one or more images (e.g., video, real-time video) of a user. As a further example, a user may perform the method on a device configured to display information to the user while capturing a "selfie" view image (e.g., video, real-time video) of the user.

[0035] At step 502, real-time video of at least a portion of a user may be received. The video may include a view of the user's eyes.

[0036] In step 504, the user's user state may be analyzed based at least on the video to determine whether the user state is an accepted user state or a rejected user state. An accepted user state may include a properly inserted contact lens on the user's eye. An accepted user state may include a properly removed contact lens on the user's eye. The user state may include one or more of the following: the degree of openness of the user's eye, the placement of the contact lens on the eye, characteristics of the user's other eye, or the placement of the user's finger relative to the eye, the lens being too high, or the lens being too low. As a non-limiting example, the analysis of the placement of the contact lens on the eye may include one or more processes described herein, for example, implementing one or more processes described with reference to FIGS. 6-10. Other processes may also be used.

[0037] In step 506, visual cues indicating the user's user state for the user's eyes may be output via the user interface and based on the video. The visual cues may have a first state indicating a rejected user state and a second state indicating an accepted user state. The user interface may be on any device or display. As an example, the user interface may be on a device configured to capture an image of the user's face while displaying the user interface.

[0038] In step 508, one or more real-time prompts may be output to the user via a user interface and based on the video and user state. The user interface may be on any device or display. As an example, the user interface may be on a device configured to capture an image of the user's face while displaying the user interface. The one or more real-time prompts may include instructions for inserting or removing contact lenses.

[0039] In step 510, steps 502 to 508 may be repeated until the user state is determined to be an allowed user state.

[0040] As described herein, analyzing a user state may include analyzing one or more of the user's eye openness, the placement of a contact lens on the eye, characteristics of the user's other eye, or the placement of the user's finger relative to the eye. Such analysis may be based at least in part on an image (or video) capturing a portion of the user, such as an eye. Such images and their analysis may benefit from certain processing techniques, such as those described herein below. Analyzing the one or more images may include detecting the edge of the contact lens using an edge detection model. Analyzing the one or more images may include preprocessing the one or more images using one or more of cropping, masking, removal, or a combination thereof. Analyzing the one or more images may include preprocessing the one or more images by digitally removing artifacts from one or more second images. The artifacts may include the user's eyelashes. Analyzing the one or more images may include detecting the edge of the contact lens using a mask for edge detection.

[0041] As a non-limiting example, analysis of a user condition (e.g., contact lens placement on the eye) may include one or more preprocessing methods by which images or videos captured by the devices disclosed herein may be preprocessed. Image processing methods may function before, during, or after additional method flows disclosed herein (see FIGS. 1-5). Preprocessing methods include, but are not limited to, various techniques described herein.

[0042] By way of non-limiting example, image processing techniques may include tightly cropping a user's image or video, emphasizing portions of the user's ocular structures, adjusting modeling architecture, isolating the blue channel, simplifying analysis methods and models, and other such strategies. Such strategies may be used to achieve or progress toward one or more objectives. These objectives may be selected to improve the overall functionality of the method flows described herein. Objectives include, but are not limited to, removing obstacles to eye recognition, emphasizing the lens ridges of ocular structures, finding multiple circular bodies associated with ocular structures, and the like. Such objectives may aid in improved lens recognition. Improved lens recognition improves the functionality and output of additional method flows disclosed herein. Ocular structures may include structures associated with the eye, cornea, iris, sclera, lens, and other such visually relevant bodies. As a non-limiting example, the processing method may occur after one or more images or videos of at least a portion of a user are captured by a device (e.g., step 110, step 302, step 308, step 402, step 502, etc.), but before an output is provided to the user (e.g., step 114, step 312, step 416, step 508, etc.). As a further non-limiting example, the processing method may occur during the analysis steps (e.g., 310, 404, 504, etc.). As previously mentioned, the processing method improves the reliability and accuracy of the output of the method flows disclosed herein.

[0043] The processing may utilize various methods to improve the output to the user. The processing may improve lens recognition, detection, etc., within one or more images or videos captured by the device. In certain embodiments, the processing may be used to recognize or detect a lens using the topography of an ocular structure (e.g., the eye). In doing so, it may be said that the lens is likely to be found within the annular zone relative to the iris of the ocular structure. By identifying characteristic dimensions of the annular zone relative to the iris, one or more approximate characteristic dimensions of the lens may be discovered. These dimensions include, but are not limited to, the minimum and maximum radii of possible lenses (e.g., lens edges). These dimensions may enable the creation of a window of potential lens locations.

[0044] The window may be defined by two or more concentric circles, although other windows are possible. As a non-limiting example, the window may be defined by two or more ellipses or other round structures. As a non-limiting example, an image or video of an ocular structure may include an arc or a portion of a circle (e.g., an incomplete circle). An arc may represent two or more ocular structures. As a non-limiting example, an arc may represent a portion of an eyelid, a portion of a lens, etc. The processing methods disclosed herein may identify whether an arc is associated with a given ocular structure. A correct arc may be recognized as an arc associated with an ocular structure that is centered on the iris or within a tolerance of such a center. As a non-limiting example, the processing method may distinguish between an arc representing an eyelid and an arc representing a lens. In such an example, an arc representing an eyelid may not be centered on the iris, while an arc representing a lens is centered on the iris. Creating an image window using approximate characteristic dimensions improves the ability to accurately and reliably recognize one or more lenses in one or more images or videos captured by the device.

[0045] The present disclosure relates to methods for further improving the accuracy and reliability of processing methods. As a non-limiting example, improving the quality of an image or video of a user's ocular structure can improve the accuracy and reliability of the processing method. Improved quality includes, but is not limited to, more accurate image or video capture of the user's iris. This includes capturing an image or video that represents all existing boundaries of the iris. One such method is to increase the exposed area of ​​the user's eye. Increasing the exposed area allows the device to more accurately capture an image or video of the user's ocular structure. More accurate image capture allows the processing methods and other method flows disclosed herein to experience improved functionality.

[0046] As an illustrative example, Figure 6 shows an input image including at least a portion of a user's eye. This input image may be captured using a variety of cameras and across a variety of light spectrums. From the captured image, one or more boundary regions (e.g., iris boundary region, lens edge minimum boundary region, lens edge maximum boundary region, etc.) may be calculated (e.g., estimated). Various formulas and techniques may be used to calculate the one or more boundary regions. As an example, equation set 1 may be used, as shown below:

[0047]

number

[0048] Other formulas, estimates, and representative values ​​(e.g., 11.7 mm, 14 mm, 2 mm, etc.) may also be used. FIG. 7 shows an exemplary example of a calculated iris boundary 702 (and iris center 703), an estimated lens circumference minimum 704, and an estimated lens circumference maximum 706. These boundary 702, boundary 704, and boundary 706 may be used to calculate a window between the estimated lens circumference minimum 704 and the estimated lens circumference maximum 706. By removing (e.g., masking) portions of the image outside the calculated window, the image may be processed into a masked image 800, as shown in FIG. 8. This masked image represents a subset of the image information from the original image.

[0049] The masked image 800 may include one or more arc or boundary markers. The arcs may represent two or more ocular structures. By way of non-limiting example, the arcs may represent portions of an eyelid, portions of a lens, an edge, a boundary, and / or the like. Such arcs may be analyzed and classified as types, such as an eyelid arc 902 or a lens boundary arc 904, as shown in FIG. 9 . The type may be determined using a formula, such as Equation Set 1, or other method to identify the arc within a predetermined confidence level. For example, it may be determined that the located arc forms a portion of a representative circle. Once such a circle is estimated, the center of the circle can be determined and compared to the calculated center of the iris boundary. As an illustrative example, FIG. 10 shows the masked image 800 with an overlay of a representative circle 1002, 1002′, and a circle center 1004, 1004′ calculated based on the arc shape. A preset confidence boundary 1006, which may be derived from an equation such as equation set one, may be used to determine whether the representative circle center 1004, circle center 1004' is within or outside the confidence boundary 1006. As a further example, a circle center 1004 within the confidence boundary 1006 may be identified as a correct circle to be fitted around a lens. A circle center 1004' outside the confidence boundary 1006 may be identified as an incorrect circle to be fitted around something other than the target lens (e.g., the eyelid). The confidence boundary 1006 may be adjusted to control the identification of arcs and artifacts in the masked image 800, which may be used to indicate proper or improper lens placement or proper or improper lens removal.

[0050] Other lens detection algorithms may be used, such as the detection formula published by Kumar et al. in *Fully Automated Soft Contact Lens Detection from NIR Iris Images*, February 24, 2016, ICPRAM Publication, Computer Science. By way of further example, processing the image may include using techniques (e.g., preprocessing techniques) such as those described in *Using Iris and Sclera for Detection and Classification of Contact Lenses*, but Gragnaniello et al., *Pattern Recognition Letters*, Published by Elsevier Online ISSN: 0167-8655. Additionally and alternatively, the relevant facial structures discussed herein, including the iris, may be determined by application of a combination of one or more of deep learning, neural networks, and trained machine learning models, as would be understood by one of ordinary skill in the art. As one non-limiting example, a convolutional neural network such as that described in ContlensNet: Robust Iris Contact Lens Detection Using Deep Convolutional Neural Networks, presented at the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), may be used to identify the iris.

[0051] In some examples, lidar data, such as may be collected by certain iPhone® models, may be used in addition to or instead of video data to determine whether a user is performing an action that leads to a failure mode (e.g., improper or unsuccessful insertion or improper or unsuccessful removal) or success (e.g., proper insertion or proper removal). Either or both of these data may be collected in either a clinical or home environment and statistically aggregated over a sufficient number of users. Once collected, the data is preferably randomly ordered and tagged in a database with tags associated with known failure modes or successes. Failure modes may include, for example, i) insufficient opening of the user's eye, ii) insufficient opening of the user's other eye, iii) improper placement of the user's finger relative to the eye, and iv) a lens that is too high or too low. This data may then be used to train a machine learning model to recognize known failure modes or successes by users in real time. A foreground-specific training model may represent any suitable machine learning model that can be trained to detect and outline objects in images or lidar data, such as a collection of neural network layers. This model may then be deployed in software on a user device, for example, a cell phone equipped with either or both video and lidar sensors, to provide the user with auditory or visual feedback / prompts, including but not limited to visual thinking as described above, which are known to assist the user in successfully inserting a contact lens into the eye. [Example]

[0052] Example 1: A method for contact lens assistance, the method including: generating, via a user interface, an output of one or more lens type options; receiving, via the user interface, an indication of a lens type to select from the one or more lens type options; generating, via the user interface, an output of one or more lens action options; receiving, via the user interface, an indication of a lens action to select from the one or more lens action options; generating, via the user interface, an output of one or more eye options; receiving, via the user interface, an indication of an eye to select from the one or more eye options; and generating, via the user interface, an output of one or more user commands for inserting or removing a contact lens based on one or more of the selected lens type, the selected lens action, or the selected eye, wherein the one or more user commands incorporate user feedback based on an image of the user captured in real time.

[0053] Example 2: The method of example 1, wherein the lens type options include one or more indications of reusable or daily disposable.

[0054] Example 3: The method of any of Examples 1-2, wherein the one or more lens action options include one or more indications of inserting or removing a contact lens.

[0055] Example 4: The method of any of Examples 1-3, wherein the one or more eye options include one or more representations of the left eye or the right eye.

[0056] Example 5: A method as described in any of Examples 1 to 4, wherein causing the output of one or more user instructions for inserting or removing a contact lens is performed for the selected eye, and further comprising automatically causing the output of one or more user instructions for inserting or removing a contact lens for an unselected eye of the one or more eye options via a user interface and based on one or more of the selected lens type or the selected lens action.

[0057] Example 6: The method of any of Examples 1-5, wherein the one or more user instructions include a wash and dry instruction.

[0058] Example 7: The method of any of Examples 1-6, wherein the one or more user instructions include contact lens care instructions.

[0059] Example 8: The method of any of Examples 1-7, wherein the one or more user instructions include contact lens position instructions.

[0060] Example 9: The method of any of Examples 1-8, wherein the one or more user instructions include contact lens orientation instructions.

[0061] Example 10: The method of any of Examples 1-9, wherein the one or more user instructions include a finger position instruction.

[0062] Example 11: The method of any of Examples 1 to 10, wherein the one or more user instructions include a camera configuration instruction.

[0063] Example 12: The method of any of Examples 1 to 11, wherein the one or more user instructions include a user feedback query.

[0064] Example 13: A method for contact lens assistance, the method including: receiving one or more first images of at least a portion of a user, the one or more first images including a representation of the user's eye; causing, via a user interface and based on the one or more first images, output of visual cues indicative of a user state of the user with respect to the user's eye, the visual cues having a first state indicative of a rejected user state and a second state indicative of an accepted user state; causing, in response to the accepted user state, output of one or more user instructions for contact lens insertion; receiving one or more second images of at least a portion of the user, the one or more second images including a representation of the user's eye and a representation of the contact lens on the user's eye; analyzing placement of the contact lens on the user's eye based on the one or more second images; and outputting instructions of proper or improper placement of the contact lens based on the analyzing of the contact lens placement.

[0065] Example 14: The method of example 13, wherein the one or more first images include a representation of the iris of the user's eye.

[0066] Example 15: The method of any of Examples 13-14, wherein the user state includes one or more of the degree of opening of the user's eye, the characteristics of the user's other eye, and the user's finger placement relative to the eye.

[0067] Example 16: A method described in any of Examples 13 to 15, further comprising providing an output prompt to the user via a user interface and based on one or more first images until an allowed user state is detected.

[0068] Example 17: The method of any of Examples 13-16, wherein the one or more user instructions for inserting a contact lens include voice instructions, visual instructions, or both.

[0069] Example 18: A method described in any of Examples 13 to 17, wherein analyzing the placement of the contact lens on the user's eye based on one or more second images includes detecting the edges of the contact lens using an edge detection model.

[0070] Example 19: A method described in any of Examples 13 to 18, wherein analyzing the placement of a contact lens on a user's eye based on one or more second images includes preprocessing the one or more second images using one or more of cropping, masking, removal, or a combination thereof.

[0071] Example 20: A method described in any of Examples 13 to 19, wherein analyzing the placement of a contact lens on a user's eye based on one or more second images includes preprocessing the one or more second images by digitally removing artifacts from the one or more second images.

[0072] Example 21: The method of any of Examples 13 to 20, wherein the artifact comprises the user's eyelashes.

[0073] Example 22: A method described in any of Examples 13 to 21, wherein analyzing the placement of the contact lens on the user's eye based on one or more second images includes detecting the edges of the contact lens using a mask for edge detection.

[0074] Example 23: The method of any of Examples 13 to 22, wherein the one or more first images are captured from a video.

[0075] Example 24: The method of any of Examples 13 to 23, wherein the one or more second images are captured from a video.

[0076] Example 25: The method of any of Examples 13 to 24, wherein the one or more first images are captured from real-time video.

[0077] Example 26: The method of any of Examples 13 to 25, wherein the one or more second images are captured from real-time video.

[0078] Example 27: A method for contact lens assistance, the method comprising: receiving one or more images of at least a portion of a user, the one or more images including a representation of the user's eye and a representation of a contact lens on the user's eye; analyzing the placement of the contact lens on the user's eye based on the one or more images; and outputting an indication of proper or improper placement of the contact lens based on the analysis of the placement of the contact lens.

[0079] Example 28: The method described in Example 27, wherein analyzing the placement of a contact lens on a user's eye based on one or more images includes detecting the edges of the contact lens using an edge detection model.

[0080] Example 29: A method described in any of Examples 27-28, wherein analyzing the placement of a contact lens on a user's eye based on one or more images includes preprocessing the one or more images using one or more of cropping, masking, removal, or a combination thereof.

[0081] Example 30: A method described in any of Examples 27 to 29, wherein analyzing the placement of a contact lens on a user's eye based on one or more images includes preprocessing the one or more images by digitally removing artifacts from the one or more second images.

[0082] Example 31: The method of any of Examples 27 to 30, wherein the artifact includes the user's eyelashes.

[0083] Example 32: A method described in any of Examples 27 to 31, wherein analyzing the placement of a contact lens on a user's eye based on one or more images includes detecting the edges of the contact lens using a mask for edge detection.

[0084] Example 33: The method of any of Examples 27 to 32, wherein one or more images are captured from a video.

[0085] Example 34: The method described in any of Examples 27 to 33, wherein one or more images are captured from real-time video.

[0086] Example 35: A method for contact lens assistance, the method comprising: a) receiving real-time video of at least a portion of a user, the video including an indication of the user's eyes; b) analyzing a user state of the user based at least on the video to determine whether the user state is an accepted user state or a rejected user state; c) generating, via a user interface and based on the video, output of visual cues indicative of the user's user state related to the user's eyes, the visual cues having a first state indicative of a rejected user state and a second state indicative of an accepted user state; d) generating, via a user interface and based on the video and based on the user state, output of one or more real-time prompts to be output to the user; and repeating steps a to d until the user state is determined to be an accepted user state.

[0087] Example 36: The method of example 35, wherein the permitted user condition includes a properly inserted contact lens on the user's eye.

[0088] Example 37: The method of any of Examples 35-36, wherein the permitted user condition includes a contact lens being properly removed from the user's eye.

[0089] Example 38: The method of any of Examples 35-37, wherein the one or more real-time prompts include instructions for inserting or removing contact lenses.

[0090] Example 39: A method for contact lens support, the method comprising: receiving a representation of at least a portion of a user's face during an attempt by the user to insert a contact lens into the eye; analyzing the display to determine if the user is in a known failure mode, the analysis including applying a machine learning model configured to recognize the known failure mode based on aggregated display of users in the known failure mode; and providing, via a user interface, real-time prompts known to assist a user in successfully inserting a contact lens into the eye.

[0091] Example 40: The method described in Example 39, wherein the display includes lidar data.

[0092] Example 40: The method of example 39, wherein the display includes video data.

[0093] Example 41: The method described in Example 39, wherein the failure mode includes one of: i) insufficient opening of the user's eye; ii) insufficient opening of the user's other eye; iii) improper placement of the user's finger relative to the eye; and iv) a lens that is too high or too low.

[0094] Example 42: The method described in Example 39, wherein analyzing the machine learning model is further configured to recognize a success mode and provide positive feedback to the user via a user interface.

[0095] Many operating systems, including Linux, UNIX®, OS / 2®, and Windows®, iOS®, and Android®, are capable of running many tasks simultaneously and are called multitasking operating systems. Multitasking is the ability of an operating system to run two or more executable files simultaneously. Each executable file runs in its own address space, which means that executables have no way to share any of their memory. Therefore, no program can impair the execution of any of the other programs running on the system. However, programs have no way to exchange any information other than through the operating system (or by reading files stored in the file system).

[0096] Multi-process computing is similar to multitasking computing, as the terms task and process are often used interchangeably, although some operating systems distinguish between the two. The present invention may be or include systems, methods, and / or computer program products of any possible integration of technical details. A computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to perform aspects of the present invention. A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction-execution device.

[0097] The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge-in-groove structures with instructions recorded on them, and any suitable combination of the above.

[0098] As used herein, a computer-readable storage medium should not be construed as a signal that is itself ephemeral, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through a wire. The computer-readable program instructions described herein may be downloaded from the computer-readable storage medium to a respective computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to a computer-readable storage medium within the respective computing / processing device for storage.

[0099] Computer-readable program instructions for carrying out the operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, Instruction-Set-Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object-oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0100] In some embodiments, electronic circuitry including, for example, programmable logic circuitry, a Field-Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), may execute computer-readable program instructions by utilizing state information in the computer-readable program instructions to individualize the electronic circuitry to perform aspects of the present invention.

[0101] Aspects of the present invention are described herein with reference to flowchart and / or block diagram illustrations of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart and / or block diagram illustrations, and combinations of blocks in the flowchart and / or block diagram illustrations, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to generate a machine such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / acts specified in one or more blocks of the flowchart and / or block diagram illustrations.

[0102] These computer-readable program instructions may also be stored on a computer-readable storage medium that can cause a computer, programmable data processing device, and / or other device to function in a particular manner, such that the computer-readable storage medium having the instructions stored thereon includes an article of manufacture containing instructions that implement aspects of the functions / acts specified in the flowchart and / or block diagram blocks. The computer-readable program instructions may be loaded onto a computer, other programmable data processing device, or other device to cause a series of operational steps to be performed on the computer, other programmable device, or other device, generating a computer-implemented process such that the instructions executing on the computer, other programmable device, or other device perform the functions / acts defined in the flowchart and / or block diagram blocks. The flowcharts and block diagrams in the figures illustrate the structure, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions that contains one or more executable instructions for implementing the illustrated logical function(s).

[0103] In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by dedicated hardware-based systems that perform particular functions or operations or execute a combination of dedicated hardware and computer instructions. While specific embodiments of the present invention have been described, those skilled in the art will recognize that there are other embodiments that are equivalent to the described embodiments. It is therefore to be understood that the present invention is not limited by the specific illustrated embodiments, but only by the scope of the appended claims.

[0104] From the above description, it can be seen that the present invention provides systems, computer program products, and methods for efficiently carrying out the described techniques. References to elements in the singular in the claims do not mean "one and only," unless explicitly stated otherwise, but rather "one or more." All structural and functional equivalents to the elements of the above exemplary embodiments, now known or later known to those skilled in the art, are intended to be encompassed within the scope of the claims. No claim element herein is to be construed under the provisions of 35 U.S.C. 112, sixth paragraph, unless the element is expressly recited using the phrase "means for" or "step for."

[0105] While the foregoing written description of the present invention will enable one of ordinary skill in the art to make and use what is presently believed to be its best mode, those skilled in the art will understand and recognize that there are alternatives, adaptations, variations, combinations, and equivalents to the specific embodiments, methods, and examples herein. Those skilled in the art will understand that the present disclosure is merely exemplary and that various modifications can be made within the scope of the present invention. It should be noted that while a particular feature of the present teachings may be disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of other implementations as may be desirable and advantageous for any given function or particular functions. Furthermore, to the extent that the terms "including," "includes," "having," "has," "with," or variations thereof are used in either the Detailed Description or the Claims, such terms are intended to be inclusive in a manner similar to the term "comprising."

[0106] Other embodiments of the present teachings will be apparent to those skilled in the art from consideration of the specification or practice of the teachings disclosed herein. Accordingly, the present invention should not be limited by the described embodiments, methods, and examples, but by all embodiments and methods within the scope and spirit of the present invention. Accordingly, the present invention is not limited to the specific embodiments shown herein, but is limited only by the following claims.

[0107] [Embodiment] (1) A method for contact lens assistance, said method comprising: generating an output of one or more lens type options via a user interface; receiving, via the user interface, an indication of a lens type to select from the one or more lens type options; causing output of one or more lens operation options via the user interface; receiving, via the user interface, an indication of a lens operation to be selected from the one or more lens operation options; generating an output of one or more eye options via the user interface; and receiving, via the user interface, an indication of a selected eye from the one or more eye options; causing output of one or more user instructions for inserting or removing a contact lens via the user interface and based on one or more of the selected lens type, the selected lens action, or the selected eye; The method, wherein the one or more user instructions incorporate user feedback based on images of the user captured in real time. (2) The method of claim 1, wherein the lens type options include one or more indications of reusable or daily disposable. (3) The method of claim 1, wherein the one or more lens operation options include one or more indications of inserting or removing a contact lens. (4) The method of embodiment 1, wherein the one or more eye options include one or more displays of the left eye or the right eye. (5) The method of embodiment 1, wherein the generating of the output of one or more user instructions for inserting or removing a contact lens is performed for the selected eye and further includes automatically generating the output of one or more user instructions for inserting or removing a contact lens for a non-selected eye of the one or more eye options via the user interface and based on one or more of the selected lens type or the selected lens action.

[0108] (6) The method of embodiment 1, wherein the one or more user instructions are selected from the group consisting of cleaning and drying instructions, contact lens handling instructions, contact lens position instructions, contact lens orientation instructions, finger position instructions, camera configuration instructions, user feedback queries, and combinations thereof. (7) A method for contact lens assistance, said method comprising: receiving one or more first images of at least a portion of a user, the one or more first images including a representation of an eye of the user; generating, via a user interface and based on the one or more first images, an output of visual cues indicative of a user state of the user relating to the eye of the user, the visual cues having a first state indicative of a rejected user state and a second state indicative of an accepted user state; causing output of one or more user commands for inserting a contact lens in response to the permitted user state; receiving one or more second images of at least a portion of the user, the one or more second images including a representation of the eye of the user and a representation of a contact lens on the eye of the user; analyzing the placement of the contact lens on the eye of the user based on the one or more second images; and outputting an indication of proper or improper placement of the contact lens based on analyzing the placement of the contact lens. (8) The method of embodiment 7, wherein the one or more first images include a representation of the iris of the eye of the user. (9) The method of embodiment 7, wherein the user state includes one or more of the degree of opening of the user's eye, characteristics of the user's other eye, and the user's finger placement relative to the eye. (10) The method of embodiment 7, further comprising providing an output prompt to the user via a user interface and based on the one or more first images until an allowed user state is detected.

[0109] (11) The method of embodiment 7, wherein the one or more user instructions for inserting a contact lens include voice instructions, visual instructions, or both. (12) The analysis is i) detecting edges of said contact lens using an edge detection model; ii) pre-processing the one or more second images using one or more of cropping, masking, removal, or a combination thereof; iii) pre-processing the one or more second images by digitally removing artifacts from the one or more second images; and iv) detecting the edges of the contact lens using an edge detection mask. (13) The method of embodiment 7, wherein the one or more first images or the one or more second images are captured from video, and optionally, real-time video. (14) A method for contact lens assistance, said method comprising: receiving one or more images of at least a portion of a user, the one or more images including a representation of an eye of the user and a representation of a contact lens on the eye of the user; analyzing the placement of the contact lens on the eye of the user based on the one or more images; and outputting an indication of proper or improper placement of the contact lens based on analyzing the placement of the contact lens. (15) The analysis is i) detecting edges of said contact lens using an edge detection model; ii) pre-processing the one or more images using one or more of cropping, masking, removal, or a combination thereof; iii) pre-processing the one or more second images by digitally removing artifacts from the one or more second images; and iv) detecting the edges of the contact lens using an edge detection mask.

[0110] (16) The method of embodiment 14, wherein the one or more images are captured from video, and optionally, real-time video. (17) A method for contact lens assistance, said method comprising: a. receiving real-time video of at least a portion of a user, the video including a representation of the user's eyes; b. analyzing a user state of the user based at least on the video to determine whether the user state is an accepted user state or a rejected user state; c. generating, via a user interface and based on the video, an output of visual cues indicating the user state of the user relating to the eyes of the user, the visual cues having a first state indicating a rejected user state and a second state indicating an accepted user state; d. causing output of one or more real-time prompts to be output to the user via the user interface and based on the video and based on the user state; e. repeating steps a to d until the user state is determined to be an allowed user state. (18) The method of claim 17, wherein the permitted user conditions include one or more of a properly inserted contact lens on the eye of the user and a properly removed contact lens on the eye of the user. (19) The method of embodiment 17, wherein the one or more real-time prompts include instructions for inserting or removing contact lenses. (20) A method for contact lens assistance, said method comprising: a. receiving a representation of at least a portion of a user's face during an attempt by the user to insert a contact lens into an eye; b. analyzing the indications to determine if the user is in a known failure mode, the analysis including applying a machine learning model configured to recognize the known failure mode based on aggregated indications of users in the known failure mode; c. providing, via a user interface, real-time prompts known to assist the user in successfully inserting the contact lens into the eye.

[0111] (21) The method of claim 20, wherein the display includes one or more of lidar data and video data. (22) The method of claim 20, wherein the failure mode includes one or more of: i) insufficient opening of the user's eye; ii) insufficient opening of the user's other eye; iii) improper placement of the user's finger relative to the eye; and iv) a lens that is too high or too low. (23) The method of embodiment 20, wherein analyzing the machine learning model is further configured to recognize a success mode and provide positive feedback to the user via the user interface.

Claims

1. A method for assisting contact lenses, wherein the method is Receiving one or more first images of at least a portion of the user, wherein the one or more first images include a representation of the user's eyes, To generate an output of a visual cue indicating the user state of the user with respect to the user's eyes, via a user interface and based on one or more first images, wherein the visual cue output has a first state indicating a rejected user state and a second state indicating an accepted user state. In response to an accepted user state, the system outputs one or more user commands for inserting contact lenses, Receiving one or more second images of at least a portion of the user, wherein the one or more second images include a representation of the user's eyes and a representation of the contact lenses on the user's eyes. Based on the one or more second images, the arrangement of the contact lenses on the user's eyes is analyzed, A method comprising outputting instructions for appropriate or inappropriate placement of the contact lenses based on an analysis of the arrangement of the contact lenses.

2. The method according to claim 1, wherein the one or more first images include a display of the iris of the user's eye.

3. The method according to claim 1, wherein the user state includes one or more of the degree of openness of the user's eye, the characteristics of the user's other eye, and the placement of the user's fingers on the eye.

4. The method according to claim 1, further comprising providing the user with output prompts via a user interface and based on one or more first images until an acceptable user state is detected.

5. The method according to claim 1, wherein the one or more user commands for inserting a contact lens include voice commands, visual commands, or both.

6. The above analysis is, i) Using an edge detection model to detect the edges of the contact lens, ii) Preprocessing one or more of the second images by using one or more of the following: cropping, masking, removal, or combination thereof, iii) Preprocessing the one or more second images by digitally removing artifacts from the one or more second images, The method according to claim 1, comprising one or more of the following: iv) detecting the edges of the contact lens using an edge detection mask.

7. The method according to claim 1, wherein the one or more first images or the one or more second images are captured from video and, optionally, from real-time video.

8. A method for assisting contact lenses, wherein the method is a. Receiving real-time video of at least a portion of the user, the video including a view of the user's eyes, b. Analyzing the user's state based at least on the video to determine whether the user's state is an acceptable or unacceptable user state, c. To generate, via a user interface and based on the video, an output of a visual cue indicating the user's state relating to the user's eyes, wherein the visual cue has a first state indicating a rejected user state and a second state indicating an accepted user state. d. To generate the output of one or more real-time prompts to be output to the user via the user interface, and based on the video and the user state, e. A method comprising repeating steps a to d until the user state is determined to be an acceptable user state.

9. The method according to claim 8, wherein the permitted user state includes one or more of the contact lenses properly inserted over the user's eye and the contact lenses properly removed from the user's eye.

10. The method according to claim 8, wherein the one or more real-time prompts include instructions for inserting or removing a contact lens.

Images