Refractive state determinations and related methods and apparatus

Abstract

The present invention relates to a system to assess myopia by measurement of diopters and axial length of an eye. The system comprises a smartphone with picture taking capability, a computer programming product installed on the smartphone capable of navigating the camera on the phone and extracting data from images. Another computer programming product is provided on the cloud and is capable of analyzing digital data from images of eyes for statistical features and assess refractive error in diopter and / or axial length of an eye thereof. The system further includes an optical assembly attachment attached to the back of the smartphone to guide light from a flashlight on a smartphone towards an eye for better image quality captured by the camera.

Description

REFRACTIVE STATE DETERMINATION AND RELATED METHODS ANDAPPARATUSBACKGROUND

[0001] Myopia is a growing epidemic around the world, with roughly 36% of children in the United States identified as having myopia. In Asia, the ratio of children with myopia is higher, with South Korea having a rate of 79% in children with myopia. In the world, an estimate of 30% of the adult population have some form of eye refractive error.

[0002] There are two prevailing thoughts on the causes on myopia: spending more time indoors, which means looking at close objects more often, causing the human eyes to adapt to make these distances more comfortable; and lack of sunlight, which causes an increase in chemical signals causing the retina to keep growing.

[0003] Myopia is a preventable problem. However, it requires early identification and monitoring of the treatment, both of which currently require a visit to an eye care professional for an eye exam. Current myopia monitoring can be infrequent, usually a few times per year, more often once per year. Fast onset myopia can develop rapidly in a little as a couple of months, in which case it would not be identified early enough with sporadic check-ups. In the U.S., current business models in the ophthalmic industry7are centered around retailing of spectacles and contact lenses. Therefore, having multiple myopia check-ups (resulting in no new spectacle or contact lenses sales sales) does not bring in revenue equivalent to servicing regular patients.

[0004] There exists a demand for detecting and monitoring development of myopia at home with a high degree of frequency and accuracy, and monitoring and confirming that myopia treatment is or has been effective. The present invention addresses this and other related needs in the art.SUMMARY

[0005] According to frequently included embodiments, there is provided a system for assessing myopia in human eyes, the system comprises: a smartphone with a camera capable of taking pictures of eyes and saving pictures as digital eye images;a computer programming product configured to separate the digital eye images into red, green, and blue channels, calculate statistical features of the images from the red, green, and blue channels, and calculate a refractive error in diopter and / or axial length of an eye using the statistical features; and a user interface capable of displaying the refractive error in diopter and / or axial length of the eye.

[0006] According to often included embodiments, there is provided a system as above, wherein the smartphone further comprises a flashlight configured to provide light in conjunction with picture taking by the camera in the smartphone.

[0007] According to frequently included embodiments, there is provided a system as above wherein the computer programming product is further configured to assess the quality of the pictures captured by the camera.

[0008] According to often included embodiments, there is provided a system as above, wherein the quality of the picture is assessed by resolution, lighting, focus, or blurriness.

[0009] According to frequently included embodiments, there is provided a system as above, wherein the computer programming product is further configured to locate the pupil image data with the image data.

[0010] According to frequently included embodiments, there is provided a system as above, wherein locating the pupil image comprises searching for an object with a circular or substantially circular boundary.

[0011] According to often included embodiments, there is provided a system as above, w herein the computer programming product is further configured to extract the pupil image data from the image data.

[0012] According to frequently included embodiments, there is provided a system as above, wherein the computer programming product is further configured to compress the pupil image data and upload the pupil image data to a cloud server.

[0013] According to often included embodiments, there is provided a system as above, wherein the calculation of statistical features of the image from the red, green, and blue channels is conducted by an artificial intelligence program with machine learning capabilities.

[0014] According to frequently included embodiments, there is provided a system as above, wherein the statistical features of the image are at least one of red mean, red median, red standard deviation, red geomean, red brightness weighted average, green mean, green median, green standard deviation, green geomean, green brightness weighted average; blue mean, blue median, blue standard deviation, blue geomean, blue brightness weighted average, blue mean - red mean, blue median - red median, blue standard deviation - red standard deviation, blue geomean - red geomean, blue brightness weighted average - red brightness weighted average; green mean - red mean, green median - red median, green standard deviation - red standard deviation, green geomean - red geomean, green brightness weighted average - red brightness weighted average; green mean - blue mean, green median - blue median, green standard deviation - blue standard deviation, green geomean - blue geomean, green brightness weighted average - blue brightness weighted average; blue mean / red mean, blue median / red median, blue standard deviation / red standard deviation, blue geomean / red geomean, blue brightness weighted average / red brightness weighted average; green mean / red mean, green median / red median, green standard deviation / red standard deviation, green geomean / red geomean, and green brightness weighted average / red brightness weighted average. wherein red mean refers to the mean intensity of all red pixels in the image, blue mean refers to the mean intensity of all blue pixels in the image, green mean refers to the mean intensity of all green pixels in the image, wherein red geomean refers to the geometric mean intensity of all red pixels, green geomean refers to the geometric mean intensity of all green pixels, blue geomean refers to the geometric mean intensity of all blue pixels, wherein red median refers to the median intensity of all red pixels in the image, green median refers to the median intensity of all green pixels in the image, blue median refers to the median intensity of all blue pixels in the image, wherein red brightness average refers to the average of brightness of all red pixels in the image, green brightness average refers to the average of brightness of all green pixels in the image, blue brightness average refers to the average of brightness of all blue pixels in the image, wherein red standard deviation refers to the amount of variations of red pixels in the image, blue standard deviation refers to the amount of variations of blue pixels in the image, green standard deviation refers to the amount of variations of green pixels in the image,wherein blue mean - red mean refers the mean intensity of all blue pixels in the image minus by mean intensity of all red pixels in the image, green mean - red mean refers the mean intensity of all green pixels in the image minus by mean intensity of all red pixels in the image, green mean - blue mean refers the mean intensity of all green pixels in the image minus by mean intensity of all blue pixels in the image, wherein blue median - red median refers to the median intensity of all blue pixels in the image minus by median intensity of all red pixels in the image, green median - blue median refers to the median intensity of all green pixels in the image minus by median intensity of all blue pixels in the image, green median - red median refers to the median intensity' of all green pixels in the image minus by median intensity of all red pixels in the image, wherein blue standard deviation - red standard deviation refers to the amount of variations of blue pixels in the image minus the amount of variations of red pixels in the image, green standard deviation - blue standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of blue pixels in the image, green standard deviation - red standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of red pixels in the image, wherein blue geomean - red geomean refers to the geometric mean intensity of all blue pixels in the image minus by geometric mean intensity of all red pixels in the image, green geomean - blue geomean refers to the geometric mean intensity' of all green pixels in the image minus by geometric mean intensity' of all blue pixels in the image, green geomean - red geomean refers to the geometric mean intensity' of all green pixels in the image minus by geometric mean intensity of all red pixels in the image. wherein blue brightness weighted average - red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average - blue brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of blue pixel brightness values that exceed the median brightness level, green brightness weighted average - red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness levelminus by the weighted average of red pixel brightness values that exceed the median brightness level, wherein blue mean / red mean refers the mean intensity of all blue pixels in the image divided by mean intensity of all red pixels in the image, green mean / red mean refers the mean intensity of all green pixels in the image divided by mean intensity of all red pixels in the image, wherein blue median / red median refers to the median intensity of all blue pixels in the image divided by the median intensity of all red pixels in the image, green median / red median refers to the median intensity of all green pixels in the image divided by the median intensity of all red pixels in the image. wherein blue standard deviation / red standard deviation refers to the amount of variations of blue pixels in the image divided by the amount of variations of red pixels in the image, green standard deviation / red standard deviation refers to the amount of variations of green pixels in the image divided by the amount of variations of red pixels in the image, wherein blue geomean / red geomean refers to the geometric mean intensity of all blue pixels in the image divided by geometric mean intensity of all red pixels in the image, green geomean / red geomean refers to the geometric mean intensity of all green pixels in the image divided by geometric mean intensity of all red pixels in the image, wherein blue brightness weighted average / red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average / red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level.

[0015] According to often included embodiments, there is provided a system as above, wherein the computer programming product is further configured to use the artificial intelligence program to calculate refractive error in diopter and / or axial length of an eye from the statistical features.

[0016] According to frequently included embodiments, there is provided a system as above, wherein the artificial intelligence program comprises two distinct machine learning models, and wherein one of the two distinct machine learning models calculates diopters of theeye using the eye image, and the other of one the two distinct machine learning models calculates axial length of the eye using the eye image.

[0017] According to often included embodiments, there is provided a system as above, wherein the machine learning models use a Support Vector Machine algorithm to select appropriate statistical features for classification and rank the statistical features based on their contribution to either diopter or axial length prediction using Minimum Redundancy Maximum Relevance.

[0018] According to frequently included embodiments, there is provided a system as above, wherein the user interface is displayed on the smartphone within the system.

[0019] According to often included embodiments, there is provided a system as above, wherein the user interface is displayed on a computing article other than the smartphone within the system.

[0020] According to frequently included embodiments, there is provided a system as above, the system further comprises an optical assembly attachment the optical assembly attachment comprises two polarizers, a mirror, a beam splitter, and a converging lens, wherein the optical assembly attachment is configured to attach to the back of a smartphone.

[0021] According to often included embodiments, there is provided a system as above, wherein one of the two polarizers is situated above a flashlight on the smartphone, the mirror is situated above and at an angle to the polarizer above the flashlight, the beam splitter is situated parallel to the mirror, the converging lens is situated above the camera between the camera and the beam splitter, and the other of the two polarizers is situated above and parallel to the converging lens.

[0022] According to frequently included embodiments, there is provided a system as above, wherein the optical assembly attachment further comprises a light trap to diffuse light from the beam splitter.

[0023] According to often included embodiments, there is provided a system as above, wherein the optical assembly attachment further comprises a back plate to attach the optical assembly attachment to the smartphone.

[0024] According to frequently included embodiments, there is provided a system as above, wherein the computer programming product is further configured to analyze the locationof the camera relative to a human eye and give commands for maneuvering the camera to a predetermined location.

[0025] According to often included embodiments, there is provided a system as above, wherein the commands are given in by voice or by visual display.

[0026] According to frequently included embodiments, there is provided a computer method to assess myopia of an eye, the method comprising: taking a picture of an eye using a camera and saving the picture as digital image data; locating a pupil image within the digital image data; extracting the pupil image data from the picture of the eye; separating image data into red, green, and blue channels; calculating statistical features of the image from red, green, and blue channels; using a Support Vector Machine algorithm to select appropriate statistical features for classification and rank the statistical features based on their contribution to either diopter or axial length prediction using Minimum Redundancy Maximum Relevance; and calculate diopter or axial length of the eye based on the statistical features ranking.

[0027] According to often included embodiments, there is provided a method as above, wherein locating a pupil image within the digital image data comprises searching for an object with a circular or substantially circular boundary.

[0028] According to often included embodiments, there is provided a method as above, wherein the statistical features of the image are at least one of red mean, red median, red standard deviation, red geomean, red brightness weighted average, green mean, green median, green standard deviation, green geomean, green brightness weighted average; blue mean, blue median, blue standard deviation, blue geomean, blue brightness weighted average, blue mean - red mean, blue median - red median, blue standard deviation - red standard deviation, blue geomean - red geomean, blue brightness weighted average - red brightness weighted average; green mean - red mean, green median - red median, green standard deviation - red standard deviation, green geomean - red geomean, green brightness weighted average - red brightness weighted average; green mean - blue mean, green median - blue median, green standard deviation - blue standard deviation, green geomean - blue geomean, green brightness weighted average - blue brightnessweighted average; blue mean / red mean, blue median / red median, blue standard deviation / red standard deviation, blue geomean / red geomean, blue brightness weighted average / red brightness weighted average; green mean / red mean, green median / red median, green standard deviation / red standard deviation, green geomean / red geomean, and green brightness weighted average / red brightness weighted average, wherein red mean refers to the mean intensity of all red pixels in the image, blue mean refers to the mean intensity of all blue pixels in the image, green mean refers to the mean intensity' of all green pixels in the image, wherein red geomean refers to the geometric mean intensity of all red pixels, green geomean refers to the geometric mean intensity of all green pixels, blue geomean refers to the geometric mean intensity' of all blue pixels, wherein red median refers to the median intensity' of all red pixels in the image, green median refers to the median intensity of all green pixels in the image, blue median refers to the median intensity of all blue pixels in the image, wherein red brightness average refers to the average of brightness of all red pixels in the image, green brightness average refers to the average of brightness of all green pixels in the image, blue brightness average refers to the average of brightness of all blue pixels in the image, wherein red standard deviation refers to the amount of variations of red pixels in the image, blue standard deviation refers to the amount of variations of blue pixels in the image, green standard deviation refers to the amount of variations of green pixels in the image, wherein blue mean - red mean refers the mean intensity of all blue pixels in the image minus by mean intensity' of all red pixels in the image, green mean - red mean refers the mean intensity of all green pixels in the image minus by mean intensity of all red pixels in the image, green mean - blue mean refers the mean intensity of all green pixels in the image minus by mean intensity of all blue pixels in the image, wherein blue median - red median refers to the median intensity of all blue pixels in the image minus by median intensity of all red pixels in the image, green median - blue median refers to the median intensity' of all green pixels in the image minus by median intensity' of all blue pixels in the image, green median - red median refers to the median intensity’ of all green pixels in the image minus by median intensity of all red pixels in the image,wherein blue standard deviation - red standard deviation refers to the amount of variations of red pixels in the image minus the amount of variations of red pixels in the image, green standard deviation - blue standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of blue pixels in the image, green standard deviation - red standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of red pixels in the image, wherein blue geomean - red geomean refers to the geometric mean intensity of all blue pixels in the image minus by geometric mean intensity of all red pixels in the image, green geomean - blue geomean refers to the geometric mean intensity of all green pixels in the image minus by geometric mean intensity of all blue pixels in the image, green geomean - red geomean refers to the geometric mean intensity of all green pixels in the image minus by geometric mean intensity of all red pixels in the image. wherein blue brightness weighted average - red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average - blue brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of blue pixel brightness values that exceed the median brightness level, green brightness weighted average - red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level, wherein blue mean / red mean refers the mean intensity of all blue pixels in the image divided by mean intensity of all red pixels in the image, green mean / red mean refers the mean intensity of all green pixels in the image divided by mean intensity of all red pixels in the image, wherein blue median / red median refers to the median intensity of all blue pixels in the image divided by the median intensity of all red pixels in the image, green median / red median refers to the median intensity of all green pixels in the image divided by the median intensity of all red pixels in the image, wherein blue standard deviation / red standard deviation refers to the amount of variations of blue pixels in the image divided by the amount of variations of red pixels in the image, greenstandard deviation / red standard deviation refers to the amount of variations of green pixels in the image divided by the amount of variations of red pixels in the image, wherein blue geomean / red geomean refers to the geometric mean intensity of all blue pixels in the image divided by geometric mean intensity of all red pixels in the image, green geomean / red geomean refers to the geometric mean intensity of all green pixels in the image divided by geometric mean intensity of all red pixels in the image, wherein blue brightness weighted average / red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average / red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level.

[0029] According to often included embodiments, there is provided a method as above, wherein the calculating of diopter and axial length is conducted by two distinct machine learning models.

[0030] According to frequently included embodiments, there is provided a method as above, wherein the camera is present on a smartphone and the method is conducted by a computer programming product installed on the same smartphone.

[0031] According to often included embodiments, there is provided a method as above, the method further comprises assessing the quality of the picture captured by the camera.

[0032] According to frequently included embodiments, there is provided a method as above, wherein the quality of the picture is assessed by resolution, lighting, focus, or blurriness.

[0033] According to often included embodiments, there is provided a method as above, the method further comprises compressing the pupil image data and upload the pupil image data to a cloud server, wherein the cloud server hosts the machine learning models.

[0034] According to frequently included embodiments, there is provided a method as above, the method further comprises displaying the diopter or axial length calculation result on a user interface.

[0035] According to often included embodiments, there is provided a method as above, the method further comprises sending the diopter or axial length calculation result to a computing article.

[0036] According to frequently included embodiments, there is provided a method as above, the method further comprises attaching an optical assembly to the smartphone, the optical assembly comprises two polarizers, a mirror, a beam splitter, and a converging lens, wherein the optical assembly attachment is configured to attach to the back of a smartphone.

[0037] According to often included embodiments, there is provided a method as above, wherein one of the two polarizers is situated above a flashlight on the smartphone, the mirror is situated above and at an angle to the polarizer above the flashlight, the beam splitter is situated parallel to the mirror, the converging lens is situated above the camera between the camera and the beam splitter, and the other of the two polarizers is situated above and parallel to the converging lens.

[0038] According to frequently included embodiments, there is provided a method as above, wherein the optical assembly attachment further comprises a light trap to diffuse light from the beam splitter.

[0039] According to often included embodiments, there is provided a method as above, wherein the optical assembly attachment further comprises a back plate to attach the optical assembly attachment to the smartphone.

[0040] According to frequently included embodiments, there is provided a method as above, the method further comprises analyzing the location of the camera relative to an eye and giving commands for maneuvering the camera to a predetermined location.

[0041] These and other embodiments, features, and advantages will become apparent to those skilled in the art when taken with reference to the following more detailed description of various exemplary embodiments of the present disclosure in conjunction with the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The skilled person in the art will understand that the drawings, described below, are for illustration purposes only.

[0043] FIG. 1 depicts a perspective view of an optical assembly attachment according to embodiments herein, as attached to a smartphone.

[0044] FIG. 2 depicts a top view of the optical assembly attachment as attached to the smartphone in FIG. 1.

[0045] FIG. 3 depicts a cross-sectional view of the optical assembly attachment as attached to the smartphone, along line D - D shown in FIG. 2.

[0046] FIG. 4 depicts a side view of the optical assembly attachment as attached to the smartphone in FIG. 1.

[0047] FIG. 5 depicts a cross-sectional view of the optical assembly attachment as attached to the smartphone, along line A - A shown in FIG. 4.

[0048] FIG. 6 depicts a cross-sectional view of the optical assembly attachment as attached to the smartphone, along line B - B shown in FIG. 4.

[0049] FIG. 7 depicts the operation of the optical assembly attachment in directing light rays during use with a smartphone having an embedded camera and flashlight.

[0050] FIG. 8 depicts light rays travelling paths within the optical assembly attachment, shown as a cross-sectional view of the optical assembly attachment as attached to the smartphone along line D - D shown in FIG. 2.

[0051] FIG. 9 depicts the flow chart of a refractive determination process using a smart phone with an optical assembly attachment according to embodiments of the present invention.

[0052] FIG. 10 depicts the flow chart of a refractive determination process using a smart phone without an optical assembly attachment according to embodiments of the present invention.

[0053] FIG. 11 depicts the flow chart of a digital image processing procedure performed by an apparatus according to embodiments of the present invention.

[0054] FIG. 12 is a graph showing the relationship between Actual Refractive Error v. Measured Refractive Error, with Measured Refractive Error values measured in a human eye using the system according to embodiments herein.

[0055] FIG. 13 depicts a smartphone with an optical assembly attachment photographing a model eye according to methods described herein.

[0056] FIG. 14 is a graph showing the axial length of the model eye, as simulated by moving the piston and as measured by Zeiss OIL Master 300.

[0057] FIG. 15 is a graph showing the Bland-Altmann plot of the axial length of the model eye, as simulated by moving the piston and as measured by Zeiss OIL Master 300.

[0058] FIG. 16 is a graph show ing the refractive error of the model eye, as simulated by moving the piston and as measured by Shin Nippon K900.

[0059] FIG. 17 is a graph showing the Bland- Altmann plot of the refractive error of the model eye. as simulated by moving the piston and as measured by Shin Nippon K900.

[0060] FIG. 18 depicts a smartphone with an optical assembly attachment photographing a model eye with glasses simulating a refractive error.

[0061] FIG. 19 is a plot showing Support Vector Machine performance against number of features in refractive error measurement data analysis.

[0062] FIG. 20 is a graph showing Support Vector Machine performance against linear, quadratic, and cubic functions in refractive error measurement data analysis.

[0063] FIG. 21 is a plot showing Support V ector Machine performance against Gaussian Kernel Scale in refractive error measurement data analysis.

[0064] FIG. 22 is a plot showing Support Vector Machine performance against number of features in axial length measurement data analysis.

[0065] FIG. 23 is a graph showing Support Vector Machine performance against linear, quadratic, and cubic functions in axial length measurement data analysis.

[0066] FIG. 24 is a plot showing Support Vector Machine performance against Gaussian Kernel Scale in axial length measurement data analysis.DETAILED DESCRIPTION

[0067] For clarity of disclosure, and not by way of limitation, the detailed description of the invention is divided into the subsections that follow.

[0068] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as is commonly understood by one of ordinary skill in the art to which this invention belongs. All patents, applications, published applications and other publications referred to herein are incorporated by reference in their entirety. If a definition set forth in this section is contrary to or otherwise inconsistent with a definition set forth in the patents, applications, published applications and other publications that are herein incorporated by reference, the definition set forth in this section prevails over the definition that is incorporated herein by reference.

[0069] As used herein, “a” or ‘'an” means '‘at least one” or ‘'one or more.”

[0070] As used herein, the term “and / or” may mean “and,” it may mean “or,” it may mean “exclusive-or,” it may mean “one,” it may mean “some, but not all,” it may mean “neither,” and / or it may mean “both.”

[0071] In embodiments, the present invention discloses a system comprising a smartphone with photo capture capability and an optical assembly attachment to assist with directing light and enhancing image quality, the system is equipped and configured to measure refractive states of a human eye and output a result on the refractive state. The present invention also discloses computer method to measure refractive states of a human eye using this system.

[0072] In embodiments, a smartphone with photo capture capability is equipped w ith a computer program product, such as an application operable on a smartphone, the computer program product is configured to receive data on an image captured by the smart phone.

[0073] In embodiments, an optical assembly attachment is provided as part of this system, as different smartphones have different cameras and different optical systems. The optical assembly attachment comprises a mirror, a beam splitter (polarizer), a converging lens, and polarizers. The lens gives a larger image of the eye, resulting in more pixels within the eye image, hence more data is available. The mirror and beam splitter ensure that light source is coaxial with the camera, maximizing the amount of data. This optical assembly attachment is intended for phones not meeting the minimum requirements to perform the measurements required for operation of the system according to embodiments herein. The mechanical attachment will vary based on the model of phone. The mechanical attachment allows more smartphones to be suitable for use with this application. Any system that generates a “brightpupil” retro reflection can use this optical assembly attachment to reduce the “bright pupil effect.

[0074] In embodiments, the optical assembly attachment is attached to the back of a smartphone for maneuvering. FIG. 1 is a perspective view of the optical assembly attachment attached to a smartphone with a camera. The optical assembly attachment 100 comprises a back plate 107 and a body 108. Within the body 108, a beam splitter, a flash mirror, two linear polarizers, and a light trap are arranged. The back plate 107 is attached to the smartphone 10 without rotation capability. The optical assembly attachment 100 is attached to the back of the smartphone by glue, rivets, magnetic, or other chemical, mechanical, or physical forces between the back plate 107 and the smartphone back.

[0075] FIG. 2 depicts a top view of the optical assembly attachment 100 as attached to the back of a smartphone 10. The optical assembly attachment 100 covers the flashlight and the camera 11 on the smartphone, but the camera can still receive light through a lens and a polarizer.

[0076] FIG. 3 depicts the cross-sectional view along line D - D shown in FIG. 2. A first linear polarizer 101 is situated within the body 107 of the optical assembly attachment 100 such that it is placed directly above the flashlight on a smartphone when the optical assembly attachment 100 is attached to the smartphone. Above the first linear polarizer 101 is an aperture 110 for light to pass through before hitting a flash mirror 102, the flash mirror 102 is located above the first linear polarizer 101 at an angle above the first linear polarizer 101. A 50 / 50 beam splitter 105 is situated opposite to the flash mirror 102 and at the same angle with the flash mirror 102, such that the 50 / 50 beam splitter 105 and the flash mirror 102 are parallel to each other. Below the 50 / 50 beam splitter 105 and towards the smartphone, a second linear polarizer 103 is located such that it is placed above the converging lens 104 of the smartphone camera. A chamber with a light trap 106 is located to the side of the 50 / 50 beam splitter, opposite from the flash mirror. Components of the body is enclosed within a housing.

[0077] FIG. 4 depicts the side view of the optical assembly attachment 100 attached to the back of a smartphone 10. It can be seen that the body of the optical assembly attachment 100 is slight raised above the back of the smartphone 10 when the optical assembly attachment 100 is attached to the smartphone 10.

[0078] FIG. 5 depicts the cross-sectional view of FIG. 4 along line A - A. From this view, converging lens 104 can be seen with second linear polarizer 103 above it as the body ofthe optical assembly attachment 100 is placed above the smartphone 10 during use. FIG. 6 depicts the cross-sectional view of FIG. 4 along line B - B. From this view, first linear polarizer 101 can be seen with aperture 110 above and extending to flash mirror 102. First linear polarizer 101 sits on top of a flashlight available on the smartphone 10 when the optical assembly attachment 100 is attached to the smartphone 10.

[0079] FIG. 7 illustrates the operation mechanism of the optical assembly attachment. When a camera within a smartphone takes a picture with the flashlight function turning on, the flashlight produces light. This light travels from the flashlight 111 and reaches the polarizer 101 within the optical assembly attachment. Light then travels through the polarizer 101 to an aperture 110 located behind the polarizer 101 to reach the flash mirror 102. This flash mirror is set up at an angle with the aperture 110, such that light travelling through the aperture 110 hits the flash mirror 102 and deflects from the flash mirror 102 to a beam splitter 105 sets parallel to the flash mirror 102. Light after hitting the beam splitter then deflects and reaches the eyeball 20, namely the pupil of the eye. A picture of the eye is thus taken with light coming from the flashlight. The dotted lines in FIG. 7 denote rays of light from the flashlight reaching the human eye. When a picture is taken, an outgoing ray, denoted by a broken line, goes from the eye to the camera, thereby capturing an image of the eye. The outgoing ray passes through a polarizer 103 and a converging lens 104 before going into the camera 11. This eye image is saved as digital image data and is used for analysis by an algorithm described below.

[0080] The light travelling path when the optical assembly attachment is attached to the smartphone and a photo is taken by the smartphone with the flashlight function activated is as follows. Light from the flashlight 111 travels to the first polarizer 101 and is polarized when passing through a filter in the first polarizer 101 before hitting the flash mirror 102. Light is then reflected back from the mirror 102 and reaches the 50 / 50 beam splitter 105. Fifty percent (50%) of the light reaching the beam splitter 105 is diffused in the light trap (not shown in FIG. 7) and 50% of the light travels through the beam splitter 105 to reach the object to be photographed, which is a human eye.

[0081] Light reaching the eye travels back to the camera and reaches the beam splitter 105, where 50% of the light is diffused to the light trap. Thus, only 25% of the light initially mitted by the flashlight makes it back to the camera. Light coming back from the human eye goes through a second linear polarizer 103 placed in front of a converging lens 104 before reaching the camera.

[0082] FIG. 8 depicts the light travelling path of FIG. 7 but as shown inside the optical assembly attachment 100. The Field of Vision (FOV) 112 of the camera is the area within which light travel from the eye back to the camera. Light path starts at the flashlight travelling through the first linear polarizer 101 before reaching the flash mirror 102, the beam splitter 105, then the eye 20, before travelling back from the eye 20 to the beam splitter 105, then to the second linear polarizer 103 then the converging lens 104 and arriving at the camera within the smartphone.

[0083] Smartphones used in this system according to embodiments are smartphones with photo capture capability. Photos captured by smartphones are saved and stored as files containing data. Images are stored as pixels, each pixel being the smallest unit of the image. In some embodiments, colors in images are stored as red green blue (RGB) color scheme images. Colors in images can also be stored in other color schemes, including RGB-IR (RGB -Infrared). Each pixel is thus capable of being analyzed for color composition and Bayer pattern.

[0084] Once a photo of a human eye anterior is captured by a smartphone with an optical assembly attachment according to embodiments herein, the image is transmitted to the computer programming product installed in the smartphone. The computer programming product identifies the pupil image within the eye image and extracts the pupil image from the eye image. Within the eye image, red, blue, and green pixels are identified and separated to create red, blue, and green channels. The computer program product then extracts digital data from the image and transmits the digital data within the image to the cloud for analysis using artificial intelligence (Al). Al then generates statistical features for the different colors, then generates further statistical features comparing the different colors. From the statistical features, a visual acuitymeasurement or axial length measurement result is produced.

[0085] In embodiments, two distinct machine learning models are provided, one for predicting prescription power, quantified in diopters, and another for estimating axial length, measured in millimeters. Each learning model uses a Support Vector Machine (SVM) algorithm to select appropriate features for classification. These features were subjected to a Minimum Redundancy Maximum Relevance (MRMR) algorithm to rank them based on their contribution to either prescription power (diopters) or axial length prediction. Features scoring below the median relevance, as determined by MRMR, were subsequently excluded from the classification process. The pruned dataset, consisting only of the selected features, was then partitioned into separate training and testing sets. The training set was used to train the SVM model, while the testing set served to evaluate the model on previously unseen data. This entire process wasiteratively repeated, utilizing different portions of the complete dataset as testing sets, to assess model performance and identify any specific vulnerabilities.

[0086] In embodiments, images of eyes taken by a camera and saved in RGB color scheme are divided into pixels of red. green, or blue. From these pixels, statistical features are extracted: red mean, red median, red standard deviation, red geomean, red brightness weighted average; green mean, green median, green standard deviation, green geomean, green brightness weighted average; blue mean, blue median, blue standard deviation, blue geomean, blue brightness weighted average; blue mean - red mean, blue median - red median, blue standard deviation - red standard deviation, blue geomean - red geomean, blue brightness weighted average - red brightness weighted average; green mean - red mean, green median - red median, green standard deviation - red standard deviation, green geomean - red geomean, green brightness weighted average - red brightness weighted average; green mean - blue mean, green median - blue median, green standard deviation - blue standard deviation, green geomean - blue geomean, green brightness weighted average - blue brightness weighted average; blue mean / red mean, blue median / red median, blue standard deviation / red standard deviation, blue geomean / red geomean, blue brightness weighted average / red brightness weighted average; green mean / red mean, green median / red median, green standard deviation / red standard deviation, green geomean / red geomean, green brightness weighted average / red brightness weighted average. Note that brightness weighted average is the dominant brightness of the image and is quantified by computing the weighted average of pixel values that exceed the median brightness level. Other statistical features may be extracted from the pixels.

[0087] Red mean refers to the mean intensify of all red pixels in the image. Red geomean refers to the geometric mean intensity of all red pixels. Red median refers to the median intensity of all red pixels in the image. Red brightness average refers to the average of brightness of all red pixels in the image. Red standard deviation refers to the amount of variations of red pixels in the image. Similar definition applies to other colors. "‘Green mean / red mean"’ refers the mean intensity of all green pixels in the image divided by mean intensity of all red pixels in the image. “Blue mean - red mean” refers the mean intensity of all blue pixels in the image minus by mean intensify of all red pixels in the image. “Blue geomean - red geomean” refers to the geometric mean intensity of all blue pixels in the image minus by geometric mean intensify of all red pixels in the image. Similar definition applies to other colors and other calculations, with denotesminus and ‘7” denotes divide. Other statistical features as above can be discerned with appropriate colors.

[0088] At the outset, images of eyes with known refractive error in diopter and / or axial length are provided. The above statistical features are calculated from the initial set of images and several of them are chosen and used to evaluate their potential in accurately deducing the diopter and / or axial length of the eye. The Support Vector Machine (SVM) algorithm analyzes and classifies these statistical features against its correlation to diopter and / or axial length values. The statistical features with the strongest correlations are chosen and subject to a Minimum Redundancy Maximum Relevance (MRMR) algorithm. The MRMR algorithm ranks the statistical features according to their contribution to diopters or axial length. Statistical features scoring below median relevance, as determined by MRMR, were subsequently excluded from the classification process. Statistical features not excluded are included in a dataset for use in measurement of diopters and axial length. Statistical features and their relevance may change as the machine learning process continues. Statistical features can be ranked by their relevancy, such that their ranking may be changed depending on specific circumstances and feedback during the machine learning process.

[0089] These statistical features are correlated with the known diopters and / or axial length of the eyes of which the images are taken. This initial dataset is used for the initial training of the SVM algorithm. Once the machine learning models are sufficiently trained, the models can be used. Data collected from usage is fed into the algorithm for continuous learning.

[0090] Once trained, the algorithm can receive an image of an eye in RGB pixels, calculate statistical features from the image, and use the statistical features in the algorithms embedded in the computing article to calculate diopters and / or axial length of the eye. The algorithm forms a part of the computer programming product operable on a computing article, such as a smartphone, the computer programming product is configured to receive images of an eye from the smartphone and analyze the images for diopter or axial length reading of the eye.

[0091] FIG. 9 depicts the operating sequence of a smartphone equipped with the system according to embodiments herein, including an optical assembly attachment. The optical assembly attachment is first attached to the back of the smartphone. The computer programming product starts to calibrate the device. Upon successfully calibration, the system is maneuvered to the correct location for picture taking, using voice or visual guidance from a computerprogramming product installed on the smartphone. Once the correct position is obtained, the computer programming product takes a picture of the eye, which can happen automatically or by pressing a button. The flash function on the smartphone is used, and the light emitted from the flashlight passes through the optical assemble attachment as described herein, resulting in the light emitted by the flashlight returning to the camera at 25% strength. The image captured is stored in digital data in smartphone and processed by the computer programming product as described below.

[0092] The system then analyzes the image for quality assessment. Quality of images include whether the front of the eye is captured in its entirety in the image, the resolution of the image, whether the image was taken in adequate lighting, whether the image was taken at a still, whether the image is in focus, whether the image is blurry, or other features defined in the computer programming product. Good quality images are processed in the next step, otherwise the apparatus is maneuvered to a different location for picture taking. When the image quality is good, the image is processed in a next step. From the good image, the computer programming product locates and extracts the pupil by analyzing the pixels in the relevant area of the images. The computer programming product model is trained to recognize areas with features corresponding to the pupil in the eye, such as an object with a circular or near circular boundary. The digital data of the pupil image is then compressed to upload to a cloud server. If uploading is not successful, the computer programming product attempts to upload again. After the nthattempt at uploading and uploading is still unsuccessful, with “n"’ being an integer greater than 1, the process will terminate early with a termination notice displayed at a user interface.

[0093] Upon successful uploading, the digital data of the image is then processed by statistical analysis using the Al model as above. If the processing is unsuccessful, i.e. no results can be drawn from the statistical analysis using the Al model, the computer programming product attempts to re-upload the extracted pupil image to the cloud server. After an nthattempt at processing and processing is still unsuccessful, the process will terminate early with a termination notice displayed at a user interface. After successful processing, results are retrieved and displayed in the user interface, which may be available on the smartphone and / or on a dashboard available on a different computer article. Note that “n” is an integer greater than 1 and can be set within the computer programming product.

[0094] FIG. 10 depicts the operating sequence of a smartphone equipped with the computer programming product according to embodiment herein, but without an opticalassembly atachment. The operating sequence remains essentially the same, however, there is no optical assembly atachment. The light from the flashlight on the smartphone thus travels to the eye and back to the camera without being filtered by the mirror, polarizer, or beam spliter within the optical assembly atachment.

[0095] FIG. 1 1 depicts the operating sequence of image processing to analyze an image taken of an eye for myopia monitoring. This operating sequence is denoted as "Process Image” in FIGS. 9 and 10. Once the Al algorithm located in the cloud server has received digital data of an image in compressed form, it decompresses the data. Upon decompression, the data is separated into red, green, and blue channels based on the RGB color scheme. Statistical features are extracted from the red, green and blue channels of the image as above. If the extraction is unsuccessful, e.g. image quality is low and meaningful statistical features are undecipherable, the Al algorithm returns an error to the computer programming product, and the computer programming product atempts to upload the compressed image data again. After a set amount of failed atempt, the computer programming product may return to taking another image or terminating the procedure. When statistical features are successfully extracted, the features will be analyzed by the Artificial Intelligence algorithm as described above. The result from the algorithm is retrieved by the computer program product and are sent to the smartphone and / or on a dashboard available on a different computing article. The results may be displayed at a user interface on a computing article, including the smartphone that captured the eye image.

[0096] In embodiments, a user interface is provided on the smartphone as part of the computer program product used in this system. The user interface is configured to display results from the photo capture and analysis. High risk issues, such as double vision or significant refractive errors are flagged with visual red indicators. Axial length measurements are provided, together with other indicators, such as confidence level that each test was performed correctly. Graphs showing measurement results over time and trend of results are also available for displayed. Other features, such as notifications to others via electronic communication of the results, orders for subsequent steps, or prescription are also available. The computer programming product is configured to electronically communicate with cloud systems and / or software systems used by medical professionals to exchange data.

[0097] In embodiments, the initial dataset used to train the Al algorithm can be from images of human eyes with known axial length and / or diopter.Obtaining Dataset with a Human Eye

[0098] An initial dataset was obtained by using the system according to embodiment herein, comprising a smartphone, an optical assembly attachment, and a computer programming product to take pictures of a human eye, then upload the images to the cloud server to be processed by an Al algorithm according to embodiments herein. Pictures of a human eye were taken using an iPhone 14 pro having a built-in camera. The pictures are analyzed by Al algorithm and results were given in diopter.

[0099] Measurement of refractive error using a system according to embodiment herein was performed on a human. Refractive error was induced with glasses, with each pair of glasses being placed in front of the human’s eyes. For each pair of glasses, 15 images are captured by the iPhone 14 having a built-in camera. The human was given 30 seconds of rest between each pair of glasses. A total of 30 sets of images were taken, amounting to 2250 images in total.

[0100] The total number of measurements taken is 2452, with the number of erroneous measurements being 83. The error rate is thus 3.39%. The average error in measurement is 0.46D (diopters). The results are summarized in Table 1 :Table 1

[0101] The Expected Value is the value in diopter given by the Al algorithm after analysis of the eye image. The Measured Value is the value in diopter as measured by auto refractor Shin Nippon K900. A confusion matrix analysis was prepared and Positive Error and Negative Error from the confusion matrix are included in Table 1. A graph showing arelationship between Actual v. Measured Value is shown in FIG. 12. Note that in FIG. 12, “Actual Value'’ is the same as “Expected Value'’.

[0102] Discussion

[0103] The average error is 0.03, which is under 1 / 3 of a temporal resolution of 1.5D. For +3.0D measurements point, the error was 0.00 across 477 measurement. In the graph shown in FIG. 12, the linear equation is y = 1.0312x - 0.0301. For a perfect fit of linear equation to dataset, a gradient of 1 and intercept of 0 is required. The dataset collected from measuring the human eye with 5 different pairs of glasses thus shows that over a large sample data, the system is capable of measuring diopter in the human eye. With additional training data, the Al machine learning algorithm will further increase the accuracy of the prediction model.Design of Model Eye

[0104] The initial dataset can also be from an eye model (based on an OEMI-7 model eye from Ocular Instruments) with a movable retina, an artificial lens, and an artificial cornea made from plastic, such as acrylic or acrylic polymers. The model eye has properties similar to a human eye. with an ability to change axial length to mimic human eyes. The model eye mimics a human eye but can be changed to simulate different level of refractive error in diopter and / or axial length, thereby providing a large dataset that can be used for training the artificial intelligence model to determine diopter and / or axial length from a human eye image.

[0105] The OEMI-7 model eye consists of a cornea and a lens, both made from acrylic. Acrylic has a refractive index of 1.49 (at 632.8 nm) which is significantly higher than a human eye. Various models have been created to model the refractive index of the human eye; one such model is the Gullstrand model eye. It is understood that the human eye has different refractive indexes. The crystalline lens is given 2 refractive indexes, as it is understood that the refractive index increases towards the center of the lens. These changes in refractive indexes between the OEMI-7 and human eye result in different focal lengths and hence different axial lengths for the emmetropic eye, with the human eye on average being 23.5 mm and the OEMI-7 being 25.8 mm.

[0106] To determine the refractive error of the system, the axial length must be converted. There is no standard method for converting between the two, this required a mathematical model to be developed which showed the relationship between axial length and refractive error. The mathematical model simulated the OEMI-7 model eye and the change inrefractive index with wavelength, with the refractive error on the eye being induced by simulating a lens in front of the model eye in the same method as normal ocular corrections. The refractive error was altered by changing the power of the lens, which would change the focal length. This focal length can be used to determine the axial length (with the axial length resulting in the induced refractive error). The following relationship was determined.

[0107] Equation

[0108] Where:

[0109] D is the refractive error in diopters

[0110] x is the axial length in mm

[0111] Xois the axial length of the emmetropic model eye in mm (25.8 mm)

[0112] Therefore, every71 mm is equivalent to 2.6392 diopters. It should be noted that due to differences between the OEMI-7 model eye and human eye. that this only applies to the model eye and a different relationship would need to be determined for the human eye.

[0113] FIG. 13 illustrates the model eye and the camera with an optical assembly used in the experiments described below. The model eye consists of a fluid chamber (distilled water) with a cornea, anterior chamber and crystalline lens assembly at the front and a moveable retina at the rear, the chamber has an overfill chamber which ensures that the chamber within the optical path is always completely submerged without pressurizing the chamber. The anterior assembly is taken from the OEMI-7 model (Ocular Instruments) with the lens and cornea both manufactured from acrylic (PMMA). The moveable retina was manufactured by casting polyurethane from a silicone mold of a Computer Numerical Control (CNC) part. Polyurethane has flesh color to mimic human retina. For movement the model eye utilized a linear stage motor (Thorlabs MTS-M-Z8) with an incremental repeatable movement of 0.8 pm. The motor has a controller with USB interface, allowing for the motor to be controlled from a personal computer (PC) and is closed loop.

[0114] The motor interacts with the PC via Thorlabs APT software. The system was controlled by a Matlab program which connected to the APT software via Microsoft’s ActiveX. The software allows for the axial length to be set directly or to be specified as a refractive error in diopters, this is converted to an axial length via Equation 1, as above.

[0115] To calibrate the model eye chamber was loosened allowing the chamber to move, the motor was homed and set to 0 mm position. Vernier calipers are used to determine the overall length of the moveable retina assembly. A slip gauge was then used to set the position of the moveable retina and the model eye casing. This allows for determination of the exact position of the retina in relation the comeal apex.Model Eye Verification Experiment

[0116] A model eye as described above was developed to facilitate rapid testing of ophthalmic instruments by changing the refractive error and axial length of the model eye. To verify the model eye optical performance, the retina was moved to different locations to simulate a change in axial length of the eye using the linear stage motor controlled by a software. Axial length at each location is then measured by Zeiss IOL Master 300. The retina on the model eye is moved along the axial length to simulate axial length of 28.07 mm to 23.52 mm at 0.19 mm intervals, which is equivalent to -6 to +6 D at 0.5 D intervals, with the IOL Master 300 taking 5 measurements at each location. In particular, the model eye was moved to the positions, a singular measurement from the IOL Master 300 was recorded, the model eye was then moved to next position and repeated until all positions were recorded. This sequence was repeated fuve (5) times, with an average of the results of all positions being determined. At each point, the axial length set by the software and the axial length measured by Zeiss IOL Master 300 are collected. The data was subject to a paired T-test Model Eye / IOL Master 300 and a Bland- Altman.

[0117] Data analysis was conducted in GraphPad Prism 10 and Microsoft Excel. The data was tested for normality with the Kolmogorov-Smirnov test. Correlation was tested with R- squared, a paired t-test was used to verify that there was a significant difference in the means, and a Bland-Altmann was used to verify agreement.

[0118] Results

[0119] Normality': A one sample Kolmogorov-Smirnov test was performed, with a D value of 24 and a P-value greater than 0.05, meaning the null hypothesis was not rejected and the data is normally distributed.

[0120] Correlation: The coefficient of determination (R squared) at a D value of 24 was calculated at 0.9998. Therefore, the data is highly correlated and is shown by FIG. 14.

[0121] T-test: A paired T-test was performed on the data. Model Eye (M=25.8, SD =1.39) IOL Master 300 (M=25.98, SD=1.41) which had a t(24)=31.85, p=<0.0001, showing there is a significant difference, as such the null hypothesis can be rejected. The t-test shows there is a significant difference in the mean values between the data, therefore the data is not random and both methods are statistically measuring the same variable at a confidence level of 95%.

[0122] Agreement and bias: Agreement was verified by a Bland-Altmann. The Bland- Altmann shows the low axial lengths are being over-estimated and high axial lengths are being under-estimated. Most of the data is within ±1.96 std which gives a confidence level of 95%, thus showing good agreement. Bland-Altman plot is provided in FIG. 15.

[0123] Discussion

[0124] The results show that there is a good correlation and agreement between the axial length measurements from the IOL Master 300 and the known axial lengths produced by the model eyes. Thus, the model eye is performing similarly to a real eye and is valid for accessing the performance of ophthalmic instruments that measure axial length. The model exhibited some bias at low and high axial lengths, over-estimating and under-estimating respectively. However, it is predictable, rather than random variations, thus can be calibrated for. The use of the model eye as a datum and source of data for developing axial length measurement devices will allow performance to be tracked back to industry standard measurement devices.Model Eye Auto-Refractor Verification Experiment

[0125] The same model eye as above was used for rapid testing of ophthalmic instruments by changing the refractive error and axial length of the model eye to act as an analogue for a human eye. The model also has the facility for optics to be placed in front of the anterior eye assembly.

[0126] The refractive error of the eye is induced by changing the axial length of the eye. However, to determine the refractive error of the system, the axial length must be converted. There is no standard method for converting between the two, this required a mathematical model to be developed which showed the relationship between axial length and refractive error. Since this model uses OEMI-7 model eye anterior, the relationship between refractive error and axial length is determined by Equation 1, as above.

[0127] To verify the physical model eye, it was tested with a Shin Nippon K900 autorefractor. The Shin Nippon K900 autorefractor relies on the measurement of a reflected infrared ring (expanding cone of light) based on the Scheiner principle. The size of the reflected ring can be directly related to the refracted error present in the system and the eccentricity of the ring related to astigmatism.

[0128] Testing was conducted in the range of -15D to +8 D at 1.0 D intervals. The model eye was moved to the positions, a singular measurement from the Shin Nippon K900 was recorded, the model eye was then moved to next position and repeated until all positions were recorded. This sequence was repeated 3 times, with an average of the results of all positions being determined.

[0129] Data analysis was conducted in GraphPad Prism 10 and Microsoft Excel. The data was tested for normality with the Kolmogorov-Smirnov test. Correlation was tested with R- squared, a paired t-test was used to verify’ that there was a significant difference in the means and a Bland-Altmann was used to verify agreement.

[0130] Results

[0131] Normality: A one sample Kolmogorov-Smirnov test was performed, with a D value of 24 and a P-value greater than 0.05, meaning the null hypothesis was not rejected and the data is normally distributed.

[0132] Correlation: The coefficient of determination (R squared) at a D value of 24 was calculated at 0.9985. Therefore, the data is highly correlated and is shown by FIG. 16.

[0133] T-test: The paired T test result for myopia (negative values, <0.0D) data was Model Eye (M=-7.5, SD =4.76) K900 (M=-7.83, SD=5.02) which had at(15)=4.669, p=<0.0001. The paired T test result for hyperopia (positive values, >0.0D) data was Model Eye (M=4, SD =2.74) K900 (M=4.57, SD=3.37) which had a t(8)=2.574, p=<0.0001. There is a significant difference both the hyperopia and myopia data, as such the null hypothesis can be rejected. The t-test shows there is a significant difference in the mean values between the data, therefore the data is not random and both methods are statistically measuring the same variable at a confidence level of 95%. The t-test indicates a significant difference in the mean values between the data, suggesting that the data is not random.

[0134] Agreement and Bias: Agreement was verified by a Bland-Altmann. The Bland- Altmann shows negative powers are being over-estimated and higher lengths are being underestimated. The data is within ±1.96 std which gives a confidence level of 95%, thus showing good agreement. The Bland- Altman plot is shown in FIG. 17.

[0135] Discussion

[0136] The results show that there is a good correlation and agreement between the refractive error measured by the K900 Autorefractor and the known refractive errors produced by the model eyes. Thus, the model eye is performing similarly to a real eye and is valid for accessing the performance of ophthalmic instruments that measure refractive error.

[0137] The model exhibited some bias at low and high refractive errors, over-estimating and under-estimating respectively. However, it is predictable, rather than random variations, thus can be calibrated for. The use of the model eye as a datum and sources of data for developing refractive error measurement devices will allow performance to be tracked back to industry standard measurement devices and techniques.Model Eye Over-refraction Verification Experiment

[0138] The same model eye as above was used for determination of over-refraction induced refractive error from color imaging. The refractive error in the system was induced by manually changing prescriptions glasses in front of the model eye. FIG. 18 depicts the setup of the model eye. the prescription glasses, and the camera with an optical assembly mechanical attachment. The image capturing was software controlled from a computer. Testing was conducted at a prescription range of -3 to ±3D at 0.5D intervals, with 15 images captured for each prescription to form a set. There was a total of 15 sets, with a dataset size of 3900.

[0139] The capturing device was an iPhone 14 pro, using the telephoto camera with an optical assembly mechanical attachment. The phone flashlight was used to increase the brightness of the pupil and generate a bright pupil response. The optical assembly mechanical attachment (detailed in FIGS. 1 - 8) allowed for the telephoto camera to focus on 120mm, providing a high number of pixels covering the pupil of the model eye amount 360,000. It also ensured the torch light was co-axial with the camera, this was required due to the close distance to the object being captured to ensure a fully lit pupil. The attachment consists of 10D (f=100mm) condensing lens, a cold front surface mirror, a 50 / 50 beam splitter, 2 polarizers (foriPhone 14 pro telephoto camera and iPhone 14 pro flashlight) and an aperture for the flashlight.The attachment is mounted to an iPhone 14 pro case through mechanical fastenings.

[0140] The refraction of the system was adjusted manually through changing the prescription of the glasses in front of the eye. Testing was conducted in the range of -3 to +3 D at 0.5D intervals. The capture process consisted of the flash being turned on and the camera capturing 3 images, the flash was then turned off. This was repeated 5 times for a total of 15 images, with the prescription glasses being changed after. This was completed for each pair of glasses until all have been captured, which was a complete set of data. A total of twenty (20) sets were performed and a total image number of 3900.

[0141] The captured images went through a processing pipeline consisting of: locating the pupil; cropping the image to the pupil size; masking the pupil so all pixels outside the pupil were set to 0; separating RGB channels from the raw image (cameras have a Bayer pattern, where each pixel in an RGB camera only measures red. green or blue light) and creating a new image for each channel; extracting features from the RGB channels.

[0142] The extracted features consisted of standard statistical properties such as mean, median, geomean and standard deviation. Further features w ere generated by comparing these properties betw een channels, such as ratio of red and blue mean and the difference between red and blue mean. This process was repeated for all the images captured and stored in a table.

[0143] Initial attempts at modelling the data were performed with Polynomial Regression Models, whereby a single feature was selected from all the data e.g. ratio of red to blue mean, this w as compared against the recorded axial length, a Polynomial Regression Model was then generated to describe the data. However, this failed to provide sufficient accuracy, as such Support Vector Machines (SVM) were used. SVMs are supervised classification and regression models were designed for binary classifications but could be expanded for multi-class and regression applications. SVM is a Kemelized algorithm, which clusters a features space based on named labels with maximum distance between the clusters’ borders.

[0144] The functions that were implemented and evaluated within this study are Linear, 2ndOrder Polynomial (Quadratic), 3rdOrder Polynomial (Cubic) and Gaussian Kernel functions. Twenty-four arbitrary Kernel scales were selected and evaluated in the cross-validation process (0.005, 0.01, 0.015, 0.02, 0.025, 0.03, 0.035, 0.04, 0.045, 0.05, 0.1, 0.15, 0.2, 0.3, 0.4, 1, 1.4, 2, 3, 4, 5, 5.7).

[0145] A total of 45 features were extracted from the raw images, to speed up processing time it was important to minimize the number of features. The features were selected using minimal-redundancy-maximal-relevance (MRMR) technique. This technique finds the most optimum subset of features that characterize the recorded axial length. Using MRMR Shannon’s Entropy, features were identified that are minimally redundant (mutually exclusive) but highly relevant to the classification clusters (mutually inclusive, specifically axial length clusters). These features were then utilized in the SVM.

[0146] In this study, the accuracy of the Support Vector Machine (SVM) was evaluated using both cross-validation accuracy and the Fl score. The cross-validation accuracy ensured that the model's performance w as consistent across different subsets of the data, while the Fl score provided a nuanced measure of the model's precision and recall balance. By employ ing these metrics, the study ensured a comprehensive evaluation of the model's classification capabilities.

[0147] The Fl score is a crucial metric for evaluating classification models, particularly in cases where the dataset is imbalanced. It is the harmonic mean of precision and recall, providing a balanced measure that considers both false positives and false negatives. Precision represents the proportion of true positive predictions out of all positive predictions made by the model, while recall measures the proportion of true positive predictions out of all actual positives in the dataset. The Fl score is beneficial because it balances these two aspects, offering a single metric that ensures both high precision and high recall.

[0148] Cross-validation accuracy is used to assess the performance of a model in a more robust manner. This technique involves splitting the dataset into 'k' folds, training the model on 'k-1 ' folds, and testing it on the remaining fold. This process is repeated 'k' times, with each fold serving as the test set once. The average accuracy from these iterations is the cross-validation accuracy, providing a reliable estimate of the model's generalizability' and performance on unseen data.

[0149] Results

[0150] Number of Features: The cross validation and Fl score of the SVM model improved with the number of features as shown by FIG. 19, w ith it levelling off after 5 features.

[0151] SVM Function Comparison: The cross validation and Fl score of the SVM model increased with higher order functions as shown by FIG. 20, although there was less than 2%between the linear, quadratic and cubic function. The best performance with Gaussian fdter was best achieved at a Kernel scale of 3 with 92% performance for both Fl Score and cross validation average, as shown in FIG. 21. However, the linear, quadratic and cubic functions both had better performance at over 98% performance for both Fl Score and cross validation average.

[0152] Discussion

[0153] The SVM model developed was able to achieve a performance of near 100% for both Fl Score and cross validation accuracy. This was achieved for both the quadratic and cubic functions, with the cubic function having marginally better performance. This demonstrates that a machine learning model can predict effectively refractive error induced through incorrect prescription glasses.Model Eye Axial Length Verification Experiment

[0154] The same model eye as above was used for determining the axial length of the model eye from color imaging. This was repeated 7 times, with a total dataset size of 3840. A number Support Vector Machine (SVM) model was implemented with different functions. The model with the best performance was the cubic function, with cross validation and Fl score of 99.38% and 100% respectively. This shows that the model is highly accurate at determining the correct axial length from an image of the model eye.

[0155] The capturing device was an iPhone 14 pro, using the telephoto camera with a mechanical attachment. The phone flashlight was used to increase the brightness of the pupil and generate a bright pupil response.

[0156] The optical assembly attachment (shown in FIGS. 1-8) allowed for the telephoto camera to focus on 120mm, providing a high number of pixels covering the pupil of the model eye amount 360,000. It also ensured the torch light was co-axial with the camera, this was required due to the close distance to the object being captured to ensure a fully lit pupil. The optical assembly mechanical attachment consists of 10D (f=100mm) condensing lens, a cold front surface mirror, a 50 / 50 beam splitter, 2 polarizers (for iPhone 14 pro telephoto camera and iPhone 14 pro flashlight) and an aperture for the flashlight. The optical assembly mechanical attachment is mounted to an iPhone 14 pro case through mechanical fastenings.

[0157] The iPhone 14 pro is mounted on a base in front of the model eye, this ensured that the iPhone 14 pro was stable and remained in the location throughout the testing. This setup is show n in FIG. 13. The iPhone 14 pro was controlled from a computer, which communicated to an app installed on the iPhone through Wi-Fi. This allow ed for the camera settings to be set and trigger the capture. It was not possible to use the in-built flash functionality of the iPhone, due to it triggering auto-focus and auto-exposure of the camera, it is not possible to disable this. Therefore, to use the flashlight, it had to be manually turned on and off, this requires precise timing as the flashlight needs to be fully lit before the image has begun capturing. The process typically involved turning on the flashlight, capturing multiple images (3 images as standard) and then turning off the flashlight. The captured images were then sent to the computer over WiFi.

[0158] Analysis

[0159] The system simulated and captured images in an axial length range of 24.663 mm to 26.937 mm at 0.038 mm intervals. Although the model eye can do greater ranges, this was chosen to reduce data size. The data was recorded in sets, with each set consisting of 3 images at each axial length location. There was a random pause between each axial length position of between 10 to 60 seconds. This was due to the phone increasing in temperature w ith small pause times, which would affect the intensity of the pixels in captured images. Therefore, by having a random time there is a diverse dataset that can be fed into the model. Once a set was complete, a new set would be commenced, with there being a total of 7 sets which resulted in a total data size of 3840 images.

[0160] The captured images went through a processing pipeline which consisted of: locating the pupil; cropping the image to the pupil size; masking the pupil so all pixels outside the pupil were set to 0; separating RGB channels from the raw image (cameras have a Bayer pattern, where each pixel in an RGB camera only measures red, green or blue light), and create new image for each channel; extracting features from the RGB channels.

[0161] The extracted features consisted of standard statistical properties such as mean, median, geomean and standard deviation. Further features were generated by comparing these properties between channels, such as ratio of red and blue mean and the difference between red and blue mean. This process was repeated for all the images captured and stored in a table.

[0162] Initial attempts at modelling the data were performed with PolynomialRegression Models, w ereby a single feature was selected from all the data e.g. ratio of red toblue mean, this was compared against the recorded axial length, a Polynomial Regression Model was then generated to describe the data.

[0163] However, this failed to provide sufficient accuracy, as such Support Vector Machines (SVM) were used. SVMs are supervised classification and regression models were designed for binary classifications but could be expanded for multi-class and regression applications. SVM is a Kemelized algorithm, which clusters a features space based on named labels with maximum distance between the clusters' borders.

[0164] The functions that were implemented and evaluated within this study are Linear, 2ndOrder Polynomial (Quadratic). 3rdOrder Polynomial (Cubic) and Gaussian Kernel functions. Twenty -four arbitrary Kernel scales were selected and evaluated in the cross-validation process (0.005, 0.01, 0.015, 0.02, 0.025, 0.03, 0.035, 0.04, 0.045, 0.05, 0.1, 0.15, 0.2, 0.3, 0.4, 1, 1.4, 2, 3, 4, 5, 5.7).

[0165] A total of 45 features were extracted from the raw images, to speed up processing time it was important to minimize the number of features. The features were selected using minimal-redundancy-maximal-relevance (MRMR) technique. This technique finds the most optimum subset of features that characterize the recorded axial length. Using MRMR Shannon’s Entropy, features were identified that are minimally redundant (mutually exclusive) but highly relevant to the classification clusters (mutually inclusive, specifically axial length clusters). These features were then utilized in the SVM.

[0166] In this study, the accuracy of the Support Vector Machine (SVM) was evaluated using both cross-validation accuracy and the Fl score. The cross-validation accuracy ensured that the model's performance was consistent across different subsets of the data, while the Fl score provided a nuanced measure of the model's precision and recall balance. By employing these metrics, the study ensured a comprehensive evaluation of the model's classification capabilities.

[0167] The Fl score is a crucial metric for evaluating classification models, particularly in cases where the dataset is imbalanced. It is the harmonic mean of precision and recall, providing a balanced measure that considers both false positives and false negatives. Precision represents the proportion of true positive predictions out of all positive predictions made by the model, while recall measures the proportion of true positive predictions out of all actual positivesin the dataset. The Fl score is beneficial because it balances these two aspects, offering a single metric that ensures both high precision and high recall.

[0168] Cross-validation accuracy is used to assess the performance of a model in a more robust manner. This technique involves splitting the dataset into 'k' folds, training the model on 'k-1 ' folds, and testing it on the remaining fold. This process is repeated 'k' times, with each fold serving as the test set once. The average accuracy from these iterations is the cross-validation accuracy, providing a reliable estimate of the model's generalizability and performance on unseen data.

[0169] Results

[0170] Number of Features: The cross validation and Fl score of the SVM model improved with the number of features as shown by FIG. 22, with it levelling off after 10 features.

[0171] SVM Function Comparison: The cross validation and Fl score of the SVM model increased with higher order functions as shown by FIG. 23, although the difference between quadratic and cubic was less than 0.5%. The performance of the Gaussian function improved with Kernel scale up till 3 with it falling after (FIG. 24).

[0172] Discussion

[0173] Initial attempts at creating a model using Polynomial Regression Models was unsuccessful. The Polynomial Regression Models were generated based a singular feature, with no feature producing a model with sufficient accuracy. This is due to the features having a relatively small change over the entire data set, for example the red mean has a variable of 389, which is only 9.5% of the range for a 12bit number (iPhone Camera is 12bit RGB). Therefore, any small changes in features can result in incorrect classification. However, what w as noted is that multiple features would change at the same time, which implies they could be linked. As such, SVM was investigated as it allowed multiple features to be considered as part of the model.

[0174] The SVM model that performed the best was the cubic function which has a cross validation and Fl score of 99.38% and 100% respectively. This shows that the model is highly accurate at determining the correct axial length.Assessments from the Model Eye Experiments

[0175] The Experiments above show the model eye developed for data collection for use in training the Al model is effective. The model eye can be used to simulate a human eye, from which auto-refraction, over refraction, and axial length can be measured by photo capturing and color imaging. Images and data collected from photo capturing this model eye accurately reflect the same data from human eyes, and thus can be used to train the Al model described herein.

[0176] In a first embodiment, a system for assessing myopia in human eyes is provided, the system comprises: a smartphone with a camera capable of taking pictures of eyes and saving pictures as digital eye images; a computer programming product configured to separate the digital eye images into red, green, and blue channels, calculate statistical features of the images from the red, green, and blue channels, and calculate a refractive error in diopter and / or axial length of an eye using the statistical features; and a user interface capable of displaying the refractive error in diopter and / or axial length of the eye.

[0177] In a second embodiment, the first embodiment includes the smartphone further comprises a flashlight configured to provide light in conjunction with picture taking by the camera in the smartphone.

[0178] In a third embodiment, each of the first to second embodiment includes the computer programming product is further configured to assess the quality of the pictures captured by the camera.

[0179] In a fourth embodiment, the third embodiment includes the quality of the picture is assessed by resolution, lighting, focus, or blurriness.

[0180] In a fifth embodiment, each of the first to second embodiment includes the computer programming product is further configured to locate the pupil image data within the image data.

[0181] In a sixth embodiment, the fifth embodiment includes locating the pupil image comprises searching for an object with a circular or substantially circular boundary.

[0182] In a seventh embodiment, each of the fifth and sixth embodiment includes the computer programming product is further configured to extract the pupil image data from the image data.

[0183] In an eighth embodiment, the seventh embodiment includes the computer programming product is further configured to compress the pupil image data and upload the pupil image data to a cloud server.

[0184] In the ninth embodiment, each of the first and eighth embodiment includes the calculation of statistical features of the image from the red. green, and blue channels is conducted by an artificial intelligence program with machine learning capabilities.

[0185] In the tenth embodiment, each of the first, second, fourth, sixth, and eighth embodiment includes wherein the statistical features of the image are at least one of red mean, red median, red standard deviation, red geomean, red brightness weighted average, green mean, green median, green standard deviation, green geomean, green brightness weighted average; blue mean, blue median, blue standard deviation, blue geomean, blue brightness weighted average, blue mean - red mean, blue median - red median, blue standard deviation - red standard deviation, blue geomean - red geomean, blue brightness weighted average - red brightness weighted average; green mean - red mean, green median - red median, green standard deviation - red standard deviation, green geomean - red geomean, green brightness weighted average - red brightness weighted average; green mean - blue mean, green median - blue median, green standard deviation - blue standard deviation, green geomean - blue geomean, green brightness weighted average - blue brightness weighted average; blue mean / red mean, blue median / red median, blue standard deviation / red standard deviation, blue geomean / red geomean, blue brightness weighted average / red brightness weighted average; green mean / red mean, green median / red median, green standard deviation / red standard deviation, green geomean / red geomean, and green brightness weighted average / red brightness weighted average, wherein red mean refers to the mean intensity of all red pixels in the image, blue mean refers to the mean intensity of all blue pixels in the image, green mean refers to the mean intensity of all green pixels in the image, wherein red geomean refers to the geometric mean intensity of all red pixels, green geomean refers to the geometric mean intensity of all green pixels, blue geomean refers to the geometric mean intensity of all blue pixels,wherein red median refers to the median intensity of all red pixels in the image, green median refers to the median intensity of all green pixels in the image, blue median refers to the median intensity of all blue pixels in the image, wherein red brightness average refers to the average of brightness of all red pixels in the image, green brightness average refers to the average of brightness of all green pixels in the image, blue brightness average refers to the average of brightness of all blue pixels in the image, wherein red standard deviation refers to the amount of variations of red pixels in the image, blue standard deviation refers to the amount of variations of blue pixels in the image, green standard deviation refers to the amount of variations of green pixels in the image, wherein blue mean - red mean refers the mean intensity of all blue pixels in the image minus by mean intensity of all red pixels in the image, green mean - red mean refers the mean intensity of all green pixels in the image minus by mean intensity of all red pixels in the image, green mean - blue mean refers the mean intensity of all green pixels in the image minus by mean intensity of all blue pixels in the image, wherein blue median - red median refers to the median intensity of all blue pixels in the image minus by median intensity of all red pixels in the image, green median - blue median refers to the median intensity of all green pixels in the image minus by median intensity of all blue pixels in the image, green median - red median refers to the median intensity of all green pixels in the image minus by median intensity of all red pixels in the image, wherein blue standard deviation - red standard deviation refers to the amount of variations of blue pixels in the image minus the amount of variations of red pixels in the image, green standard deviation - blue standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of blue pixels in the image, green standard deviation - red standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of red pixels in the image, wherein blue geomean - red geomean refers to the geometric mean intensity of all blue pixels in the image minus by geometric mean intensity of all red pixels in the image, green geomean - blue geomean refers to the geometric mean intensity of all green pixels in the image minus by geometric mean intensity of all blue pixels in the image, green geomean - red geomean refers to the geometric mean intensity of all green pixels in the image minus by geometric mean intensity of all red pixels in the image,wherein blue brightness weighted average - red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average - blue brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of blue pixel brightness values that exceed the median brightness level, green brightness weighted average - red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level, wherein blue mean / red mean refers the mean intensity' of all blue pixels in the image divided by mean intensity of all red pixels in the image, green mean / red mean refers the mean intensity of all green pixels in the image divided by mean intensity of all red pixels in the image, wherein blue median / red median refers to the median intensity' of all blue pixels in the image divided by the median intensity7of all red pixels in the image, green median / red median refers to the median intensity' of all green pixels in the image divided by the median intensity of all red pixels in the image, wherein blue standard deviation / red standard deviation refers to the amount of variations of blue pixels in the image divided by the amount of variations of red pixels in the image, green standard deviation / red standard deviation refers to the amount of variations of green pixels in the image divided by the amount of variations of red pixels in the image, wherein blue geomean / red geomean refers to the geometric mean intensity of all blue pixels in the image divided by geometric mean intensity of all red pixels in the image, green geomean / red geomean refers to the geometric mean intensity of all green pixels in the image divided by geometric mean intensity7of all red pixels in the image, wherein blue brightness yveighted average / red brightness yveighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average I red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level

[0186] In the eleventh embodiment, the eighth embodiment includes the computer programming product is further configured to use the artificial intelligence program to calculate refractive error in diopter and / or axial length of an eye from the statistical features.

[0187] In the twelfth embodiment, the eleventh embodiment includes the artificial intelligence program comprises two distinct machine learning models, and wherein one of the two distinct machine learning models calculates diopters of the eye using the eye image, and the other of one the two distinct machine learning models calculates axial length of the eye using the eye image.

[0188] In the thirteenth embodiment, the twelfth embodiment includes the machine learning models use a Support Vector Machine algorithm to select appropriate statistical features for classification and rank the statistical features based on their contribution to either diopter or axial length prediction using Minimum Redundancy Maximum Relevance.

[0189] In the fourteenth embodiment, each of the first, second, fourth, sixth, eighth, tenth, eleventh, twelfth, and thirteenth includes the user interface is displayed on the smartphone within the system.

[0190] In the fifteenth embodiment, each of the first, second, fourth, sixth, eighth, tenth, eleventh, twelfth, and thirteenth includes the user interface is displayed on a computing article other than the smartphone within the system.

[0191] In the sixteenth embodiment, each of the first, second, fourth, sixth, eighth, tenth, eleventh, twelfth, and thirteenth further comprises an optical assembly attachment, the optical assembly attachment comprises two polarizers, a mirror, a beam splitter, and a converging lens, wherein the optical assembly attachment is configured to attach to the back of a smartphone.

[0192] In the seventeenth embodiment, the sixteenth embodiment includes one of the two polarizers is configured to situate above a flashlight on the smartphone, the other of the two polarizers is configured to situate above the camera on the smartphone, the mirror is situated above and at an angle to the polarizer above the flashlight, the beam splitter is situated parallel to the mirror, and the converging lens is situated above and parallel to the polarizer above the camera.

[0193] In the eighteenth embodiment, the seventeenth embodiment includes the optical assembly attachment further comprises a light trap to diffuse light from the beam splitter.

[0194] In the nineteenth embodiment, each of the seventeenth embodiment and the eighteenth embodiment includes the optical assembly attachment further comprises a back plate to attach the optical assembly attachment to the smartphone.

[0195] In the twentieth embodiment, each of the first, second, fourth, sixth, eighth, tenth, eleventh, twelfth, thirteenth, seventeenth, and eighteenth embodiment includes the computer programming product is further configured to analyze the location of the camera relative to a human eye and give commands for maneuvering the camera to a predetermined location.

[0196] In the twenty first embodiment, the twentieth embodiment includes the commands are given in by voice or by visual display.

[0197] In the twenty second embodiment, a computer method to assess myopia of an eye is provided, the method comprises: taking a picture of an eye using a camera and saving the picture as digital image data; locating a pupil image within the digital image data; extracting the pupil image data from the picture of the eye; separating image data into red, green, and blue channels; calculating statistical features of the image from red, green, and blue channels; using a Support Vector Machine algorithm to select appropriate statistical features for classification and rank the statistical features based on their contnbution to either diopter or axial length prediction using Minimum Redundancy Maximum Relevance; and calculate diopter or axial length of the eye based on the statistical features ranking.

[0198] In the twenty third embodiment, the twenty second embodiment includes locating a pupil image within the digital image data comprises searching for an obj ect with a circular or substantially circular boundary.

[0199] In the twenty fourth embodiment, each of the twenty second and twenty third embodiment includes wherein the statistical features of the image are at least one of red mean, red median, red standard deviation, red geomean, red brightness weighted average, green mean, green median, green standard deviation, green geomean, green brightness weighted average; blue mean, blue median, blue standard deviation, blue geomean, blue brightness weightedaverage, blue mean - red mean, blue median - red median, blue standard deviation - red standard deviation, blue geomean - red geomean, blue brightness weighted average - red brightness weighted average; green mean - red mean, green median - red median, green standard deviation - red standard deviation, green geomean - red geomean, green brightness weighted average - red brightness weighted average; green mean - blue mean, green median - blue median, green standard deviation - blue standard deviation, green geomean - blue geomean, green brightness weighted average - blue brightness weighted average; blue mean / red mean, blue median / red median, blue standard deviation / red standard deviation, blue geomean / red geomean, blue brightness weighted average / red brightness weighted average; green mean / red mean, green median / red median, green standard deviation / red standard deviation, green geomean / red geomean, and green brightness weighted average / red brightness weighted average, wherein red mean refers to the mean intensity of all red pixels in the image, blue mean refers to the mean intensity of all blue pixels in the image, green mean refers to the mean intensity of all green pixels in the image, wherein red geomean refers to the geometric mean intensity of all red pixels, green geomean refers to the geometric mean intensity of all green pixels, blue geomean refers to the geometric mean intensity of all blue pixels, wherein red median refers to the median intensity’ of all red pixels in the image, green median refers to the median intensity of all green pixels in the image, blue median refers to the median intensity of all blue pixels in the image, wherein red brightness average refers to the average of brightness of all red pixels in the image, green brightness average refers to the average of brightness of all green pixels in the image, blue brightness average refers to the average of brightness of all blue pixels in the image, wherein red standard deviation refers to the amount of variations of red pixels in the image, blue standard deviation refers to the amount of variations of blue pixels in the image, green standard deviation refers to the amount of variations of green pixels in the image, wherein blue mean - red mean refers the mean intensity’ of all blue pixels in the image minus by mean intensity of all red pixels in the image, green mean - red mean refers the mean intensity of all green pixels in the image minus by mean intensity of all red pixels in the image, green mean - blue mean refers the mean intensity of all green pixels in the image minus by mean intensity’ of all blue pixels in the image,wherein blue median - red median refers to the median intensity of all blue pixels in the image minus by median intensity of all red pixels in the image, green median - blue median refers to the median intensity of all green pixels in the image minus by median intensity of all blue pixels in the image, green median - red median refers to the median intensity' of all green pixels in the image minus by median intensity of all red pixels in the image, wherein blue standard deviation - red standard deviation refers to the amount of variations of red pixels in the image minus the amount of variations of red pixels in the image, green standard deviation - blue standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of blue pixels in the image, green standard deviation - red standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of red pixels in the image, wherein blue geomean - red geomean refers to the geometric mean intensity of all blue pixels in the image minus by geometric mean intensity of all red pixels in the image, green geomean - blue geomean refers to the geometric mean intensity' of all green pixels in the image minus by geometric mean intensity' of all blue pixels in the image, green geomean - red geomean refers to the geometric mean intensity' of all green pixels in the image minus by geometric mean intensity of all red pixels in the image. wherein blue brightness weighted average - red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average - blue brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of blue pixel brightness values that exceed the median brightness level, green brightness weighted average - red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level, wherein blue mean / red mean refers the mean intensity of all blue pixels in the image divided by mean intensity of all red pixels in the image, green mean / red mean refers the mean intensity of all green pixels in the image divided by mean intensity of all red pixels in the image,wherein blue median / red median refers to the median intensity of all blue pixels in the image divided by the median intensity of all red pixels in the image, green median / red median refers to the median intensity of all green pixels in the image divided by the median intensity of all red pixels in the image, wherein blue standard deviation / red standard deviation refers to the amount of variations of blue pixels in the image divided by the amount of variations of red pixels in the image, green standard deviation / red standard deviation refers to the amount of variations of green pixels in the image divided by the amount of variations of red pixels in the image, wherein blue geomean / red geomean refers to the geometric mean intensity of all blue pixels in the image divided by geometric mean intensity of all red pixels in the image, green geomean / red geomean refers to the geometric mean intensity of all green pixels in the image divided by geometric mean intensity of all red pixels in the image, wherein blue brightness weighted average / red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average / red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level.

[0200] In the twenty fifth embodiment, each of the twenty second and twenty third embodiment includes the calculating of diopter and axial length is conducted by two distinct machine learning models.

[0201] In the twenty sixth embodiment, each of the twenty second and twenty third embodiment includes the camera is present on a smartphone and the method is conducted by a computer programming product installed on the same smartphone.

[0202] In the twenty seventh embodiment, the twenty second embodiment further comprises assessing the quality of the picture captured by the camera.

[0203] In the twenty eighth embodiment, the twenty seventh embodiment includes the quality of the picture is assessed by resolution, lighting, focus, or blurriness.

[0204] In the tw enty ninth embodiments, each of the twenty seventh and twenty eighth embodiment further comprises compressing the pupil image data and upload the pupil image data to a cloud server, wherein the cloud server hosts the machine learning models.

[0205] In the thirtieth embodiments, each of the twenty second, twenty third, twenty seventh, twenty eighth, and thirtieth embodiment further comprises displaying the diopter or axial length calculation result on a user interface.

[0206] In the thirty7first embodiment, each of the twenty second, tw enty third, tw enty seventh, twenty eighth, and thirtieth embodiment further comprises sending the diopter or axial length calculation result to a computing article.

[0207] In the thirty second embodiment, the twenty second embodiment further comprises attaching an optical assembly to the smartphone, the optical assembly comprises tw o polarizers, a mirror, a beam splitter, and a converging lens, wherein the optical assembly attachment is configured to attach to the back of a smartphone.

[0208] In the thirty third embodiment, the thirty second embodiment includes one of the two polarizers is situated above a flashlight on the smartphone, the mirror is situated above and at an angle to the polarizer above the flashlight, the beam splitter is situated parallel to the mirror, the converging lens is situated above the camera between the camera and the beam splitter, and the other of the two polarizers is situated above and parallel to the converging lens.

[0209] In the thirty fourth embodiment, the thirty first embodiment includes the optical assembly attachment further comprises a light trap to diffuse light from the beam splitter.

[0210] In the thirty fifth embodiment, each of the thirty second and thirty fourth embodiment includes the optical assembly attachment further comprises a back plate to attach the optical assembly attachment to the smartphone.

[0211] In the thirty sixth embodiment, each of the thirty second and thirty fourth embodiment further comprises analyzing the location of the camera relative to an eye and giving commands for maneuvering the camera to a predetermined location.

[0212] According to each of the embodiments described and contemplated herein, this refractive state determination apparatus may further comprise an eye model for training a machine to determine refractive state. An exemplary7system for training a machine to determine refractive state is described in concurrently filed Patent Cooperation Treat Application entitled“EYE MODEL TRAINING A MACHINE TO DETERMINE REFRACTIVE STATE,” and assigned Atorney Docket No OCU2PCT, which is incorporated herein by reference in its entirety.

[0213] Other features and advantages of the invention will be apparent from the following detailed description, and from the claims.

[0214] The above examples are included for illustrative purposes only and are not intended to limit the scope of the invention. Many variations to those described above are possible. Since modifications and variations to the examples described above will be apparent to those of skill in this art. it is intended that this invention be limited only by the scope of the appended claims.

Claims

CLAIMSWe claim:

1. A system for assessing myopia in human eyes, the system comprises: a smartphone with a camera capable of taking pictures of eyes and saving pictures as digital eye images; a computer programming product operable in connection with the smartphone and configured to separate the digital eye images into red, green, and blue channels, calculate statistical features of the images from the red, green, and blue channels, and calculate a refractive error in diopter and / or axial length of an eye using the statistical features; and a user interface capable of displaying the refractive error in diopter and / or axial length of the eye.

2. The system of claim 1, wherein the smartphone further comprises a flashlight configured to provide light in conjunction with picture taking by the camera in the smartphone.

3. The system of claim 1 or 2, wherein the computer programming product is further configured to assess the quality of the pictures captured by the camera.

4. The system of claim 3, wherein the quality of the picture is assessed by resolution, lighting, focus, or blurriness.

5. The system of claim 1 or 2, wherein the computer programming product is further configured to locate the pupil image data within the image data.

6. The system of claim 5, wherein locating the pupil image comprises searching for an object with a circular or substantially circular boundary.

7. The system of claim 6, wherein the computer programming product is further configured to extract the pupil image data from the image data.

8. The system of claim 7, wherein the computer programming product is further configured to compress the pupil image data and upload the pupil image data to a cloud server.

9. The system of claim 1 or 8, wherein the calculation of statistical features of the image from the red, green, and blue channels is conducted by an artificial intelligence program with machine learning capabilities.

10. The system of claim 1, wherein the statistical features of the image are at least one of red mean, red median, red standard deviation, red geomean, red brightness weighted average, green mean, green median, green standard deviation, green geomean, green brightness weighted average; blue mean, blue median, blue standard deviation, blue geomean, blue brightness weighted average, blue mean - red mean, blue median - red median, blue standard deviation - red standard deviation, blue geomean - red geomean, blue brightness weighted average - red brightness weighted average; green mean - red mean, green median - red median, green standard deviation - red standard deviation, green geomean - red geomean, green brightness weighted average - red brightness weighted average; green mean - blue mean, green median - blue median, green standard deviation - blue standard deviation, green geomean - blue geomean, green brightness weighted average - blue brightness weighted average; blue mean / red mean, blue median / red median, blue standard deviation / red standard deviation, blue geomean / red geomean, blue brightness weighted average / red brightness weighted average; green mean / red mean, green median / red median, green standard deviation / red standard deviation, green geomean I red geomean, and green brightness weighted average I red brightness weighted average, wherein red mean refers to the mean intensity of all red pixels in the image, blue mean refers to the mean intensity of all blue pixels in the image, green mean refers to the mean intensity of all green pixels in the image, wherein red geomean refers to the geometric mean intensity7of all red pixels, green geomean refers to the geometric mean intensity of all green pixels, blue geomean refers to the geometric mean intensity of all blue pixels, wherein red median refers to the median intensity of all red pixels in the image, green median refers to the median intensity of all green pixels in the image, blue median refers to the median intensity of all blue pixels in the image,wherein red brightness average refers to the average of brightness of all red pixels in the image, green brightness average refers to the average of brightness of all green pixels in the image, blue brightness average refers to the average of brightness of all blue pixels in the image, wherein red standard deviation refers to the amount of variations of red pixels in the image, blue standard deviation refers to the amount of variations of blue pixels in the image, green standard deviation refers to the amount of variations of green pixels in the image, wherein blue mean - red mean refers the mean intensity of all blue pixels in the image minus by mean intensity of all red pixels in the image, green mean - red mean refers the mean intensity of all green pixels in the image minus by mean intensity of all red pixels in the image, green mean - blue mean refers the mean intensity of all green pixels in the image minus by mean intensity of all blue pixels in the image, wherein blue median - red median refers to the median intensity of all blue pixels in the image minus by median intensity of all red pixels in the image, green median - blue median refers to the median intensity of all green pixels in the image minus by median intensity of all blue pixels in the image, green median - red median refers to the median intensity' of all green pixels in the image minus by median intensity of all red pixels in the image, wherein blue standard deviation - red standard deviation refers to the amount of variations of blue pixels in the image minus the amount of variations of red pixels in the image, green standard deviation - blue standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of blue pixels in the image, green standard deviation - red standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of red pixels in the image, wherein blue geomean - red geomean refers to the geometric mean intensity of all blue pixels in the image minus by geometric mean intensity of all red pixels in the image, green geomean - blue geomean refers to the geometric mean intensity' of all green pixels in the image minus by geometric mean intensity' of all blue pixels in the image, green geomean - red geomean refers to the geometric mean intensity of all green pixels in the image minus by geometric mean intensity of all red pixels in the image. wherein blue brightness weighted average - red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the medianbrightness level, green brightness weighted average - blue brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of blue pixel brightness values that exceed the median brightness level, green brightness weighted average - red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level, wherein blue mean / red mean refers the mean intensity of all blue pixels in the image divided by mean intensity of all red pixels in the image, green mean / red mean refers the mean intensity of all green pixels in the image divided by mean intensity of all red pixels in the image, wherein blue median / red median refers to the median intensity of all blue pixels in the image divided by the median intensity of all red pixels in the image, green median / red median refers to the median intensity of all green pixels in the image divided by the median intensity of all red pixels in the image, wherein blue standard deviation / red standard deviation refers to the amount of variations of blue pixels in the image divided by the amount of variations of red pixels in the image, green standard deviation / red standard deviation refers to the amount of variations of green pixels in the image divided by the amount of variations of red pixels in the image, wherein blue geomean / red geomean refers to the geometric mean intensity of all blue pixels in the image divided by geometric mean intensity of all red pixels in the image, green geomean / red geomean refers to the geometric mean intensity of all green pixels in the image divided by geometric mean intensity of all red pixels in the image, wherein blue brightness weighted average / red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average / red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level.

11. The system of claim 9, wherein the computer programming product is further configured to use the artificial intelligence program to calculate refractive error in diopter and / or axial length of an eye from the statistical features.

12. The system of claim 11 , wherein the artificial intelligence program comprises two distinct machine learning models, and wherein one of the two distinct machine learning models calculates diopters of the eye using the eye image, and the other of one the two distinct machine learning models calculates axial length of the eye using the eye image.

13. The system of claim 12, wherein the machine learning models use a Support Vector Machine algorithm to select appropriate statistical features for classification and rank the statistical features based on their contribution to either diopter or axial length prediction using Minimum Redundancy Maximum Relevance.

14. The system of any of claims 1, 2, 4, 6, 8, 10, 11, 12, and 13, wherein the user interface is displayed on the smartphone within the system.

15. The system of any of claims 1, 2, 4, 6, 8, 10, 11.

12. and 13, wherein the user interface is displayed on a computing article other than the smartphone within the system.

16. The system of any of claims 1. 2, 4, 6. 8, 10, 11, 12, and 13, further comprising an optical assembly attachment, the optical assembly attachment comprises two polarizers, a mirror, a beam splitter, and a converging lens, wherein the optical assembly attachment is configured to attach to the back of a smartphone.

17. The system of claim 16, wherein one of the two polarizers is configured to situate above a flashlight on the smartphone, the other of the two polarizers is configured to situate above the camera on the smartphone, the mirror is situated above and at an angle to the polarizer above the flashlight, the beam splitter is situated parallel to the mirror, and the converging lens is situated above and parallel to the polarizer above the camera.

18. The system of claim 17, wherein the optical assembly attachment further comprises a light trap to diffuse light from the beam splitter.- so19. The system of claim 17 or 18, wherein the optical assembly attachment further comprises a back plate to attach the optical assembly attachment to the smartphone.

20. The system of any of claims 1, 2, 4, 6, 8, 10, 11, 12, 13, 17, and 18, wherein the computer programming product is further configured to analyze the location of the camera relative to a human eye and give commands for maneuvering the camera to a predetermined location.21 . The system of claim 20, wherein the commands are given in by voice or by visual display.

22. A computer method to assess myopia of an eye, the method comprises: taking a picture of an eye using a camera and saving the picture as digital image data; locating a pupil image within the digital image data; extracting the pupil image data from the picture of the eye; separating image data into red, green, and blue channels; calculating statistical features of the image from red, green, and blue channels; using a Support Vector Machine algorithm to select appropriate statistical features for classification and rank the statistical features based on their contribution to either diopter or axial length prediction using Minimum Redundancy Maximum Relevance; and calculate diopter or axial length of the eye based on the statistical features ranking.

23. The method of claim 22, wherein locating a pupil image within the digital image data comprises searching for an object with a circular or substantially circular boundary.

24. The method of claim 22 or 23, wherein the statistical features of the image are at least one of red mean, red median, red standard deviation, red geomean, red brightness weighted average, green mean, green median, green standard deviation, green geomean, green brightness weighted average; blue mean, blue median, blue standard deviation, blue geomean, bluebrightness weighted average, blue mean - red mean, blue median - red median, blue standard deviation - red standard deviation, blue geomean - red geomean, blue brightness weighted average - red brightness weighted average; green mean - red mean, green median - red median, green standard deviation - red standard deviation, green geomean - red geomean, green brightness weighted average - red brightness weighted average; green mean - blue mean, green median - blue median, green standard deviation - blue standard deviation, green geomean - blue geomean, green brightness weighted average - blue brightness weighted average; blue mean / red mean, blue median / red median, blue standard deviation / red standard deviation, blue geomean / red geomean, blue brightness weighted average / red brightness weighted average; green mean / red mean, green median / red median, green standard deviation / red standard deviation, green geomean I red geomean, and green brightness weighted average I red brightness weighted average, wherein red mean refers to the mean intensity of all red pixels in the image, blue mean refers to the mean intensity of all blue pixels in the image, green mean refers to the mean intensity of all green pixels in the image, wherein red geomean refers to the geometric mean intensity7of all red pixels, green geomean refers to the geometric mean intensity of all green pixels, blue geomean refers to the geometric mean intensity of all blue pixels, wherein red median refers to the median intensity of all red pixels in the image, green median refers to the median intensity of all green pixels in the image, blue median refers to the median intensity of all blue pixels in the image, wherein red brightness average refers to the average of brightness of all red pixels in the image, green brightness average refers to the average of brightness of all green pixels in the image, blue brightness average refers to the average of brightness of all blue pixels in the image, wherein red standard deviation refers to the amount of variations of red pixels in the image, blue standard deviation refers to the amount of variations of blue pixels in the image, green standard deviation refers to the amount of variations of green pixels in the image, wherein blue mean - red mean refers the mean intensity of all blue pixels in the image minus by mean intensity of all red pixels in the image, green mean - red mean refers the mean intensity of all green pixels in the image minus by mean intensity of all red pixels in the image,green mean - blue mean refers the mean intensity7of all green pixels in the image minus by mean intensity of all blue pixels in the image, wherein blue median - red median refers to the median intensity of all blue pixels in the image minus by median intensity of all red pixels in the image, green median - blue median refers to the median intensity of all green pixels in the image minus by median intensity of all blue pixels in the image, green median - red median refers to the median intensity' of all green pixels in the image minus by median intensity of all red pixels in the image, wherein blue standard deviation - red standard deviation refers to the amount of variations of red pixels in the image minus the amount of variations of red pixels in the image, green standard deviation - blue standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of blue pixels in the image, green standard deviation - red standard deviation refers to the amount of variations of green pixels in the image minus the amount of variations of red pixels in the image. wherein blue geomean - red geomean refers to the geometric mean intensity of all blue pixels in the image minus by geometric mean intensity of all red pixels in the image, green geomean - blue geomean refers to the geometric mean intensity' of all green pixels in the image minus by geometric mean intensity of all blue pixels in the image, green geomean - red geomean refers to the geometric mean intensity of all green pixels in the image minus by geometric mean intensity of all red pixels in the image, wherein blue brightness weighted average - red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average - blue brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of blue pixel brightness values that exceed the median brightness level, green brightness weighted average - red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level minus by the weighted average of red pixel brightness values that exceed the median brightness level,wherein blue mean / red mean refers the mean intensity of all blue pixels in the image divided by mean intensity of all red pixels in the image, green mean / red mean refers the mean intensity of all green pixels in the image divided by mean intensity’ of all red pixels in the image, wherein blue median / red median refers to the median intensity’ of all blue pixels in the image divided by the median intensity of all red pixels in the image, green median / red median refers to the median intensity of all green pixels in the image divided by the median intensity’ of all red pixels in the image, wherein blue standard deviation / red standard deviation refers to the amount of variations of blue pixels in the image divided by the amount of variations of red pixels in the image, green standard deviation / red standard deviation refers to the amount of variations of green pixels in the image divided by the amount of variations of red pixels in the image, wherein blue geomean / red geomean refers to the geometric mean intensity’ of all blue pixels in the image divided by geometric mean intensity of all red pixels in the image, green geomean / red geomean refers to the geometric mean intensity of all green pixels in the image divided by geometric mean intensity of all red pixels in the image, wherein blue brightness weighted average / red brightness weighted average refers to the weighted average of blue pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level, green brightness weighted average / red brightness weighted average refers to the weighted average of green pixel brightness values that exceed the median brightness level divided by the weighted average of red pixel brightness values that exceed the median brightness level.

25. The method of claim 22 or 23, wherein the calculating of diopter and axial length is conducted by two distinct machine learning models.

26. The method of claim 22 or 23, wherein the camera is present on a smartphone and the method is conducted by a computer programming product installed on the same smartphone.

27. The method of claim 22, further comprising assessing the quality of the picture captured by the camera.

28. The method of claim 27. wherein the quality of the picture is assessed by resolution, lighting, focus, or blurriness.

29. The method of claim 27 or 28, further comprising compressing the pupil image data and upload the pupil image data to a cloud server, wherein the cloud server hosts the machine learning models.

30. The method of any of claims 22, 23, 27, 28, 30, further comprising displaying the diopter or axial length calculation result on a user interface.

31. The method of any of claims 22, 23, 27, 28, 30, further comprising sending the diopter or axial length calculation result to a computing article.

32. The method of claim 22, further comprising attaching an optical assembly to the smartphone, the optical assembly comprises two polarizers, a mirror, a beam splitter, and a converging lens, wherein the optical assembly attachment is configured to attach to the back of a smartphone.

33. The method of claim 32, wherein one of the two polarizers is situated above a flashlight on the smartphone, the mirror is situated above and at an angle to the polarizer above the flashlight, the beam splitter is situated parallel to the minor, the converging lens is situated above the camera between the camera and the beam splitter, and the other of the two polarizers is situated above and parallel to the converging lens.

34. The method of claim 31, wherein the optical assembly attachment further comprises a light trap to diffuse light from the beam splitter.

35. The method of any of claims 32 to 34. wherein the optical assembly attachment further compnses a back plate to attach the optical assembly attachment to the smartphone.

36. The method of any of claims 32 to 34, further comprising analyzing the location of the camera relative to an eye and giving commands for maneuvering the camera to a predetermined location.

Images