Contact lens with thin camera and method of manufacturing thereof

Abstract

The group of inventions relates to optical elements with integrated electronic components and specifically to contact lenses with a built-in ultrathin digital camera, which are applicable for capturing video images, object recognition, implementing the user eye-tracking function, including, without limitation, while forming virtual, augmented, mixed or extended reality (VR / AR / MR / XR), as well as for diagnosing and compensating ophthalmic diseases, and for other applications. The contact lens housing (1) made of an optically transparent material contains a built-in electronic component containing a power source (2) and a camera (3) with at least one receiver (4) in the form of a matrix (8) of photosensitive elements. The camera (3) is equipped with an optical system implemented as at least one coding mask (5) separated from the receiver (4) by an intermediate layer (9) of homogeneous optically transparent material and disposed on the surface of or inside the lens housing (1). The invention makes it possible to significantly reduce the thickness of the contact lens containing an integrated camera and to produce an operational device that is comfortable for daily wearing.

Description

CONTACT LENS WITH THIN CAMERA AND METHOD OF MANUFACTURING THEREOF

[0001] The group of inventions relates to optical elements with integrated electronic components and specifically to contact lenses with a built-in ultrathin digital camera, which can be used for capturing video images, object recognition, implementing the user eye-tracking function, including, without limitation, while forming virtual, augmented, mixed or extended reality (VR / AR / MR / XR), as well as for diagnosing and compensating ophthalmic diseases, and for other applications.

[0002] Recent years have seen high public interest in VR / AR / MR / XR devices. These devices are an alternative evolution of already classical smart phone screens and computer monitors. The key characteristics, optimal for the end user of AR devices, are the factors of compactness and matching the digital world with the picture of the surrounding real world. The most compact seems to be an AR device in the form of a contact lens in which the matching of the digital world with the surrounding reality is implemented using digital methods of capturing the real world followed by superimposing and matching the picture of the real world with the digital content. The most complete and optimal method of capturing the real world is using a digital camera integrated into a contact lens. Considering small thickness of a contact lens and the requirement for it to contain the whole image capturing system (a camera), such a task is not trivial. Classical optical systems used in alternative photo and video capturing devices are not applicable in a contact lens due to their bulkiness. A contact lens has a limited thickness and volume, and, by its nature, a contact lens is curved rather than flat, and therefore the integration requires using a specific digital camera of an ultrathin physical size.

[0003] The prior art discloses a lens-free camera image reconstruction method based on a coding mask and a trained neural network with regularization methods to solve the technical problem of noisiness of the obtained image (see patent CN112950750B, cl. G06N3 / 08, published 08.12.2023). A lens-free camera structurally represents a sequence of CMOS (Complementary Metal–Oxide–Semiconductor) matrix and a binary mask in the form of a chromium plating on quartz glass. The discussed primary algorithm used to reconstruct the image is the truncated singular value decomposition algorithm (Learned-TSVD algorithm) based on a trainable neural network. Using this reconstruction algorithm makes it possible to significantly reduce the noise level in the resulting image.

[0004] The disadvantage of the prior art device is that the lens-free camera is not adapted for integration into the contact lens body. Specifically, the discussed lens-free camera embodiment represents a 2 mm thin structure, which is not a satisfactory thickness for using in a contact lens. And, using a flat binary mask on quartz glass is a limitation appearing as a partial overlapping of the area of visibility due to the binary mask nontransparent segments, which reduces the energy efficiency of image capturing. Also, the process of initial calibration of the lens-free camera is limited by using vertical and horizontal calibration lines, which is a limiting factor for the obtainable accuracy of the resulting image reconstruction.

[0005] The prior art discloses an intraocular video system for vision enhancing and restoring, comprising a contact lens itself to be carried on a surface of an eye, a camera, and a display configured to receive the camera image and to apply additional digital processing and filtering (see publication WO2006015315A3, cl. A61F2 / 14, published 19.04.2007). The display is positioned within the user's eye and it projects a digital image onto the user's retina. In the prior art solution, the camera in the contact lens is technically implemented as an archwise camera obscura. The key disadvantage of the prior art solution is its large size on the scale of the contact lens thickness (approximately 0.05–0.2 mm). The camera obscura, due to its principle of operation, requires using a significant optical path to build a geometrical optical projection of an image through a small hole onto the signal electro-optical reading area. Significant reduction of the camera obscura typical dimensions results in that its principles of operation change from the geometrical optics to wave optics causing image quality deterioration (appearing of parasitic light diffraction orders and diffractive chromatic bias). Furthermore, the camera obscura provides rather low image contrast causing the necessity to use a small entrance aperture and resulting in increased requirements to electro-optical detector sensitivity, low framing rate and blurring of moving objects. Thus, the possibility to implement the prior art solution in practice looks rather problematic.

[0006] The prior art discloses an image capture system integrated into an active contact lens and providing image capturing by means of a thin component and a data processing system used to perform the functions of projecting, transmitting and capturing an image visible in the user's gaze area (see publication WO2014058733A1, cl. G03B15 / 00, published 17.04.2014). In the prior art system, the primary image capturing principle is represented by a camera operating based on the classical geometric optics principles but using a focusing optical component with a reduced physical thickness (a Fresnel lens or a thin lens with a gradient refractive index). Disadvantage of this optical system consists in a low quality of the detected image, which is caused by the optical system operation transiting from purely geometrical optical physical principles to wave optics, and in appearance of diffraction effects on the scale of the contact lens thickness. Furthermore, the prior art system requires a great difference in the refractive indices between the optical focusing element and the optically transparent part of the contact lens, which is difficult to implement in practice.

[0007] The prior art discloses a smart contact lens with an image capturing function, comprising two parts of the optical system, first of which represents a transparent viewing window participating in the correct functioning of the user's eye visual perception, while the second part represents a viewing window surrounding the first viewing window and participating in the image capturing process itself (see patent TWI662318B, cl. G02C7 / 04, published 11.06.2019). The second part of the optical system comprises an image input module, a module of subsequent image transmission based on a waveguiding principle within the contact lens body, and an image electro-optical sensing module. Disadvantage of the prior art device consists in low quality of the recorded image due to diffractive blurring and accumulation of the source image distortion error in the course of multiple reflections from the contact lens body boundaries. Furthermore, mechanical deformation of such a contact lens causes significant deterioration of the recorded image quality or its complete loss on its way from the source image to the electro-optical component. Thus, operability of the prior art solution when implementing in practice seems improbable.

[0008] The closest, in terms of technical substance, to the claimed invention is the smart contact lens with an integrated femtocamera for controlling and interacting with AR, where the femtocamera represents an optical system with a curved reflective surface and electro-optical detector installed along the pathway of the reflected rays (see patent US10712564B2, cl. G02B27 / 01, published 14.07.2020). This optical system is a scaled-down and simplified analogue of the Newtonian telescope optical system. Using such an optical system in a femtocamera makes it possible to reduce the length of the camera optical system by half in comparison with classical optical systems used in the lenses of typical video cameras. However, to ensure satisfactory quality of the detected image, the size of such a camera is estimated to be not less than 1–2 mm. Further decreasing the femtocamera thickness results in significant distortion of the detected image due to aberration effects and occurrence of diffraction orders. Thus, the prior art contact lens cannot be implemented with a thickness meeting the requirement of comfortable daily use (not exceeding 0.5 mm). Furthermore, the prior art camera has a relatively massive housing and blocks the user's direct view.

[0009] The technical problem is to eliminate the above disadvantages caused by using a limited set of classical optical schemes that cannot provide integration of the prior art digital cameras into a smart contact lens for daily comfortable use by an end user.

[0010] The technical effect consists in that the thickness of a contact lens with an integrated camera is significantly reduced and an operational device is produced that is comfortable for daily wearing.

[0011] The set problem has been solved and the technical effect as related to the device has been achieved by that in the contact lens comprising a housing that is made of an optically transparent material and contains a built-in electronic component containing a power source and a camera having at least one receiver in the form of a matrix of photosensitive elements and equipped with an optical system, said optical system is implemented as at least one coding mask separated from the receiver by an intermediate layer of homogeneous optically transparent material and disposed on the surface of or inside the lens housing.

[0012] The coding mask can be implemented as an amplitude coding mask formed by transparent and nontransparent segments. In this case, transparent and nontransparent segments can form an Hadamard matrix.

[0013] The coding mask can be implemented as a phase coding mask formed by transparent segments with different optical path lengths. The segments with different optical path lengths can be created by forming a lens relief surface pattern or by forming microlenses inside the lens housing that have different focal distances, or by disposing dielectric nanoparticles of the same composition but different sizes. In this case, the segments with different optical path lengths can be made of a material with the refractive index that is higher than that of the contact lens housing, or can be disposed inside the lens housing and made of a material with the refractive index that is lower than that of the contact lens housing. Also, the segments with different optical path lengths can be formed by transparent segments with different refractive indices. The segments with different refractive indices can be formed by disposing dielectric nanoparticles of different compositions.

[0014] The intermediate layer can be made of the material of the contact lens housing. The camera can be equipped with a single closed-form receiver disposed at the contact lens periphery or with a group of spatially separated receivers disposed at the contact lens periphery and combined into a unified system. Also, the receiver can be made optically transparent and disposed in the center of the contact lens. The camera can be oriented toward the surrounding space or toward the user's eye.

[0015] The set problem has been solved and the technical effect has been achieved as related to the method by that during the contact lens manufacturing comprising preparing an electronic component containing a power source and a camera that has at least one receiver in the form of a matrix of photosensitive elements and is equipped with an optical system, disposing the electronic component in the blank contact lens, encapsulating the electronic component, and forming the contact lens housing, the camera optical system is implemented as a coding mask disposed on the surface of or inside the lens housing, the coding mask is separated from the receiver by an intermediate layer of a homogeneous optically transparent material, and, following the encapsulation and formation of the contact lens housing, the obtained camera is calibrated.

[0016] The camera inside the contact lens housing according to the first version is calibrated using the method of linear mathematical transformations by placing the point light sources at given distances from the contact lens, obtaining the images thereof using the camera, and calculating the coding mask calibration characteristics for each of said distances. The camera inside the contact lens housing according to the second version is calibrated using the method of machine learning by placing 3D objects at the given distances from the contact lens, obtaining the images thereof using the camera, downloading the 3D image of said objects at given distances, and calculating, using deep neural networks, the coding mask calibration characteristics for the range of said distances. In this case, convolutional neural networks are preferably used. Encapsulation of the electronic component is preferably carried out during solidification of the lens housing material.Fig.1

[0017] shows the general layout view of the disclosed contact lens;Fig.2

[0018] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing an Hadamard amplitude coding mask therein (with the camera oriented toward the surrounding space);Fig.3

[0019] illustrates the camera assembly 3 of;Fig.4

[0020] shows the general layout view of an eye with the contact lens ofinstalled;Fig.5

[0021] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing several amplitude masks therein (with the camera oriented toward the user's eye);Fig.6

[0022] illustrates the camera assembly 3 of;Fig.7

[0023] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing a phase coding mask in the form of a lens relief surface pattern (with the camera oriented toward the surrounding space);Fig.8

[0024] illustrates the camera assembly 3 of;Fig.9

[0025] shows the general layout view of an eye with the contact lens ofinstalled;Fig.10

[0026] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing a phase coding mask in the form of dielectric nanoparticles of the same composition but of different sizes.Fig.11

[0027] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing a phase coding mask in the form of microlenses with different focal distances;Fig.12

[0028] illustrates the camera assembly 3 of;Fig.13

[0029] shows the general layout view of an eye with the contact lens ofinstalled;Fig.14

[0030] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing a phase coding mask in the form of air bubbles;Fig.15

[0031] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing a phase coding mask in the form of dielectric nanoparticles of different composition;Fig.16

[0032] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing a camera equipped with a single rim-shaped receiver;Fig.17

[0033] shows the general layout view of an eye with the contact lens ofinstalled;Fig.18

[0034] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing a camera equipped with a group of spatially separated receivers;Fig.19

[0035] shows the general layout view of an eye with the contact lens ofinstalled;Fig.20

[0036] is the schematic diagram of forming an image inside the user's eye by the disclosed lens containing a camera equipped with an optically transparent receiver in the center of the contact lens;Fig.21

[0037] shows the general layout view of an eye with the contact lens ofinstalled;Fig.22

[0038] illustrates the operation principle diagram of the disclosed camera;Fig.23

[0039] illustrates the scheme of the camera calibration using the method of linear mathematical transformations;Fig.24

[0040] illustrates an image directly detected by receiver 4, which was obtained during the thin camera 3 calibration using a point white light source placed at a 50 cm distance;Fig.25

[0041] illustrates the resulting image 18 reconstructed using the method of linear mathematical transformations from the image of;Fig.26

[0042] illustrates the scheme of the camera calibration using the method of machine learning with various focusing depths.

[0043] The disclosed contact lens conceptually differs from analogues in that it can be implemented in a comfortable form factor based on the components that already exist on the market, and also has broad prospects of further miniaturization with the development of science and technology.

[0044] The disclosed smart contact lens consists of the housing 1 made of an optically transparent material, wherein the housing contains a built-in electronic component: the power source 2 (constant power supply and battery charging module) and the camera 3. The electronic component can also include a data transmission module, a projecting system, physiological sensors, elements for implementing the control functions, etc., however this is not the subject matter of the present invention.

[0045] The camera 3 comprises one or more receivers 4 in front of which an optical system in the form of one or more coding masks 5 is installed.

[0046] Such an optical system for a thin camera significantly differs from classical optical digital capturing systems (successive lenses or mirrors) used in the analogues of the disclosed lens. The developed design is intended to depart from classical principles of optical system construction, which have the key limitation – fundamental impossibility of reducing their thickness to values necessary for integration into a contact lens that is comfortable for the user to wear. Unlike the classical, the disclosed optical system of the thin camera 3 can be of any form and geometry and at the same time does not require to maintain coaxiality of the sequence of optical components and distance referencing in accordance with their focal characteristics (which is mandatory for classical optical systems, such as photo camera lenses).

[0047] The form and geometry of the thin camera 3 and, accordingly, of the receiver 4 can be represented as: a separate rectangular element located near the pupil (most applicable when developing a thin camera based on widely commercially available CMOS, CCD, and other rectangular electro-optical matrices) (-); a single closed-form element, curved to match the shape of the pupil and disposed on the contact lens periphery; or a group (array) of spatially separated receivers (the electro-optical matrix is divided into a number of separate functional components on the contact lens periphery), which altogether form an unified thin camera system; and combinations of said configurations.

[0048] As there are no restrictions on the form and geometry of the thin camera 3, the produced camera can have a larger receiving area in comparison with classical optical systems (which are typically limited by circular geometry). In turn, using an electro-optical detector with a larger area makes it possible to enhance the camera 3 sensitivity and obtain distinguishable images at a lower intensity of external illumination of the observed objects. Furthermore, using the thin camera 3 with a curved geometry on a curved “spherical” surface (such as a human eye) makes it possible to obtain a wider camera field of view (FOV) as compared with classical axial optical systems. The high values of the camera 3 sensitivity characteristics and wide field of view angles give a significant advantage when using the disclosed smart contact lens in the night vision and high-speed image capturing applications in a wide range of viewing angles.

[0049] In one of the embodiments, the disclosed thin camera 3 integrated within a contact lens (housing 1) is disposed with its plane surface tangentially to the contact lens inner surface – this allows to use commercially available flat photosensitive matrices. Due to non-transparency (at this stage of industrial technology development) of a number of components (layers) of the thin camera 3, it is preferable to dispose it on the contact lens periphery, i.e. outside the pupil area, so that the body of the camera 3 does not obstruct the end user's view of the surrounding world.

[0050] Nevertheless, taking into account existing developments in the field of creating completely or partially transparent receivers 4, it is possible to dispose the thin camera 3 directly in the pupil area of the contact lens (-). Such an arrangement provides the camera 3 viewing direction alignment with the end user's eye, which gives significant advantage while referencing the digital world of augmented reality to the picture of real surrounding world.

[0051] In addition to the receiver 4 itself as a separate element, the contact lens housing is required to include additional electronic components of the camera 3 that ensure full functioning of the camera within the smart contact lens unified electrical system. These additional electronic components include: the power source 2 mentioned above; the camera controller 6 performing the coding and preprocessing of the data stream from the receiver 4 into a digital format suitable for inputting into the smart contact lens common processing module; the associated electronics 7 for the camera 3 integration into the unified electrical system of the whole smart contact lens (including, without limitation, for connection to the power source 2, control and command monitoring module, wireless data receiving and transmitting module, etc.).

[0052] Physically, the camera 3 represents a sequence of thin functional layers stacked on each other (,). The thin camera basic layered structure comprises: the coding mask 5, the matrix 8 of photosensitive elements and the intermediate layer 9 of homogeneous optically transparent material (the material of the lens housing 1 or other material preliminary applied onto the matrix 8) disposed between them.

[0053] The coding mask 5 represents a phase or amplitude, completely or partially optically transparent layer with the thickness of 0.05–30 μm, disposed on the surface of or inside the lens housing 1. Such an arrangement and the very use specifically of the coding mask 5 makes it possible to create a fully autonomous device of minimal size, implemented in a form factor of a thin contact lens with an integrated camera, which is comfortable for daily use — the result that is fundamentally unachievable for classical optical systems.

[0054] The primary function of the coding mask 5 is to superimpose its optical characteristics onto the imagery picture of the surrounding world, i.e., as a matter of fact, a compression is carried out, with the input image components being mapped onto a subspace formed by the static two-dimensional (2D) coding mask 5. A classical system uses direct trivial projection of the point spatial functions (a 2D image received from a point source), wherein each local spatial function is mapped onto a single pixel. To depart from this principle and ensure that one local point spatial function is mapped onto several pixels of the matrix 8 of photosensitive elements, an additional optical path is required apart from the coding mask 5 itself. For this purpose, between the mask 5 and the receiver 4, the intermediate layer 9 of homogeneous optically transparent material is formed, where the mapping of the local point spatial function onto a number of pixels takes place. Thereby, a small deviation of the point optical source from its original position results in significant alteration of the whole imagery of the point spatial function detected by the receiver 4. The intermediate layer can be formed separately during the camera manufacturing or simultaneously with forming the contact lens housing 1 from the same material.

[0055] The thickness of the intermediate layer 9 to a great extent depends on the characteristics of the selected coding mask 5 and the required focusing characteristics of the whole optical system, as well as on many other parameters, and approximately should be in the range from 10 μm to 1 mm. The projection picture obtained using the disclosed device, which is detected by the matrix 8 (CMOS or CCD), significantly differs from a classical image representation (a shot) obtained using classical photo capturing devices (lens-type and reflex cameras, photographic cameras, etc.), and represents a mask-coded source image of the surrounding space, where each local segment of the image (down to an individual pixel) contains information from the whole image or from most of it. Thus, using the disclosed design, provided the thicknesses of various functional layers are properly selected, the discussed thin camera 3 by its thickness parameter can fit within the thickness range of a contact lens for daily comfortable wearing by a human user (within a range not exceeding 500 μm).

[0056] The coding mask 5 can be represented by various optical masks, such as: amplitude discrete mask; discrete phase mask; adiabatic phase mask, etc.

[0057] The amplitude discrete coding mask 5 is a combination of transparent segments 10 and nontransparent segments 11, and can be implemented as a thin metal layer (50–1000 nm) produced using the method of stencil deposition onto the surface of a transparent substrate. The coding mask 5 characteristic feature for the thin camera 3 application as compared with other optical masks is that it must contain the coding parameters representing a wide range of spatial frequencies. Such discrete coding masks include: Fresnel-zone plates, Hadamard matrix masks, optimized random pattern masks, etc.

[0058] In addition to the amplitude discrete coding mask 5, by analogy, a discrete phase coding mask 5 can be used. Such a phase mask 5 can use a pixel structure in the form of the segments12 (columns) of an optically transparent material with various thicknesses, i.e. with various optical paths. The segments12 with various thicknesses can be formed directly from the housing 1 material by forming a lens relief surface pattern. The discrete coding masks 5 can be produced using the methods of classical lithography or using the methods of micro 3D-printing (e.g., the method of two-photon photopolymerization), as well as other well-known methods.

[0059] In the amplitude coding mask 5, the nontransparent segment 11 normally comprises about half the total surface area of the camera 3, while the phase mask represents a completely transparent optical layer and for efficient operation uses the whole surface area of the receiver 4. This characteristic feature of the phase coding mask is advantageous when the thin camera 3 is used in conditions of low light intensity. A large effective usable area with lower optical signal levels makes it possible to distinguish objects in the dark and also reduces the size of the whole thin camera 3 surface area.

[0060] Both phase and amplitude discrete coding masks 5, due to their discrete nature, are best suitable for the thin cameras 3 set to a fixed operating focal distance. However, when the position of the recorded object in the camera 3 operating field changes (when the discrete coding mask 5 is moved farther or closer), the quality of the obtained image significantly degrades, and the time needed for its algorithmic reconstruction increases. Said problem can be solved by using an adiabatic phase coding masks 5.

[0061] Similar segments with different optical path lengths can be formed from a material (e.g., nanoparticles –) with the refractive index that is higher than that of the contact lens housing 1, or can be disposed inside the lens housing and made of a material with the refractive index that is lower than that of the contact lens housing, e.g., of air bubbles.

[0062] In the simplest case, the adiabatic phase coding mask 5 represents a combination of segments with various optical path lengths, produced by disposing dielectric nanoparticles of the same composition but different sizes inside the housing 1 (,). For this purpose, the source nanoparticles with various degree of dispersion are mixed in, for example, a polymeric buffer, and the resulting composition is applied (poured) onto a finished intermediate layer 9 to create heterogeneous segments forming the coding mask 5. The difference in concentrations of the same nanoparticles in different local segments of the coding mask 5 results in the difference in refractive indices in those local segments and formation of segments with various optical path lengths.

[0063] In another embodiment, the phase mask 5 can be produced by forming overlapping microlenses with various focal distances inside the housing 1 (-). When using the coding mask 5 in the form of overlapping microlens surfaces (microlenses) with various focal distances (focusing characteristics), a single shot can be sufficient to successfully capture the objects in a wide focusing range. Using the methods of algorithmic processing optimized for the set of focal distances, several shots with various focusing depths can be reconstructed at once per a single shot. Such a property of the adiabatic coding mask 5 makes it possible to use well-known optical methods to obtain partial 3D representation of the objects within the camera 3 field of vision, which, in its turn, makes it possible to estimate the contact lens position (distance to the objects) in the surrounding space, which is comparable with the LiDAR technology function.

[0064] Also, the coding mask 5 can be formed by transparent segments 13 with different refractive indices, wherein the segments are produced, for example, by disposing dielectric nanoparticles of different compositions (i.e. different optical properties, for example, different substances or different phases of the same substance). For this purpose, the source various nanoparticles can be simultaneously mixed in a polymeric buffer and applied as a single composition onto the finished intermediate layer 9 to form heterogeneous segments 13 with different refractive indices and different optical path lengths. The optical element with a gradient refractive index on the scale of several tens of microns performs the coding functions for the source image, acting as the phase coding mask 5.

[0065] In the context of the disclosed design, the camera 3 can be oriented not only toward the surrounding space (outward) (-), but also toward the retina 14 of the user's eye (inward) (-). Both illustrated options imply disposing the thin camera 3 plane surface tangentially to the surface of the housing 1, although it is technically possible to make the receiver 4 curved to match the contact lens form.

[0066] In case when the camera 3 is oriented toward the interior of the eye (-), the image from the external environment 15, passing and focusing through the eye optical system, is projected onto the back side of the retina 14. The projected image can be recorded by the camera 3 aimed at the retina 14. Thus, when the thin camera 3 is oriented toward the interior of the eye, the image detected by the camera is the image directly visible to the human eye. Such a configuration is characterized in that it is able to refocus by means of the eye optical system (crystalline lens) itself. Under normal functioning, the user's eye has a dynamic focusing (accommodation): depending on the object of observation (external environment 15), the eye can focus at different distances, and the objects that are in the focus of observation are clearly focused on the back side of the retina 14 of the user's eye. In this case, the camera 3, even with a single focal distance corresponding the distance to the retina 14, in the course of proper eye accommodation always enables capturing clear images visible to the user, which, in its turn, makes it possible to significantly reduce the requirements for computing algorithmic resources for image reconstruction and also to avoid the camera 3 preliminary calibration for various focal distances.

[0067] Furthermore, the camera 3 oriented toward the interior of the eye and adjusted to a corresponding spectral range (IR), makes it possible, for example, to examine the blood circulation in the retina 14 for medical purposes.

[0068] The disclosed contact lens is produced as follows.

[0069] The first stage is assembling the electronic component: the associated electronics 7 is used to connect the power source 2, the camera 3 and its controller 6, and other required elements (data receiving and transmitting module, processing module, projecting system, control module, etc.) into a unified system. The camera 3 at this stage can be directly represented by only one or several receivers 4 with the matrices 8 of photosensitive elements or by already fully functioning flat camera 3 with the optical system formed by the whole layered structure assembly (matrix 8, intermediate layer 9, and coding mask 5).

[0070] Then, the assembled electronic component is disposed in the blank contact lens, for example, directly in the polymeric composition of the contact lens housing 1 material between the forming molds (hollow forms) to produce the contact lens. Thereafter, using the UV or chemical curing, in the process of the housing 1 material solidification, the electronic component is encapsulated with simultaneous formation of the contact lens housing 1. In case the coding mask 5 represents a lens relief surface pattern (-) and the intermediate layer 9 is made of a homogeneous optically transparent material of the housing 1, the stencil for forming the mask 5 can be integrated directly into one of the molds.

[0071] After encapsulation and formation of the contact lens housing, the obtained camera 3 undergoes calibration. Since, at this stage, the image formed on the receiver 4 already depends on the real parameters and imperfections of the fabricated system, all the features of the specific product can be accounted for in a single calibration stage — thereby the quality of the computed image can be significantly enhanced, which positively effects the overall performance of the device.

[0072] The disclosed system of capturing using a contact lens with the thin camera 3 relates to computational photography. The resulting image is obtained in two successive stages.

[0073] At the first stage, the light from external environment (external imagery) 15 passes through the optical coding mask 5 and intermediate layer 9, and then is projected onto the photoelectric matrix 8 of the receiver 4. When the light passes through the coding mask 5, the matrix characteristics of the source image of the external environment 15 are “multiplied” by the matrix characteristics of the optical coding mask 5. As a result, the obtained optical imagedetected by the receiver 4 significantly differs from the source image and represents a highly mixed source image with a nonuniform intensity, which cannot be directly recognized by a human user without using computer methods.

[0074] At the second stage, the optical image is converted using the CMOS / CCD matrix 8 of photosensitive elements into the electric signal 16 in a matrix (pixel) format and enters the controller 6, and then goes to the required source image algorithmic computation and reconstruction module 17, which also can be integrated into the contact lens electronic component or can be disposed outside the device to enable the use of powerful computational capacities.

[0075] Computational algorithms can be implemented in a great variety of ways, however, in a varying degree, all of them are based on the mathematical principles of compressed scrambling (CS). In addition to classical algorithmic principles of image reconstruction in the discussed thin camera, it is possible to use the methods based on Machine Learning (ML) and deep neural networks, in particular convolutional neural networks. Approaches based solely on mathematical algorithmic reconstruction of the resulting image, which by their nature are iterative with resource-intensive matrix transformations, can cause significant delay between the real events in the environment and resulting images obtained using the thin camera 3. Whereas the ML-approaches are characterized in high efficiency in terms of the required computational power and the thin camera 3 image computational speed. The ML-methods are particularly preferable when the whole system is completely integrated into the contact lens, as the lens size largely restricts the potential computational resources and their power consumption.

[0076] Both for solely mathematical algorithms and for algorithms based on the ML-methods, obtaining the resulting image 18 from the signal 16 detected by the receiver 4 requires preliminary one-time “adjustment” prior to the thin camera 3 basic operation process. This very adjustment is called the camera 3 calibration. The calibration method depends on the chosen method for computing the resulting image 18 — either solely algorithmic mathematical method or ML-based methods.

[0077] When the calibration of the camera inside the contact lens housing is carried out using the method of linear mathematical transformations (without using the ML-approaches), the most accurate is the calibration using fixed point sources of optical emission in the thin camera 3 vision area (field of view). To calibrate using such a method, the point source 19 of white light is set at a given fixed distanced1from the contact lens, wherein the distance matches the required focusing characteristics of the thin camera 3. On the receiver 4, a shot containing the image of the point source 19 is obtained through the optical coding mask 5, following which a preliminary mathematical optimization computation of the resulting image 18 is carried out. Considering that it is known that only the point source 19 is located in the camera field of view, an iterative algorithmic process is used to calculate a calibration matrix, which is then used in the thin camera 3 operation, to compute the resulting image 18 from the image 15 detected by the receiver 4. The calibration matrix from a single point source of white light represents a matrix mathematical representation of the coding optical mask 5 used in the thin camera 3. In the primary algorithm of the resulting image 18 computation, the use of the coding mask 5 matrix representation makes it possible to perform the final computation of the resulting image 18 by “subtracting” the optical coding mask 5 matrix characteristics from the image 15 detected by the receiver 4.

[0078] An important characteristic of the thin camera 3 using the algorithm based on linear matrix transformations is that each individual focal distance (d1,d2, etc.) of the thin camera requires its own individual calibration. For each focal distance, the calibration can be carried out similarly by placing the point source 19 of white light in the corresponding plane for each focal distance. In this case, the image 15 is obtained and the coding mask 5 calibration characteristics are calculated for each of said distances. Increasing the number of successive calibrations using the point sources 19 in various spatial positions in one focal plane makes it possible to enhance the accuracy of mathematical matrix definition of the optical characteristics of the optical coding mask 5 used in the thin camera 3. More accurate mathematical definition of the coding mask 5 makes it possible to significantly enhance the accuracy of the computed resulting image 18 in relation to the source image of the external environment 15.

[0079] In case when the camera inside the contact lens housing is calibrated using the method of machine learning (ML –), three-dimensional objects 20 are placed in the thin camera 3 vision area at the given distances (d1,d2, etc.) from the contact lens, and, using the receiver 4 of the camera 3, the signal 16 is obtained. Then, the three-dimensional images of said objects 20 at the given distances (wherein the images are obtained using other well-known methods of 3D-scanning) are loaded into the algorithmic computation and reconstruction module 17, and deep neural networks are used to calculate the coding mask 5 calibration characteristics for the range of said distances. When using the optimization ML-method, it is of the utmost importance that the learned sample is represented by the most diversified scenes of the object 20 images and also to ensure that sufficient number of images (more than 1000) are provided to achieve acceptable quality of the trained ML-algorithm. The ML-algorithm characteristic feature is that the calibration can be carried out both to recognize the objects 20 as such and to identify the focal distance (distanced1,d2) to the object 20. Thus, when carrying out a complete calibration: both for the focusing depth (by presenting the object 20 at various distances from the camera 3 being calibrated) and for the objects 20 being recognized as such (by presenting different objects 20), it is possible to produce the camera 3 that can output not only 2D images 18 (like classical photo cameras), but also 3D images 18 with the reference to the spatial position in relation to the camera 3 (i.e. the camera 3, as a matter of fact, can act as a LiDAR sensor).

[0080] From the above description, it is obvious that for the thin camera 3 designed to be integrated into the contact lens housing 1, the calibration must be carried out directly in the contact lens body where it will be used by the end user (rather than during the thin camera manufacturing as a separate component, as it is done in the classical case). This results primarily from the fact that the application of additional optical layers of the housing 1 during encapsulation will cause an increase in the additional optical path leading to deviation of the characteristics computed during calibration.

[0081] Also, it should be taken into account that the user eyes can have small deviations in the surface shape, such as different radii of curvature, different degrees of curvature, etc. When using the thin camera 3 that has any flexible areas, the calibration can result in slightly different calibration characteristics for each user. However, for the users with different eye geometry the disclosed method of producing requires no modification to the thin camera 3 optical scheme — in use, it will be sufficient to load the user-specific calibration characteristic into the module 17 of algorithmic computation of the resulting image 18. Said fact makes the disclosed contact lens with a thin camera a versatile solution for a wide range of users.

[0082] The described characteristic features of the disclosed invention make it possible to significantly reduce the thickness of the contact lens containing an integrated camera and at the same time to produce an operational device that is capable of reconstructing the source image accurately and comfortable for daily wearing. Depending on the options required, the camera in the disclosed contact lens can realize the functions of a microscope, user eye tracking sensor, preprocessing filter (boundaries and moving objects), blink monitoring (including for controlling the contact lens), alert from external sensors (approaching in the driver’s or pedestrian's blind zones), identifying an object in the field of view, vision improvement, etc.Examples

[0083] Example 1

[0084] Realizing the effect of augmented reality by referencing the digital world to the user surrounding space.

[0085] This scenario belongs to the classical use of the digital camera 3 in an augmented reality (AR) application. The use of the thin camera 3 in a contact lens is similar to using classically designed digital cameras in augmented reality glasses. Digital cameras make it possible to track the objects surrounding the user and precisely match the digital world objects with the real world objects, thereby creating realistic and full immersion into the “world” of augmented reality.

[0086] The thin C-shaped or O-shapedcamera 3 in the form of a structure of successive layers is assembled at a separate initial stage preceding the thin camera 3 integration into the contact lens housing 1. The receiver 4 (commercially available electro-optical matrix 8 produced using classical lithography methods) is coated with a thin intermediate layer 9 of transparent polymeric material having the refractive index lower than the refractive index of the employed coding mask 5. The intermediate layer 9 of transparent material (100-400 μm) can be applied using classical application methods, such as centrifugation or deposition. The transparent material of the layer 9 acts as an optical gap between the coding mask 5 and the receiver 4.

[0087] Using the metal (Au, Ag, Ti, W, etc.) magnetron sputtering method, a substrate for the amplitude binary coding mask 5 is applied onto the surface of layer 9. The thickness of the deposited metal is selected so that the deposited layer completely blocks the visible light spectrum. Then, on the layer of the deposited metal, the coding mask 5 is formed by “opening” a number of the mask segments arranged in a binary sequence that forms a pattern of an Hadamard binary matrix. The “opening” of the segments can be accomplished using the methods of classical UV or electronic lithography with a photoresist application stage and subsequent chemical etching through the photoresist mask. Alternatively, said segments can be “opened” using the methods of laser engraving.

[0088] Following the process of the thin camera 3 production as a separate element, it is connected into the smart contact lens common electrical circuit. The camera 3 is connected to the power source 2, to the control and image processing system, and to the smart contact lens wireless data transmission system. The electrical connection can be accomplished using flexible conductive pastes or thin flexible conductive metal tracks.

[0089] After the process of the thin camera 3 connection and integration into the contact lens common electrical system, the camera 3 and other electronic components (auxiliaries) are coated with a protective layer (passivation) with subsequent final forming to produce the contact lens housing 1. Finally, the lens surface, using the method of chemical vapor deposition (CVD), is coated with a superficial biocompatible layer of a covering material, for example perylene.

[0090] To give the smart contact lens the properties enhancing the smart contact lens comfort and time of wearing by the end user: high oxygen permeability, high degree of softness and flexibility, etc., the contact lens can be produced using the methods involving combination of the materials of several functional layers. Thus, the contact lens, after applying the protective biocompatible layer, can be locally perforated using the methods of laser engraving, which will result in increasing gas permeability of the whole contact lens. Also, after perforating, an additional thin layer with a greater degree of softness can be applied, such as PDMS or hydrogel. The soft layers can be applied using the methods of centrifugation, spraying or molding with subsequent UV or thermal curing.

[0091] Once the smart contact lens is produced, it is installed into a calibration fixture that comprises: a human eye shaped thin rigid transparent surface; a reference digital camera installed on the back side of the thin camera (where the retina 14 is supposed to be located); a digital screen (system of screens) installed in front of the smart contact lens at a distance (distances) corresponding to the camera 3 operating focal distance requirements.

[0092] During the calibration process, the images are successively output to the digital screen (screens), the parameters of reconstruction algorithm are fine-tuned (a calibration matrix is obtained), and the reconstructed image is optimized by software algorithms using the module 17. Upon completion of the calibration process, the calibration data is recorded into the smart contact lens memory using wireless flashing methods.

[0093] The produced and calibrated smart contact lens with the integrated thin camera 3 and digital image projection system can acquire data on the object positions in the external environment and match (impose) the digital image with the surrounding world imagery. In such a device, the external environment image is detected by the camera 3 and reconstructed using the module 17, following which the reconstructed image is sent to the common data processing module (processor) that uses the augmented reality algorithms to position the digital data on the surrounding world imagery picture, and the positioned image is output to the contact lens projecting system. As a result, the camera accurately and seamlessly integrates the digital world with the real world, which makes it possible to achieve the best quality of the augmented reality effect.

[0094] Example 2

[0095] Implementing the eye (retina and eyelid) microscopy.

[0096] For this application, a thin camera, based on a phase coding mask 5 with a varying local mask thickness or with a varying local gradient refractive index, can be used.

[0097] The production of the coding mask 5 with a varying local thickness (segments 12) for the thin camera 3 integrated into the contact lens housing 1 requires several successive fabrication stages. The initial stage consists in producing a forming mold whose relief pattern is then transferred onto the thin camera 3 outer layer to form the phase coding mask 5. The characteristic size of the phase mask elements, and, correspondingly, of the forming mold, must be within 5–100 μm. The forming mold can be produced using the micro turning technique, the lithography technique with subsequent etching, or micro 3D printing (for example, by means of two-photon photopolymerization), the latter of which enables producing a continuous surface curvature, which in a model approximation can be defined by a set of microlenses with various focusing characteristics (various focal distances). The forming mold design is chosen considering the phase mask final layer refractive index and the requirement of focusing at a short distance (within the diameter of a human eye). Also, the phase mask designing must take into account the characteristics of the phase mask type: a binary mask with sharp local boundaries or an adiabatic mask with a continuous transition throughout the relief pattern thickness. It should be considered that an adiabatic mask is a preferable option when integrating the thin camera into the contact lens body, as it makes it possible to eliminate noise aberrational effects at the sharp local thickness boundaries typical of a binary mask.

[0098] Once the forming mold is produced, at the next stage the forming mold relief pattern is transferred onto the surface of the thin, not finally formed, intermediate layer 9 applied directly onto the matrix 8 of photosensitive elements. This transferring can be carried out either by a thermal method (in case the intermediate layer 9 is thermally meltable) or by UV photocuring (when the forming mold is brought into contact with the liquid polymer of which the intermediate layer 9 is made). If made of a highly rigid material, the forming mold can be reused in the phase coding masks 5 production process many times.

[0099] The phase coding mask 5 with a local gradient refractive index (segments 13) can be produced using nanoparticles. Such a coding mask 5 represents a thin layer of optically transparent material, such as PMMA (n=1.4905) or PDMS (n=1.4035), acting as a “framework”, which is filled with nanoparticles (the employed high-refractive material can be ZnO, TiO2or ZnS). The employed high-refractive material can also be a Van der Waals material consisting of two-dimensional layers bonded together by Van der Waals forces, such as, without limitation, graphite, hexagonal boron nitride, MoS2, MoSe2, WS2, WSe2, SnS2, SnSe2, PtS2, PtSe2, PtTe2, ReS2, ReSe2, Cd3As2, Cd3Sb2, Cr2AlC, Cr2C, Mn2AlC, Mo2C, Mo2Ga2C, Mo3AlC2, Nb2AlC, Nb2C, Nb4AlC3, Nb4C3, Ta2C, Ta4AlC3, Ti2AlC, Ti2AlN, Ti2C, Ti2N, Ti3AlC2, Ti3C2, Ti3CN, Ti3SiC2, Ti4N3, V2AlC, V2C, V4AlC3, V4C3, PdS2, PdSe2, PdTe2, ZrS2, ZrSe2, GaSe, Sb2Te3, GaS, GaSe, GaTe, Ca(OH)2, Mg(OH)2, MnO2, MoO3, Sb2O3, Sb2OS2, Sb2S3, Sb2Se3, Sb2Te3, As2S3, As2Se3, As2Te3, Bi2O2Se, Bi2S3, Bi2Se3, Bi2Te3, BiSbTe3, AsP, CdI2, CdPS3, CuS, CoPS3, Cr2Ge2Te6, Cr2S3, CrBr3, CrCl3, CrGeTe3, CrPS3, CrSeBr, CuCrP2S6, CuIn7Se11, FeCl2, FePS3, FePSe3, GaGeTe, GaInS3, GaSeTe, GaSSe, GaPS4, GaSTe, HfSe2, HfS2, In2S3, In2Se3, InSe, InTe, InSeBr, InSnSe, MoTe2, WTe2, NbS2, NbSe2, NbSe3, VSe2, ZrSe3, MoSSe, MoWSe2, MoWS2, MoWTe2, MoNbSe2, MoO2.5Cl0.5, MoReS2, MoTaSe2, MoVSe2, Na2Co2TeO6, Nb2SiTe4, NbReS2, NbReSe2, NbS3, Ni2SiTe4, Ni3TeO6, NiCl2, NiI2, NiPS3, PbI2, PbTe, ReNbS2, ReNbSe2, ReSSe, Sb2OS2, SbAsS3, SbSe, SbSi, SiP, SnPSe3, SnS, SnSe, TaS2, TaS3, TaSe2, TaWSe2, TlSe, TiBr3, SnTe2, TiS2, TiS3, TlGaS2, TlGaSe2, TlGaTe2, TlInS2, WNbSe2, WReS2, ZrS2, ZnIn2S4, ZnPS3, ZnPSe3, ZrGeTe4, ZrS3, ZrSe2, ZrTe2or ZrTe3, with the Gaussian distribution thereof over the “framework” material. Such a phase coding mask 5 can be produced by mixing the prepared nanoparticles in a basic transparent optical polymer, such as PMMA or PDMS, with subsequent, respectively, thermal or chemical curing. The thickness of the produced layer must not exceed 0.1–1 μm — this requirement is defined by the necessity to form a 2D coding mask (to use 2D approximation in the final image 18 reconstruction algorithm), as well as by possibility to form, within the thin layer, the nonuniform local segments 13 with various concentrations of nanoparticles and, respectively, various refractive indices.

[0100] The important characteristic feature of the thin camera 3 is that its layered structure includes the 10–500 μm thick intermediate layer 9 with homogeneous optical transparency, wherein the layer acts as an optical path between the optical coding (phase or amplitude) mask and the photoelectric receiver matrix.

[0101] The produced thin camera 3 based on the phase coding mask 5 with a varying local thickness or with a local gradient refractive index is integrated into the contact lens housing 1 and connected to the respective smart contact lens auxiliaries (power source 2, controller 6, wireless data transmission system, etc.) similarly to Example 1. In case the camera 3 is used for the eyeground microscopy, it is oriented inward, toward the retina 14. In case the camera 3 is used for the eyelid inner side microscopy, it is oriented outward and displaced toward the periphery of the contact lens housing 1.

[0102] Such a thin camera 3 integrated into the contact lens housing 1 is calibrated on a calibration fixture, which, in case the camera 3 is oriented inside the eye (for the purpose of the user's eyeground microscopy), represents an imitation / model human eye with simulated optical characteristics of a biological eye and characteristic features, such as blood vessels with a blood flow, or a retina. The retina and eyeground microscopy enables monitoring of microscopic structural disorders associated with diseases and also enables measuring biomedical parameters of the user’s eye blood. When the camera is oriented inside the eye for carrying out microscopy of capillaries and substance flow therein, the calibration must be performed with the provision of a set of various configurations and orientations of the blood vessels, as well as with various substance compositions in the vessels.

[0103] In case the camera is used for eyelid microscopy and is oriented outward in relation to the user's eye, it is necessary to use a calibration fixture capable of simulating an eyelid and eyelid characteristics required to be monitored. The human eyelid inner side has numerous near-surface capillaries with biological fluids (blood). Thus, when the thin camera is oriented from the eye toward the surrounding world, it becomes possible to carry out the real time blood microscopy based on the smart contact lens.

[0104] In both applications (user's eyeground microscopy and eyelid microscopy), the camera must be calibrated using a dataset corresponding to the final objective. This dataset can be provided by circulating a test substance (simulating a blood flow) in a set of various designs of capillary configurations.

[0105] In other respects, the thin camera calibration process is similar to Example 1: a set of various data configurations (more than 10,000 different configurations) is input, and an algorithmic optimization search for the transmission matrix is carried out to achieve the maximum accuracy of the reconstructed resulting image 18 in relation to the original image obtained by a classical camera. Once the calibration matrix is obtained, it is loaded into the smart contact lens memory. The possibility to load various sets of calibration matrices makes it possible to take into account the variety and specifics of the user eyes and to perform quick programmed choice of characteristics for the most accurate obtainment of the required target values and images.

[0106] Example 3

[0107] Expanding spectral possibilities of the user's vision.

[0108] Unlike the human vision whose spectral sensitivity ranges from 350 to 760 nm, the camera 3 can have a wider spectrum of optical sensibility. Whereas the human eye is unable to distinguish objects in the dark (at low light intensity), the camera 3 can be tuned to the IR range. To tune the thin camera 3 integrated into the smart contact lens housing 1 to the IR range, it must contain the electro-optical matrix 8 of photosensitive elements that are sensitive to the IR spectrum. Such a matrix 8 can be produced based on the quantum dots of cadmium selenide (CdSe), quantum dots of india arsenide (InAs), quantum dots of lead germanide (PbGe), quantum dots of gallium arsenide (GaAs), etc.

[0109] Also, it is necessary to use the coding mask 5 optimized for the IR range (made of such materials as: zinc selenide (ZnSe) – for the range from 0.6 to 22 μm; germanium (Ge) – for the IR range up to 12 μm; as well as selenium (Se); zinc sulfide (ZnS); calcium fluoride (CaF2)). In this case, the coding mask 5 can be both amplitude and phase one.

[0110] The amplitude mask can be produced by drilling holes in a thin layer of the material used in the IR range, or micro laser cutting methods can be used to produce the amplitude coding mask 5 with better alignment of geometry of the local binary elements with geometry of an individual photosensitive element of the matrix 8.

[0111] The phase coding mask 5 can be produced either using the micro turning method of processing a thin material layer, used in IR engineering, or using the laser cutting or laser lithography with subsequent wet etching of the thin layer according to a design corresponding to various local thickness characteristics of the phase coding mask 5 being formed.

[0112] The optical coding mask 5, both amplitude and phase one, can be placed into the matrix 8 as follows: the photosensitive elements of the matrix 8 are coated with the 10–500 μm thick intermediate layer 9 (the material applied by spraying or centrifugation can be the same as the material of the contact lens housing 1, i.e. PMMA or PDMS), after that a thin layer of the material used in the IR spectral range is applied (for example, by gluing a rigid thin plate onto the surface of the liquid PDMS or PMMA layer with subsequent curing of the liquid phase). Produced as a separate element, the thin camera 3 with IR sensitivity can be integrated into the contact lens housing 1 similarly to Example 1. The camera must be oriented in accordance with the intended use.

[0113] The resulting image 18 from the IR camera 3 integrated into the contact lens housing 1, by means of digital processing, is converted into the visible spectrum and directly projected toward the eye using an optical projecting system. The projected digital image can be additionally processed to highlight the object boundaries within the user's and camera 3 field of visibility with subsequent superimposition of the digital information onto the imagery of the surrounding world. Matching the digital image received from the IR camera 3 with the imagery of the external environment 15 enables expanding the dynamic spectral visibility range of the human eyes.

[0114] Also, the camera 3 can be optimized for the UV range, x-ray range, etc., and any combination thereof. Alternatively, in a single smart contact lens housing 1, it is possible to use several cameras 3 tuned to different spectral sensitivity. It should be considered that each spectral range being expanded requires using the materials with the range-specific spectral characteristics — this relates to both the material of the matrix 8 of photosensitive elements and to the material of the coding mask 5. Also, it is important to ensure that the materials of other elements (intermediate layer 9, as well as thin camera 3 protective layers, and the contact lens itself), located on the optical path from the source of emission to the matrix 8, have the transparency windows for the target capture spectrum.

[0115] Example 4

[0116] User eye tracking using a contact lens.

[0117] The requirement to track the user's gaze in augmented reality systems is a key factor for the task of correctly superimposing the digital image onto the imagery of the surrounding world, as well as for the augmented reality control and monitoring by the user. The disclosed contact lens with the camera 3 oriented outward makes it possible to implement this function by referencing the control points to the external surrounding objects. A change in the camera position or orientation causes transition to another camera angle to surrounding objects, which makes it possible to identify the motion direction and the final position.

[0118] The thin camera 3 for such an application can be produced using the process similar to Example 1 (based on an amplitude mask) or Example 2 (based on a phase mask). The camera 3 as a separate element in the contact lens housing 1 in this application performs an auxiliary function for the process of forming an augmented reality. The camera 3 can be installed closer to the housing 1 periphery to avoid blocking the user’s eye vision area, with a certain geometrical correction of the angle in relation to the gaze direction.

[0119] Once fabricated, the contact lens with the integrated camera 3 is installed into the calibration fixture simulating the external environment 15 and the contact lens movements in space, after which it is calibrated using classical algorithmic methods of image reconstruction by means of linear mathematical transformations or algorithms employing machine learning. The important characteristic feature of the calibration process is that the calibration parameters (calibration matrix) are selected for the task of obtaining the maximum accuracy of the contact lens spatial position reconstruction, rather than for the maximum accuracy of the final image reconstruction. For this purpose, most appropriate will be the use of the machine learning methods with a task equivalent of enhancing the quality of positioning by external environment.

[0120] In such an application, the camera 3 must continuously receive a video signal referenced to a specific object. When the user's eyes move in relation to this object, from the relative motion the position and direction of the user's eye in space can be precisely identified. When using a system of two smart contact lenses with cameras 3, there is an additional possibility to track the user gaze focusing depth (by viewing direction orientation of one eye in relation to the direction of the other eye) for the task of representing and superimposing the objects in augmented reality.

[0121] Example 5

[0122] Highlighting the object boundaries to enhance the vision quality in inoperable cases of human vision loss.

[0123] In medical practice, patients often suffer from eyesight problems for which surgical methods of vision correction are not applicable. The causes of inoperability of the patient's eyes can include: vascular, autoimmune, and immunodeficiency disorders and diseases reducing the body’s ability to recover from illnesses. The problem related to the need to enhance a patient’s vision without surgical intervention can be solved using a contact lens with the thin camera 3 and a projecting optical system aimed at the user’s retina.

[0124] The thin camera 3 for this application can be produced similarly to Example 1 (based on an amplitude mask) or Example 2 (based on a phase mask). The thin camera must be oriented toward the external environment 15 (outward from the user's eye).

[0125] In this combination, the thin camera enables capturing the imagery of the external picture of the user’s surrounding world, processing the image, and sending to the projecting optical system, which is chosen and designed in such a way that the image that is projected onto the user’s retina and transmitted to the user’s visual channel has high contrast and sharpness. Thus, the combination of the thin camera and the projecting optical system makes it possible to bypass the user's eye optical system (which is supposed to be unable to function normally and represents an inoperable case) while projecting the image of the external environment 15 from the camera 3 to the retina 14 or into the user’s visual channel. Such a system comprising the contact lens with the thin camera 3 and the projecting system is a full functional digital analogy of the human eye optical system.

[0126] Such a system can have additional elements integrated therein to process the video signal from the digital thin camera 3, such as the modules that provide highlighting of the object boundaries (can be used as an additional superimposition of the object boundary digital mask onto the real imagery when the user has a reduced quality of vision). Such an additional digital superimposition of the object boundary digital imagery onto the real picture formed by the user’s optically impaired eye makes it possible to significantly enhance the vision quality, as the user acquires more accurate perception of the object size and boundaries. Also, it is possible to use an additional module providing visibility in a spectral range in which the patient has deficiencies, or to use any other methods of further digital signal processing to meet the requirements for improving the user's / patient's vision.

Claims

A contact lens comprising a housing 1 that is made of an optically transparent material and contains a built-in electronic component containing a power source 2 and a camera 3 having at least one receiver 4 in the form of a matrix 8 of photosensitive elements and equipped with an optical system, wherein the optical system is implemented as at least one coding mask 5 separated from the receiver 4 by an intermediate layer 9 of homogeneous optically transparent material and disposed on the surface of or inside the lens housing 1.The contact lens according to claim 1, wherein the coding mask 5 is implemented as an amplitude coding mask formed by transparent 10 and nontransparent 11 segments.The contact lens according to claim 2, wherein the transparent 10 and nontransparent 11 segments form an Hadamard matrix.The contact lens according to claim 1, wherein the coding mask 5 is implemented as a phase coding mask formed by transparent segments 12 with different optical path lengths.The contact lens according to claim 4, wherein the segments 12 with different optical path lengths are created by forming a lens relief surface pattern.The contact lens according to claim 4, wherein the segments 12 with different optical path lengths are created by forming microlenses inside the lens housing 1 that have different focal distances.The contact lens according to claim 4, wherein the segments 12 with different optical path lengths are formed by disposing dielectric nanoparticles of the same composition but different sizes.The contact lens according to claim 4, wherein the segments 12 with different optical path lengths are made of a material with the refractive index that is higher than that of the contact lens housing 1.The contact lens according to claim 4, wherein the segments 12 with different optical path lengths are disposed inside the lens housing 1 and made of a material with the refractive index that is lower than that of the contact lens housing 1.The contact lens according to claim 4, wherein the segments 12 with different optical path lengths are formed by transparent segments with different refractive indices.The contact lens according to claim 10, wherein the segments 12 with different refractive indices are formed by disposing dielectric nanoparticles of different compositions.The contact lens according to claim 1, wherein the intermediate layer 9 is made of the material of the contact lens housing 1.The contact lens according to claim 1, wherein the camera 3 is equipped with a single closed-form receiver disposed at the contact lens periphery.The contact lens according to claim 1, wherein the camera 3 is equipped with a group of spatially separated receivers disposed at the contact lens periphery and combined into a unified system.The contact lens according to claim 1, wherein the receiver 4 is made optically transparent and is disposed in the center of the contact lens.The contact lens according to claim 1, wherein the camera 3 is oriented toward the surrounding space.The contact lens according to claim 1, wherein the camera 3 is oriented toward the user's eye.A contact lens manufacturing method comprising preparing an electronic component containing a power source 2 and a camera 3 that has at least one receiver 4 in the form of a matrix 8 of photosensitive elements and is equipped with an optical system, disposing the electronic component in the blank contact lens, encapsulating the electronic component, and forming the contact lens housing 1, wherein the camera 3 optical system is implemented as a coding mask 5 disposed on the surface of or inside the lens housing 1, the coding mask 5 is separated from the receiver 4 by an intermediate layer 9 of a homogeneous optically transparent material, and, following the encapsulation and formation of the contact lens housing 1, the obtained camera 3 is calibrated.The method according to claim 18, wherein the camera 3 inside the contact lens housing 1 is calibrated using the method of linear mathematical transformations by placing the point light sources 19 at given distances from the contact lens, obtaining the images thereof using the camera 3, and calculating the coding mask 5 calibration characteristics for each of said distances.The method according to claim 18, wherein the camera 3 inside the contact lens housing 1 is calibrated using the method of machine learning by placing 3D objects 20 at given distances from the contact lens, obtaining the images thereof using the camera 3, downloading the 3D image of said objects 20 at given distances, and calculating, using deep neural networks, the coding mask 5 calibration characteristics for the range of said distances.The method according to claim 20, wherein convolutional neural networks are used.The method according to claim 18, wherein the encapsulation of the electronic component is carried out during solidification of the lens housing material.

Images