Augmented reality interface control system and method

Abstract

The invention relates to the augmented reality (AR) interface control systems and methods. The technical effect consists in expanding the augmented reality system functionality through the possibility to control the AR interface by means of bioelectric signals detected using the contact lenses, and enhancing the compactness of the wearable part of the control system. The control system comprises a contact lens, a projecting module for projecting the augmented reality interface into the user's eye, a sensor configured to detect the user's bioelectric signals, a processing module, and a control module configured to modify the augmented reality interface in accordance with the control signals. Said sensor comprises at least two primary electrodes integrated into said contact lens. The processing module is configured to generate control signals based on said sensor's data reflecting changes over time in the voltage difference between its electrodes.

Description

AUGMENTED REALITY INTERFACE CONTROL SYSTEM AND METHOD

[0001] The group of inventions relates to the systems and methods of user-to-computer interaction and, in particular, to the systems and methods of augmented reality (AR) interface control using bioelectric signals (in particular, tongue movements or steady state visually evoked potentials) detected by means of contact lenses.

[0002] The prior art discloses a contact electrode with a gaze direction tracking system, wherein said electrode comprises a contact lens with a visual electrophysiological contact electrode integrated therein and being in contact with the user's eye cornea (see publication CN210055982U, cl. A61B3 / 113, A61B5 / 0496, published 2020-02-14). In the prior art device, while said electrode is measuring the electrophysiological parameters of the cornea, a gaze tracking system is used to detect the eyeball movement. The main disadvantages of the prior art solution are its limited functionality (can only be used for passive signal detecting solely by a contact method) and impossibility to form a fully functional user-to-computer interaction. Furthermore, the annular electrode is made of a nontransparent material (that hampers visibility) and only reads an averaged electric signal from the cornea (which is less informative).

[0003] The prior art discloses a device based on a contact lens with an electrode array for simultaneous detection of electric potentials in several locations on the eye surface, wherein said device comprises a dielectric substrate with a concave inner surface and at least five detecting electrodes configured to be electrically connected with the anterior surface of the eye (see publication US2008294066A1, cl. A61B5 / 05, published 2008-11-27). Each detecting electrode is in electrically conductive communication with its corresponding isolated conductive contact to quickly connect to a signal processor that analyses the detected electrophysiological potentials reflecting the spatial distribution of the retina activity. The disadvantages of the prior art solution are its limited functionality making it unusable for user interface control, as well as the complexity of manufacture and the necessity to use a differential amplifier connected to a ground electrode on the forehead.

[0004] The prior art discloses a system and method for obtaining information about the spatial distribution of photoreceptor activity and neural activity in the retina using simultaneously detected multiple bioelectric signals (see publication US2007188710A1, cl. A61B3 / 10; published 2007-08-16). The information thus gathered is used to assess retinal dysfunction due to trauma or disease.   The bioelectric signals are detected from the surface of the user’s eye and head using a plurality of electrodes (including those integral to a contact lens) and are recorded before, during and after the application of an optical stimulus. The analysis and interpretation of the biopotential signals are quantitative and make use of an electromagnetic model of the user’s eye. The disadvantages of the prior art solution are as well its limited functionality making it unusable for user interface control, and impossibility to combine with wearable augmented reality devices.

[0005] The closest, in terms of technical substance, to the claimed invention is the augmented reality interface control system comprising a contact lens, a projecting module for projecting an augmented reality interface into the user's eye, a sensor configured to detect the user's bioelectric signals, a processing module for processing said bioelectric signals that is configured to generate the control signals, and a control module configured to modify the augmented reality interface in accordance with the control signals received from said processing module (see patent US10990175B2, cl. G02B27 / 01, G06F3 / 01, G06T19 / 00, A61B5 / 316, A61B5 / 378, A61B5 / 38, published 2021-04-27). This document also discloses an augmented reality interface control method consisting in that a contact lens or glasses are put onto the user's eye, an augmented reality interface is projected into the user's eye, user-generated bioelectric signals are detected, the detected bioelectric signals are processed and used as a basis to generate the control signals, and the augmented reality interface is modified in accordance with said control signals. The prior art system is implemented in a headset including an augmented reality display, a battery, and a printed circuit board that has the sensors to read bioelectric signals from the back of the user’s head. In the prior art solution, the output data provides a feedback by stimulating several sensory systems of the user's brain, including audio and visual on the augmented reality display, or audio and haptic in terms of vibration patterns that a user may feel. All together this forms a closed-loop system: the detected bioelectric signal initiates a feedback, which in turn enhances the biosignal. The main disadvantages of the prior art solution are bulkiness of the wearable device and the necessity to simultaneously stimulate several sensory systems of the user's brain.

[0006] The technical problem is to eliminate said disadvantages and to create a simple and user-friendly augmented reality interface control system.

[0007] The technical effect consists in enhancing the compactness of the wearable part of the control system.

[0008] As related to the device, the set problem has been solved and the technical effect has been achieved by that in the augmented reality interface control system, comprising at least one contact lens, a projecting module for projecting the augmented reality interface into the user's eye, a sensor configured to detect the user's bioelectric signals, a processing module for processing said bioelectric signals configured to generate the control signals, and a control module configured to modify the augmented reality interface in accordance with the control signals received from said processing module, said sensor comprises at least two primary electrodes integrated into said contact lens, and the bioelectric signal processing module is configured to generate the control signals based on this sensor's data reflecting changes over time in the voltage difference between its electrodes. The processing module can be integrated into said contact lens or disposed in a companion device configured to receive the sensor's data via wireless communication. Said companion device can be equipped with secondary electrodes configured to independently detect the user’s bioelectric signals. The projecting module can be disposed inside said companion device as well or integrated into said contact lens. The sensor can be equipped with a reference electrode and a ground electrode that are different from the primary electrodes. The reference electrode and / or ground electrode can be implemented as an annular conductor integrated into said contact lens, or mounted on the user's skin and electrically connected to the primary electrodes via a conductive structure. Said conductive structure preferably comprises a conductive tattoo made on the inside of the user’s eyelid and a mating contact area implemented on the contact lens outer surface to form electric contact with said conductive tattoo. The primary electrodes can be disposed on the contact lens inner surface to form electric contact with the user's eye and can be equipped with a conductive cover integrated into said contact lens to provide a shield against external electromagnetic field. Alternatively, the primary electrodes can be implemented as contactless capacitive electrodes. The primary electrodes are preferably made transparent in the visible range of the spectrum and formed from a van der Waals material consisting of two-dimensional layers bonded with each other by van der Waals forces.

[0009] As related to the method in accordance with the first embodiment, the set problem has been solved and the technical effect has been achieved by that in accordance with the augmented reality interface control method consisting in that at least one contact lens is put onto the user’s eye, an augmented reality interface is projected into the user's eye, the user-generated bioelectric signals are detected, the detected bioelectric signals are processed and used as a basis to generate the control signals, and the augmented reality interface is modified in accordance with said control signals, the bioelectric signals are detected using a sensor comprising at least two primary electrodes integrated into said contact lens, and the detected user’s bioelectric signals are processed to detect changes over time in the voltage difference between the electrodes of said sensor, the frequency signals corresponding to the tongue movement are isolated and used as a basis to generate the control signals. While projecting the augmented reality interface into the user's eye, interface items are formed spatially distributed over the field of view, and the control signals are generated in such a way that the selection of the respective interface item is defined by the tongue movement in the mouth cavity in that same direction in which the selected interface item is located.

[0010] As related to the method in accordance with the second embodiment, the set problem has been solved and the technical effect has been achieved by that in accordance with the augmented reality interface control method consisting in that at least one contact lens is put onto the user’s eye, an augmented reality interface is projected into the user's eye, the user-generated bioelectric signals are detected, the detected bioelectric signals are processed and used as a basis to generate the control signals, and the augmented reality interface is modified in accordance with said control signals, while projecting the augmented reality interface into the user's eye, interface items are formed spatially distributed over the field of view and flickering with various frequencies, the bioelectric signals are detected using a sensor comprising at least two primary electrodes integrated into said contact lens, and the detected user’s bioelectric signals are processed to detect changes over time in the voltage difference between the electrodes of said sensor, the frequency signals corresponding to the interface item flickering frequency are isolated and used as a basis to generate the control signals. Preferably, when the bioelectric signals with the flickering frequency of one of the interface items are dominating, the selection control signals are generated corresponding to selection of that interface item, and, while the augmented reality interface is changing in accordance with said selection control signals, the glowing intensity and / or form of the selected interface item are changed.Fig.1

[0011] is a schematic diagram of the suggested control system with maximum integration into a contact lens;Fig.2

[0012] is a schematic diagram of the suggested control system with maximum integration into a companion device;Fig.3

[0013] is a system operation diagram according to the first embodiment of the suggested control method using a companion device;Fig.4

[0014] is an embodiment of conductive tattoos in the periocular area;Fig.5

[0015] is a diagram of possible tongue movements when the augmented reality interface is controlled according to the first embodiment of the suggested method;Fig.6

[0016] is a mismatch matrix for the experimental implementation of the first embodiment of the suggested control method when isolating a slow-wave component and using a CSP+LDA filter;Fig.7

[0017] is the same as, but when using a XDAWN filter;Fig.8

[0018] shows the average signal-to-noise ratio for all the channels for the experimental implementation of the second embodiment of the suggested control method when performing a cognitive selection of the “arrow up” interface item flickering with frequency of 17 Hz;Fig.9

[0019] is the same as, but when performing a cognitive selection of the “arrow down” interface item flickering with frequency of 19 Hz.

[0020] The suggested augmented reality interface control system-is based on the bioelectric signal sensor 1 with the primary electrodes 2 integrated into the contact lens 3, and (optionally) the secondary electrodes 4 that are configured to independently detect the user’s bioelectric signals and can be mounted on the user's body. The primary electrodes 2 are preferably disposed on the contact lens 3 inner surface to form electric contact with the user's eye. To prevent the effect of external noise (surrounding electromagnetic fields that are not attributed to the user’s bioelectric signals) on the detecting results, the primary electrodes 2, on the side opposite to the eye, are equipped with a conductive cover integrated into said contact lens 3 and operating based on the Faraday cage principle. Alternatively, the primary electrodes 2 can be implemented as contactless capacitive electrodes disposed on the outer surface and fully integrated into the contact lens 3.

[0021] In principle, for the system to be operational, two primary electrodes 2 are needed (to make it possible to determine the voltage difference between them), however, to obtain more comprehensive data for further processing, it is practical to form from 4 to 12 electrodes along the whole perimeter of the lens 3. The system can be connected to both one contact lens 3 and two contact lenses put onto the user's both eyes.

[0022] The contact lens 3 can be a rigid scleral lens, soft hydrogel lens, combined (hybrid) lens, or any other structure of polymer or another material adapted to be fixed on the user's cornea. To enable comfortable use of such contact lenses 3 in everyday life, the primary electrodes 2 are made of a transparent material. It is preferable to use thin layers of metal (for example, gold or copper) or van der Waals materials consisting of two-dimensional layers bonded with each other by van der Waals forces, in particular: graphene, black arsenic, black phosphorus, Cd3As2, Cd3Sb2, Cr2AlC, Cr2C, Mn2AlC, Mo2C, Mo2Ga2C, Mo3AlC2, Nb2AlC, Nb2C, Nb4AlC3, Nb4C3, Ta2C, Ta4AlC3, Ti2AlC, Ti2AlN, Ti2C, Ti2N, Ti3AlC2, Ti3C2, Ti3CN, Ti3SiC2, Ti4N3, V2AlC, V2C, V4AlC3, V4C3, Sb2Te3, PdSe2, PtS2, PtSe2, Sb2Se3, Sb2S3, As2Te3, Bi2Se3, Bi2TeO2, BiSbTe3, Bi2Te3, or AsP. Such materials, formed as a layer having the thickness of approximately several tens of nanometers (less than 100 nm), have sufficiently good conductivity and transparency in the optical band (including the visible range of the spectrum).

[0023] To interact with the augmented reality interface, the control system also should include the module 5 for projecting the augmented reality interface into the user's eye, the module 6 for processing the bioelectric signals, and the control module 7. For maximum compactness, all these modules 5, 6, 7 are integrated directly into the body of the contact lens 3. Herewith, the projecting module 5 is implemented as a microdisplay with an optical system forming a quasi-collimated beam directed to the crystalline lense of the user’s eye, essentially, along its optical axis.

[0024] However, to simplify the design and make it possible to use powerful computing resources, the processing module 6can be disposed externally in the companion device 8 or can represent an external software – hardware package.

[0025] Likewise, the modules 5, 7 can be disposed externally in the companion device 8-, which makes it possible to use larger electronic components and to distribute their weight comfortably. Herewith, the projecting module 5 is implemented as a common microprocessor and the contact lens 3 is equipped with an optical diffraction element (for example, a hologram, a diffraction grating, or another optical structure, including one based on a metasurface) turning the emission from such external projector into the user's eye. The companion device 8 can represent glasses, a headband, a hoop, etc., and can be installed, for example, on the user's head or neck. To interact with the primary electrodes 2 disposed in the contact lens 3, the companion device 8 and the sensor 1 are configured to exchange data via wireless communication (equipped with respective transceivers), and, additionally, cloud-based data storage systems can be utilized in said interaction. The contact lens 3 can also comprise an integrated transceiving antenna, a power source (for example, a rechargeable battery), and an ASIC-type control microchip (not shown in the drawings).

[0026] The processing module 6 is connected to the sensor 1 via a wire or wireless means and converts the detected bioelectric signals into control signals. For this purpose, the module 6 is configured to generate control signals based on the sensor’s 1 data reflecting changes over time in the voltage difference between its electrodes. The voltage difference can be measured directly between the primary electrodes 2 in the contact lens 3 or between the primary electrodes 2 and the secondary electrodes 4.

[0027] To detect bioelectric signals in the form of brain waves, the standard setup makes use of active electrodes, a reference electrode, and a ground electrode. Herewith, as the target signal, the sensor 1 detects changes over time in the voltage difference between the primary electrode 2 and the reference electrode, while the ground electrode is used to account for external electromagnetic field.

[0028] The most simple in terms of minimizing the components is the embodiment in which the reference electrode and / or ground electrode is represented by virtual elements with the potential thereat representing a design value obtained by mathematical processing of the electric potential of all real primary electrodes 2.

[0029] However, to enhance accuracy, it is possible to use a real reference electrode and a ground electrode that are different from the primary electrodes 2. In this case, the reference electrode and / or the ground electrode are implemented as a thicker annular conductor 9 (for example, a 1-10 µm thick copper wire) integrated into said contact lens 3 along its periphery around the transparent primary electrodes 2.

[0030] Alternatively-, the reference electrode can be implemented as one or more conductive stickers 10 that comprise the secondary electrode 4 connected with the companion device 8, and are fixed on the skin above the eyebrow, in the eye corners, on the movable / immovable eyelid, or on another part of the user's head. In this case, the electric contact with the primary electrodes 2 can be implemented as a connection via a conductive structure that entirely or partially represents a conductive tattoo 11. The distal part of such conductive tattoo is made on the inside of the user's upper or lower eyelid, and a mating contact area is formed on the contact lens 3 outer surface to enable electric contact between the primary electrodes 2 and said conductive tattoo and reference electrode. Herewith, the ground electrode can be implemented as a common electroencephalography (EEG) secondary electrode 4 fixedly attached on the companion device 8 and being in contact, for example, with the preocular or back part of the user's head.

[0031] The control signals generated by the processing module 6 based on the sensor’s 1 data are sent to the control module 7 that forms the target image of the augmented reality interface and sends it to the projecting module 5. The new image of the interface is perceived by the user, the user forms a feedback choice reaction, and this reaction generates new bioelectric signals. The sensor 1 again detects the temporal changes of this signal, sends it to the processing module 6, this module generates the control signals for the control module 7 which, in turn, modifies the augmented reality interface, and the interaction cycle starts over again.

[0032] Thereby, the disclosed device, with the wearable part being as compact as possible, can be used to form an efficient interface control system for augmented reality (AR) as well as for virtual reality (when used in combination with wearable VR devices). Herewith, the possibility to establish efficient biological feedback channels and neurointerfaces depends on the variety of different parameters detected from the human eye. The bioelectric signals that are used can be represented by any user-generated electric signals including those based solely on the brain activity (for example, cognitive changes), other neuronal (for example, visually evoked electric potentials) and / or muscular activity (activity of facial or jaw muscles; squinting, other movements of periocular area, eyelids, tongue, eyebrows, nostrils, lips, ears, forehead, etc.; combination of inhalation, exhalation, and breath holding, biting of soft tissues of the mouth cavity; forming sounds, etc.), and any combination thereof. According to one of the promising applications of the combined bioelectric signal, the processing module 6 can be calibrated to generate control signals converted from speech (including subvocal speech): in this case, the sensor detects the complex signals resulting from the movements of the user's lips and tongue, as well as the movements of muscles of mastication and throat muscles. Such application appears to be the most intuitively understandable, but requires a long-lasting data collection and processing to realize sufficiently accurate calibration for a specific user.

[0033] Below described are the embodiments of the augmented reality interface control methods that appear to be promising, but do not limit the scope of claim as related to the device.

[0034] The suggested augmented reality interface control method according to the first embodiment consists in that the bioelectric signal, detected by the above-described sensor 1 as changes over time in the voltage difference, is used to isolate from it the frequency signals resulting from the movements of the user's tongue 12, specifically high-frequency and / or low-frequency signals.

[0035] To implement the suggested method, the contact lens 3 with integrated primary electrodes 2 is put onto the user's eye, an augmented reality interfaceis projected into the user's eye, the user-generated bioelectric signals are detected using the sensor 1, the detected bioelectric signals are processed and used as a basis to generate the control signals, following which the augmented reality interface is modified in accordance with said control signals. To establish an intuitive control strategy, while projecting the augmented reality interface into the user's eye, interface items 13 are formed spatially distributed over the field of view, and the control signals are generated in such a way that the selection of the respective interface item 13 is defined by the tongue 12 movement in the mouth cavity in that same direction in which the selected interface item is located relative to the central direction of the user's gaze.andvisually represent the image with open mouth, however in real use this is not necessary, and the mouth can be closed.

[0036] As an experimental embodiment, the bioelectric signals recorded in the periocular area during the movements of the tongue 12 were processed as the event-related potentials (ERP) and represented the measured brain response being the direct result of the motor event. That is, the user executed the commands to select a particular interface item 13 (“left”, “right”, “up”, “down”) by moving the tongue 12 in the mouth cavity in that same direction, the processing module 6 converted the movements into a control command, following which the correctness of the selection identification was analyzed. The analysis of the detected bioelectric signals representing changes over time in the voltage difference was carried out in the range from 0.5 to 7 Hz, which corresponds to a slow wave (low frequency) component of the glossokinetic potential. The slow wave activity during the movement of the tongue 12 occurs as a result of the movement of the dipole formed by the tongue tip which has a positive charge relative to the tongue root. The possibility to use tongue movement-based classifier in the augmented reality interface control system is demonstrated by the mismatch matrix calculations for various combinations of spatial filters and classifiers-. Herewith, the obtained results belong to the standard, parameter-unoptimized classifiers and have significant potential for improvement (the resulting accuracy of such potential can exceed 95%).

[0037] Forming a classifier based on the tongue movement in the mouth cavity in that same direction in which the selected interface item is located is the most intuitively understandable, however other embodiments are also potentially possible: inverted commands, trajectory commands (the form of the interface item can define the activating trajectory of the tongue movement), etc.

[0038] The suggested augmented reality interface control method according to the second embodiment consists in using the bioelectric signal (detected by the above described sensor 1 as changes over time in the voltage difference) to isolate from it the frequency signals corresponding to the flickering frequency of the interface items 13, i. e. the evoked potentials. That is, in the suggested method, the rhythmic activity signals are detected from the primary electrodes 2 of the contact lens, and not by the occipital electrodes.

[0039] Detecting the evoked potentials of the brain is a well-developed objective and noninvasive method of testing the central nervous system functions of both humans and animals. Using high-sensitivity amplifiers and digital devices makes it possible to isolate weak signals, which are 5-100 times weaker than a common spontaneous brain activity (EEG), electromyogram (EMG) and other bioelectric signals, by averaging a large number of weak brain responses. The stimuli in the form of sinusoidally modulated monochromatic light pulses evoke low amplitude evoked potentials referred to as the steady state visually evoked potentials (SSVEP). The constituting discrete frequency components of such signals remain almost constant in amplitude and phase during a long period of time. At the same time, the amplitude of an unprocessed EEG in the time domain can vary, while the amplitude distribution of the SSVEP spectral content with indicative peaks remains stable in time. The SSVEPs are well resistant to recording interferences, such as blinking, eye movements, and myographic artifacts. The SSVEP spectrum peaks can be amplified using dedicated filters and used to form a control signal in various devices. The SSVEP occurs mainly in the range from 3 to 50 Hz, but can reach the magnitudes exceeding 80 Hz.

[0040] Coherent analysis of EEG is a method of mathematical processing aimed to estimate commonality (similarity) of the spectral content between two selected leads, i. e. reflects the degree of similarity of the compared EEG in the frequency domain. Coherence gives information about stability of correlation, estimates the statistical relation between the respective frequency components of two processes, and has high sensitivity. Quantitatively, the degree of similarity of spectra can be expressed via a dimensionless parameter of similarity – coherence factor (coherence).

[0041] Mathematically, coherence can be calculated both as the complex degree of coherence and degree of coherence.

[0042]

[0043] where Cxy(f) is the complex degree of coherence, C2xy(f) is the squared degree of coherence, Sxy(f) is the cross-spectrum value at the given frequency, Sxx(f) and Syy(f) are the values of the auto-spectra of the signals х and у at the same frequency.

[0044] Using the projecting module 5 built into the contact lens 3 implies displacement of all of the interface items during the eye movement, which is why the control using a simple change in gaze direction appears to be not much promising. To overcome this complication, the stimulus that should induce occurrence of the evoked potentials in the suggested method is represented by the interface items 13 flickering with predetermined various frequencies (during the experimental implementation, initially the following frequencies were used: f1=5 Hz, f2=15 Hz, f3=20 Hz, f4=35 Hz, and thereafter, accordingly, 11 Hz, 13 Hz, 17 Hz, and 19 Hz). In this case, the interface items 13 are spaced from the central optical axis towards the peripheral part of the field of view (during the experimental implementation, up, right, down, and left displacements were used). Additionally, the interface items can have different forms and colors to help their cognitive identification.

[0045] To select a particular item 13, the user makes a cognitive effort consisting in focusing (without changing the direction of gaze) on one of the interface items 13. Herewith, the bioelectric signals with the flickering frequency of the selected interface item 13 become dominating (for example, in amplitude or degree of stability) over the others. Based on detecting such domination, the processing module 6 forms the control selection signals corresponding to selection of said interface item 13 and sends them to the control module 7. To help interaction and reduce the risk of false activation, once the selection has been made the control module 7 changes the glow intensity and / or form of the selected interface item 13, after which the sensor 1 can read an additional bioelectric signal corresponding to the confirmation operation of the selection. In case there is a stable calibration, switching to a new interface (for example, another menu) corresponding to the selected item 13 is performed right after its selection.

[0046] To implement the suggested method, the contact lens 3 with integrated primary electrodes 2 is put onto the user's eye, the augmented reality interface with the interface items 13 (spatially distributed over the field of view and flickering with various frequencies) is projected into the user's eye, the user-generated bioelectric signals are detected using the sensor 1, the detected bioelectric signals are processed and used as a basis to generate the control signals, following which the augmented reality interface is modified in accordance with said control signals.

[0047] To generate the control signals, the processing module 6 was configured to perform preliminary spectral analysis of the received bioelectric signal, which consisted in decomposing the signal into its frequency and spectral components and assessing their characteristics, such as amplitude, phase, power, power spectral density, etc. Considering that the pacing can occur not only at the stimulation frequency, but also at multiple harmonics, carrying out a multistage mathematical processing (using fast Fourier transform and deriving the signal-to-noise ratio) is the mandatory requirement when using multiple stimulus frequencies (interface items flickering). The performed experimental implementation has demonstrated that identification of cognitive selection was possible via the respective processing of the detected bioelectric signal in the module 6 (see-). The most accurate control signals can be obtained when using the interface item flickering frequencies from 10 to 20 Hz, said range being close to the eye resolution (24 frames per second). And also, to avoid occurrence of harmonic waves, the frequencies should be selected from those that are not multiples of each other, ideally as prime numbers.

Claims

An augmented reality interface control system comprising:at least one contact lens,a projecting module for projecting the augmented reality interface into the user's eye,a sensor configured to detect the user's bioelectric signals,a processing module for processing said bioelectric signals configured to generate control signals, anda control module configured to modify the augmented reality interface in accordance with the control signals received from said processing module,characterized in thatsaid sensor comprises at least two primary electrodes integrated into said contact lens,and the bioelectric signal processing module is configured to generate control signals based on said sensor’s data reflecting changes over time in the voltage difference between its electrodes.The control system according to claim 1, characterized in that the processing module is integrated into said contact lens.The control system according to claim 1, characterized in that the processing module is disposed in a companion device configured to receive the sensor’s data via wireless communication.The control system according to claim 3, characterized in that said companion device is equipped with secondary electrodes configured to independently detect the user’s bioelectric signals.The control system according to claim 3, characterized in that the projecting module is disposed inside said companion device.The control system according to claim 1, characterized in that the projecting module is integrated into said contact lens.The control system according to claim 1, characterized in that said sensor is equipped with a reference electrode and a ground electrode that are different from the primary electrodes.The control system according to claim 7, characterized in that the reference electrode and / or ground electrode are implemented as an annular conductor integrated into said contact lens.The control system according to claim 7, characterized in that the reference electrode and / or ground electrode are mounted on the user's skin and electrically connected to the primary electrodes via a conductive structure.The control system according to claim 9, characterized in that said conductive structure comprises a conductive tattoo made on the inside of the user’s eyelid and a mating contact area implemented on the contact lens outer surface to form electric contact with said conductive tattoo.The control system according to claim 1, characterized in that the primary electrodes are disposed on the contact lens inner surface to form electric contact with the user's eye and equipped with a conductive cover integrated into said contact lens to provide a shield against external electromagnetic field.The control system according to claim 1, characterized in that the primary electrodes are implemented as contactless capacitive electrodes.The control system according to claim 1, characterized in that the primary electrodes are made transparent in the visible range of the spectrum and formed from a van der Waals material consisting of two-dimensional layers bonded with each other by van der Waals forces.An augmented reality interface control method consisting in thatat least one contact lens is put onto the user’s eye,an augmented reality interface is projected into the user's eye,the user-generated bioelectric signals are detected,the detected bioelectric signals are processed and used as a basis to generate the control signals, andthe augmented reality interface is modified in accordance with said control signals,characterized in thatthe bioelectric signals are detected using a sensor comprising at least two primary electrodes integrated into said contact lens, andthe detected user’s bioelectric signals are processed to detect changes over time in the voltage difference between the electrodes of said sensor, the frequency signals corresponding to the tongue movement are isolated and used as a basis to generate the control signals.The control method according to claim 14, characterized in that, while projecting the augmented reality interface into the user's eye, interface items are formed spatially distributed over the field of view, and the control signals are generated in such a way that the selection of the respective interface item is defined by the tongue movement in the mouth cavity in that same direction in which the selected interface item is located.An augmented reality interface control method consisting in thatat least one contact lens is put onto the user’s eye,an augmented reality interface is projected into the user's eye,the user-generated bioelectric signals are detected,the detected bioelectric signals are processed and used as a basis to generate the control signals, andthe augmented reality interface is modified in accordance with said control signals,characterized in thatwhile projecting the augmented reality interface into the user's eye, interface items are formed spatially distributed over the field of view and flickering with various frequencies,the bioelectric signals are detected using a sensor comprising at least two primary electrodes integrated into said contact lens, andthe detected user’s bioelectric signals are processed to detect changes over time in the voltage difference between the electrodes of said sensor, the frequency signals corresponding to the interface item flickering frequencies are isolated and used as a basis to generate the control signals.The control method according to claim 16, characterized in that, when the bioelectric signals with the flickering frequency of one of the interface items are dominating, the selection control signals are generated corresponding to selection of that interface item, and, while the augmented reality interface is changing in accordance with said selection control signals, the glowing intensity and / or form of the selected interface item are changed.

Images