Eye motion state recognition system based on transparent reconfigurable lens integrated antenna

Abstract

The invention provides an eye motion state recognition system based on a transparent reconfigurable lens integrated antenna, which comprises a transparent flexible reconfigurable antenna, a data acquisition module and a data processing and recognition module, and is characterized in that the transparent flexible reconfigurable antenna comprises a radiator, a feeder line, a director, a grounding plate and a switching diode; the two electrodes are arranged on the same surface of the transparent flexible medium substrate; the data acquisition module acquires reflection coefficient or transmission coefficient data of the transparent flexible reconstruction antenna during eye movement; and the data processing and identification module comprises two working modes and is used for outputting an identification result of the eye movement state. The transparent flexible reconstruction antenna is made of a transparent conductive material, can dynamically adjust the radiation direction, does not occupy extra space when being integrated on the lens, does not block the sight line, and achieves the light, flexible and portable recognition of the eye movement state.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of radio waves, in particular to an eye movement state recognition system based on a transparent reconfigurable lens integrated antenna. BACKGROUND

[0002] Wireless Body Area Network (WBAN) is a communication network with the human body as the center, composed of network elements related to the human body, including personal terminals, sensors distributed on the human body, clothes, within a range of 2 meters around the human body, even inside the human body, networking equipment. At present, WBAN is widely used and plays an important role in medical, military, entertainment and sports fields, especially in the medical field. Through the sensor nodes on the human body, the human health signals can be monitored and recorded for a long time, and the data can be accurately presented to medical staff and patients in real time, assisting medical staff to implement medical means.

[0003] With the development of WBAN, transparent and flexible antenna integration into wearable devices has become a research hotspot. Wearable antennas can achieve portable, flexible, low-cost and portable wireless communication and sensing, which is a key element in designing wireless wearable devices, and can be compatible with devices of different shapes, sizes and materials, creating additional space for other components to achieve device miniaturization.

[0004] Existing transparent antenna research focuses on windows, car windshields and mobile wireless communication fields. Some research proposes to integrate transparent and flexible antennas on glasses for communication, or integrate them on lenses close to the eyes of glaucoma patients to measure intraocular pressure. However, the existing technology has the following problems: 1. Lack of eye state monitoring solutions that can be integrated into lenses without occupying space and blocking vision; 2. Existing camera-based eye recognition solutions are bulky, costly and not portable; 3. Existing glasses integrated antennas are only used for communication and cannot achieve eye movement state sensing and monitoring.

[0005] Through the retrieval of patent documents, it is found that the patent for an invention with publication number CN106506823A discloses an intelligent wearable device, system, method and device for controlling terminal equipment, which comprises: an eye movement acquisition antenna for acquiring multiple electromagnetic wave signals generated during eye movement; a characteristic information extraction circuit for extracting characteristic information of the multiple electromagnetic wave signals; an eye movement recognition circuit for importing the characteristic information into a pre-trained motion recognition model for matching to obtain an eye movement pattern, wherein the motion recognition model is used to represent the corresponding relationship between the characteristic information and the eye movement pattern; and a recognition result sending antenna for sending the eye movement pattern to a pre-connected terminal equipment for the terminal equipment to perform an operation related to the eye movement pattern. The embodiment realizes more convenient control of the terminal equipment. The patent only focuses on terminal control, the main acquisition antenna is arranged on the spectacle frame of the frame glasses, the auxiliary acquisition antenna is arranged on the temple and / or the nose pad in contact with the skin of the user, and the problems of large volume and high cost of the camera type scheme are not solved, and the signal fidelity, acquisition integrity and scene adaptability are insufficient.

[0006] In summary, in view of the problems of the prior art, eye state recognition and blink detection are in urgent need in the fields of sleepiness detection, facial expression recognition, driver fatigue monitoring, psychological state analysis, human-computer interaction technology and the like, and research on an eye movement state recognition system based on a transparent reconfigurable lens integrated antenna becomes a key task to be solved at present. SUMMARY

[0007] In view of the defects in the prior art, the purpose of the present application is to provide an eye movement state recognition system based on a transparent reconfigurable lens integrated antenna.

[0008] According to the eye movement state recognition system based on the transparent reconfigurable lens integrated antenna provided by the present application, the following are included: The transparent flexible reconfigurable antenna adopts a coplanar waveguide structure, and comprises a radiator, a feed line, a director, a ground plate and a switch diode. The radiator, the feed line, the director, the ground plate and the switch diode are all arranged on the same surface of a transparent flexible dielectric substrate. The transparent flexible reconfigurable antenna is made of a transparent conductive material, and the radiation direction of the radiator is dynamically adjusted by controlling the on-off of the switch diode. The data acquisition module is used to acquire the reflection coefficient data of the transparent flexible reconfigurable antenna changed due to eye movement after the lens integrated with the transparent flexible reconfigurable antenna is worn on the human body. The data processing and recognition module includes two working modes: In the first mode, the reflection coefficient data is filtered, denoised and spectrum analyzed, and the time domain features and frequency domain features representing the eye movement are extracted based on the analysis results, and the recognition result of the eye movement state is output. In the second mode, the reflection coefficient data is filtered and denoised, and a short-time Fourier transform is performed to generate a time-frequency spectrum containing the time-frequency characteristics of eye movement, and a pre-trained deep learning model is used to classify the time-frequency spectrum to output the recognition result of the eye movement state.

[0009] Preferably, the radiator is a planar vibrator, and the planar vibrator has a shape of a rectangle, a square, a trapezoid, a triangle, a circle, an ellipse, a polygon, or a fractal structure; the radiator is connected to the signal source through a transmission line, and the transmission line is in the form of a microstrip line, a coplanar waveguide, or a parallel double line.

[0010] Preferably, the transparent flexible reconfigurable antenna is a rectangular monopole antenna, and the transparent conductive film is prepared on the transparent flexible dielectric substrate; the antenna structure formed on the transparent conductive film includes a radiator, a feed line, and two ground plates respectively located on the left and right sides of the feed line; the radiator is connected to the feed line through a trapezoidal transition module; the trapezoidal transition module serves as a gradual change structure and is used for realizing wideband impedance matching; on the left and right sides of the radiator, one parasitic rectangular vibrator as a director is arranged respectively; a gap is formed in the middle of each director; a switching diode is connected across the gap; by controlling the on-off state of the switching diode, the electrical length of the director is changed, so that the radiation direction of the radiator is dynamically adjusted, and the antenna works in different modes.

[0011] Preferably, the radiator, the feed line, the directors, and the ground plates are all made of transparent conductive materials and are printed on the transparent flexible dielectric substrate; the transparent conductive materials include indium tin oxide, fluorine-doped tin oxide, tungsten-doped indium oxide, aluminum-doped zinc oxide, silver metal mesh, or copper metal mesh; the thickness of the transparent conductive materials is less than 1 μm; the surface resistance of the transparent conductive materials is less than 3 ohms per unit area; and the light transmittance of the transparent conductive materials is greater than 70%.

[0012] Preferably, the material of the transparent flexible dielectric substrate includes polyvinyl naphthalene, polydimethylsiloxane, polyimide, or polyethylene terephthalate.

[0013] Preferably, in the data acquisition module, the transparent flexible reconfigurable antenna is integrated on the lens in the following ways: directly printed on the surface of the lens or first printed on a transparent flexible film and then attached to the surface of the lens; and the position of the transparent flexible reconfigurable antenna is directly opposite the center of the eye.

[0014] Preferably, the data acquisition module constructs a data acquisition link, the data acquisition link starts from the transparent flexible reconfigurable antenna integrated on the lens, the feed line of the transparent flexible reconfigurable antenna is connected to the conductive wire of a coaxial cable through conductive silver paste, the two ground plates of the transparent flexible reconfigurable antenna are connected to the ground wire of the coaxial cable through conductive silver paste, the coaxial cable is led out from the inside of the frame and the temple, the other end of the coaxial cable is connected to a signal transceiver device through an SMA joint, and the data acquisition module acquires and records the antenna reflection coefficient data through the signal transceiver device.

[0015] Preferably, in the data processing and identification module, the wavelet filtering, the moving average filtering or the Kalman filtering is used for filtering and denoising the reflection coefficient data.

[0016] Preferably, in the first mode of the data processing and identification module, the time domain features include the open eye duration, the closed eye duration and the blink frequency, and the frequency domain features include the blink dominant frequency. In the second mode of the data processing and identification module, the time-frequency spectrum includes the energy distribution and the instantaneous frequency change in different states.

[0017] Preferably, in the second mode of the data processing and identification module, the deep learning model uses a Vgg16 deep neural network model, uses a multi-layer small-size convolution kernel stack, increases the network depth through repeated modular structures, and trains the Vgg16 deep neural network model using the time-frequency spectrum as a training set until convergence, so as to obtain the pre-trained deep learning model.

[0018] Compared with the prior art, the present application has the following beneficial effects: 1. The transparent flexible reconfigurable antenna of the present application is made of transparent conductive material, and has a light transmittance of greater than 70%. When integrated into the lens, it does not occupy additional space and does not block the line of sight, and can monitor the eye movement state at any time and anywhere without additional components.

[0019] 2. Unlike existing eye recognition using a camera, the present application collects reflection coefficient or transmission coefficient data through the transparent flexible reconfigurable antenna, achieving portable, flexible and low-cost wireless sensing. 3. Unlike existing antennas integrated into glasses for only wireless communication, the present application integrates a transparent, flexible and reconfigurable transparent flexible reconfigurable antenna into the lens, which is specifically used for sensing and monitoring the human eye movement state, and can dynamically adjust the radiation direction by controlling the on-off of the switch diode in the transparent flexible reconfigurable antenna, adapting to different use scene requirements. BRIEF DESCRIPTION OF DRAWINGS

[0020] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings: Figure 1 is a schematic diagram of an eye movement state recognition system based on a transparent reconfigurable lens integrated antenna in an embodiment of the present application; Figure 2 is a shape and size diagram of a transparent flexible antenna in an embodiment of the present application; Figure 3 is an equivalent circuit diagram of a switch diode in a cut-off state and an on state in an embodiment of the present application; Figure 4is a size parameter of the transparent flexible reconfigurable antenna structure in the embodiment of the present application; Figure 5 is a simulation value of the reflection coefficient parameter of the transparent flexible reconfigurable antenna in the embodiment of the present application; Figure 6 is a simulation value of the transparent flexible reconfigurable antenna in the diode off state in the embodiment of the present application; Figure 7 is a simulation value of the transparent flexible reconfigurable antenna in the diode on state in the embodiment of the present application; Figure 8a is an overall experimental device of the transparent flexible reconfigurable antenna integrated on the lens in the embodiment of the present application; Figure 8b is a schematic diagram of the transparent flexible reconfigurable antenna connected with the coaxial cable in the embodiment of the present application; Figure 9 is an experimental measurement of 8 groups of antenna reflection coefficient amplitude graphs in the embodiment of the present application; Figure 10 is an experimental measurement of 8 groups of antenna reflection coefficient phase graphs in the embodiment of the present application; Figure 11 is a 25-second antenna reflection coefficient amplitude sequence of a uniform blink once per second in the embodiment of the present application; Figure 12 is a result of the antenna reflection coefficient amplitude data of Figure 11 in the embodiment of the present application after Fourier transform (FFT); Figure 13 is a time-frequency spectrum of two eye movements in the embodiment of the present application; Figure 14 is a Vgg16 deep neural network model constructed in the embodiment of the present application. DETAILED DESCRIPTION

[0021] The present application will be described in detail below with specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of changes and improvements can be made. These all belong to the protection scope of the present application.

[0022] This invention discloses an eye movement state recognition system based on a transparent reconfigurable lens integrated antenna. The system includes a transparent flexible reconfigurable antenna, a data acquisition module, and a data processing and recognition module, which are connected sequentially. The transparent flexible reconfigurable antenna includes a radiator, a feed line, a director, a ground plane, and a switching diode, all placed on the same side of a transparent flexible dielectric substrate. The data acquisition module integrates the transparent flexible reconfigurable antenna onto the lens and acquires the reflection coefficient data or transmission coefficient data of the transparent flexible reconfigurable antenna during eye movement. The data processing and recognition module includes two working modes: In the first mode, the reflection coefficient data is filtered, denoised, and subjected to spectral analysis. Based on the analysis results, the time-domain and frequency-domain features characterizing eye movement are extracted, and the recognition result of the eye movement state is output. In the second mode, the reflection coefficient data is filtered, denoised, and subjected to short-time Fourier transform to generate a time-frequency spectrum containing the time-frequency characteristics of eye movement. A pre-trained deep learning model is used to classify the time-frequency spectrum, and the recognition result of the eye movement state is output. The transparent flexible reconfigurable antenna, made of transparent conductive material, can dynamically adjust its radiation direction. Integrated onto a lens, it occupies no extra space and does not obstruct the line of sight, enabling lightweight, flexible, and portable eye movement recognition. This invention can monitor eye status anytime, anywhere, and calculate blink frequency.

[0023] Based on the principle that the difference in dielectric constant between the eyelid and the eyeball during blinking causes the antenna reflection coefficient and transmission coefficient to change with eye movement, this invention proposes to integrate a transparent, flexible, and reconfigurable antenna onto a lens. By monitoring changes in antenna reflection coefficient data or transmission coefficient data, the invention monitors the state of the human eye (open, closed, blinking) and calculates the blinking frequency by performing FFT transformation on the data, and performs deep learning classification on the time-frequency spectrum of the data.

[0024] Example 1: Figure 1 This is a schematic diagram of an eye movement state recognition system based on a transparent reconfigurable lens integrated antenna in an embodiment of the present invention.

[0025] like Figure 1 As shown, this embodiment provides an eye movement state recognition system based on a transparent reconfigurable lens integrated antenna, including: The transparent flexible reconfiguration antenna adopts a coplanar waveguide (CPW) structure. The transparent flexible reconfiguration antenna includes a radiator 1, a feed line 3, a director 5, a ground plane 4, and a switching diode 6. The radiator 1, feed line 3, director 5, ground plane 4, and switching diode 6 are all placed on the same side of the transparent flexible dielectric substrate. The transparent flexible reconfiguration antenna is made of transparent conductive material, and the radiation direction of the radiator is dynamically adjusted by controlling the switching of the switching diode.

[0026] Specifically, the radiator 1 is a planar vibrator, and the planar vibrator has a shape of a rectangle, a square, a trapezoid, a triangle, a circle, an ellipse, a polygon, or a fractal structure; the radiator 1 is connected to a signal source through a transmission line, and the transmission line adopts a form of a microstrip line, a coplanar waveguide, or a parallel double line, and in this embodiment, CPW feeding is adopted.

[0027] Figure 2 is a shape and size diagram of the transparent flexible antenna in the embodiment of the present application.

[0028] As shown in Figure 2 , the transparent flexible reconfigurable antenna is a rectangular monopole antenna, and a transparent conductive film (dark part in the figure, and ITO is adopted in this embodiment) is prepared on a transparent flexible dielectric substrate (light part in the figure, and PET material is adopted in this embodiment), and an antenna structure formed on the transparent conductive film includes the radiator 1, the feed line 3, and the ground plate 4 located on the left and right sides of the feed line 3 respectively. The radiator 1 is connected to the feed line 3 through a trapezoidal transition module 2, and the trapezoidal transition module 2 serves as a gradual change structure and is used for realizing wideband impedance matching. On the left and right sides of the radiator 1, a parasitic rectangular vibrator serving as a director 5 is arranged on each side. A gap is formed in the middle of each director 5, and a switching diode 6 is connected across the gap. By controlling the on-off state of the switching diode 6 to change the electrical length of the director, the radiation direction of the radiator 1 is dynamically adjusted, so that the antenna works in different modes. The switching diode 6 adopts direct current bias, and the equivalent circuits of the switching diode 6 in the off state and the on state are shown in Figure 3

[0029] The radiator 1, the feed line 3, the director 5, and the ground plate 4 are all made of transparent conductive material and are printed on the transparent flexible dielectric substrate.

[0030] The transparent conductive material includes indium tin oxide (ITO), fluorine-doped tin oxide (FTO), tungsten-doped indium oxide (IWO), aluminum-doped zinc oxide (AZO), silver metal mesh, or copper metal mesh; the thickness of the transparent conductive material is less than 1 μm, the surface resistance is less than 3 ohms per unit area, and the light transmittance is greater than 70%. In this embodiment, ITO is used, the conductive film thickness of which is 650 nm, and the surface resistance is 3 ohms per unit area.

[0031] The material of the transparent flexible dielectric substrate includes polyethylene naphthalene (PEN), polydimethylsiloxane (PDMS), polyimide (PI), or polyethylene terephthalate (PET). The thickness of the dielectric substrate PET in this embodiment is 0.125 mm, and the light transmittance of the ITO-PET antenna thus formed is greater than 73%.

[0032] ​Coplanar waveguide-fed (CPW-fed) technology avoids the problem of significant antenna performance degradation due to feeder position deviation, and has the advantages of low cost, high integration, simple manufacturing process, and convenient connection with various passive or active devices. This transparent flexible antenna structure meets the requirements of high transparency, flexibility, and low surface resistance. It can be attached to the lens using ultra-thin transparent PET double-sided adhesive, exhibiting good compatibility with transparent materials, facilitating device miniaturization and minimizing visual impact.

[0033] The specific values ​​of the key dimensions of the antenna are as follows: Figure 4 As shown.

[0034] like Figure 4 As shown, the transparent flexible antenna has a total width W=44 and a total length L=26; the radiator 1 has a width w2=5 and a length l2=8.5, and the trapezoidal transition module 2 has a height d1=1; the feed line 3 has a width w3=3, the ground plane 4 has a width w4=2.5, and the ground plane 4 has a length l3=4; the director 5 has a width w1=4, and the director 5 has a length l1=12 after being disconnected (half the length of the director 5), and the gap width in the middle of the director 5 is d4=0.7; the director 5 stub has a width w5=3, and the director 5 stub has a length d5=1, the distance between the ground plane 4 and the director 5 is d2=10; and the distance between the ground plane 4 and the feed line 3 is d3=0.5.

[0035] In this embodiment, the antenna is designed and simulated using the three-dimensional electromagnetic field simulation software CST. The simulation diagram of the reflection coefficient of the transparent flexible reconfigurable antenna in diode cutoff and conduction modes is shown below. Figure 5 As shown, Mode1 is the diode cutoff mode, and Mode2 is the diode conduction mode. Figure 5 The horizontal axis represents frequency (GHz), and the vertical axis represents reflection coefficient (dB). Simulation results show that the antenna's reflection coefficient is less than -10dB in both modes at 5.8GHz. Figure 6 , Figure 7 The two-dimensional far-field radiation patterns of the transparent flexible reconfiguration antenna at 5.8 GHz, simulated using CST software in both diode-off and diode-on states, are as follows: Figure 6 When the display diode is off, the antenna radiates to the left and right of the lens, with a gain of 4.69 dBi. Figure 7 When the display diode is turned on, the antenna radiates in front of and behind the lens, with a gain of 3.16 dBi.

[0036] The data acquisition module is used to collect the reflection coefficient data or transmission coefficient data of the transparent flexible reconfigurable antenna as it changes due to eye movements after the human body wears a lens with an integrated transparent flexible reconfigurable antenna.

[0037] The basic principle of the data acquisition module is that eye movement (such as opening eyes and closing eyes) changes the dielectric environment around the antenna, thereby causing the reflection coefficient and transmission coefficient of the transparent flexible reconfigurable antenna to change with eye movement.

[0038] In this embodiment, the lens integrated with the transparent flexible reconfigurable antenna includes a transparent conductive film and a transparent flexible dielectric substrate.

[0039] Specifically, in the data acquisition module, the transparent flexible reconfigurable antenna is integrated on the lens in the following manner: being directly printed on the lens surface or being first printed on a transparent flexible film and then attached to the lens surface; the position of the transparent flexible reconfigurable antenna is directly opposite the eye center. In this embodiment, one transparent flexible reconfigurable antenna is arranged on each of the left and right lenses to obtain more comprehensive data.

[0040] Figure 8a is the overall experimental device in which the transparent flexible reconfigurable antenna in the embodiment of the application is integrated on the lens, Figure 8b is a schematic view in which the transparent flexible reconfigurable antenna in the embodiment of the application is connected with a coaxial cable.

[0041] As shown in Figure 8a and Figure 8b , the data acquisition module constructs a complete data acquisition link. The data acquisition link starts from the transparent flexible reconfigurable antenna 100 integrated on the lens, the feed line 3 of the transparent flexible reconfigurable antenna 100 is connected with the conductive wire of the coaxial cable 300 through conductive silver paste, the two ground plates 4 of the transparent flexible reconfigurable antenna 100 are connected with the ground wire of the coaxial cable 300 through conductive silver paste, the coaxial cable 300 is led out from the inside of the frame and the temple, and the other end of the coaxial cable 300 is connected to a signal transceiver device (in this embodiment, a vector network analyzer) through an SMA joint. The data acquisition module collects and records the antenna reflection coefficient (or transmission coefficient) data through the signal transceiver device.

[0042] The data processing and recognition module includes two working modes: In the first mode, the reflection coefficient data (or transmission coefficient) is subjected to filtering and denoising and spectrum analysis, and time domain features and frequency domain features representing eye movement are extracted based on the analysis results, and a recognition result of the eye movement state is output. The time domain features include the open-eye duration, the close-eye duration and the blink frequency, and the frequency domain features include the blink dominant frequency.

[0043] The first mode is a recognition path based on feature extraction, and this path has high calculation efficiency and is suitable for real-time monitoring of simple indexes such as the blink frequency.

[0044] Further specifically, the filtering and denoising adopts methods such as wavelet filtering, sliding average filtering or Kalman filtering.

[0045] In the second mode, the reflection coefficient data is filtered and denoised and short-time Fourier transformed to generate a time-frequency spectrum containing time-frequency characteristics of eye movement, and a pre-trained deep learning model is used to classify the time-frequency spectrum to output an identification result of the eye movement state.

[0046] The second mode is a deep learning-based identification path. In the second mode, energy distribution, instantaneous frequency change and other rich features under different eye states can be extracted from the time-frequency spectrum (as shown in Figure 13 The set of these time-frequency features constitutes an eye movement dataset for deep learning.

[0047] The deep learning model is trained using the eye movement dataset. In this embodiment, the deep learning model uses a Vgg16 deep neural network model. As shown in Figure 14 The core feature of the Vgg16 deep neural network model is the use of multiple layers of small size convolution kernels (3x3) stacked, increasing the network depth through repeated modular structures, and using Figure 13 The Vgg16 deep neural network model is trained using the time-frequency spectrum as a training set until it converges, i.e., a pre-trained deep learning model is obtained. The pre-trained deep learning model can automatically classify different eye movement states (such as open eyes, closed eyes, and blinking) and output identification results (including blink frequency, etc.).

[0048] The following is a detailed layer composition and meaning of the Vgg16 deep neural network model (explained in the order from input to output): First layer: 3x3Conv+ReLU(64), 64 3x3 convolution kernels, step 1, padding=1 (maintain spatial resolution), activation function ReLU (introduce nonlinearity); 3x3Conv+ReLU(64), same as above, further extract local features; 2x2Max-pooling, step 2, reduce the feature map size by half (224→112), output size: 112x112x64.

[0049] Second layer (Block2): 3x3Conv+ReLU(128), double the number of channels (64→128), extract more complex features; 3x3Conv+ReLU(128), same as above; 2x2Max-pooling, step 2, size halved (112→56), output size: 56x56x128.

[0050] The third layer (Block3): 3x3Conv+ReLU (256), the number of channels is 256, further increasing the feature diversity, 3x3Conv+ReLU (256), 3x3Conv+ReLU (256), repeating 3 times of convolution, strengthening the feature combination ability; 2x2Max-pooling, the size is halved (56→28), the output size is: 28x28x256.

[0051] The fourth layer (Block4): 3x3Conv+ReLU (512), the number of channels is 512, capturing higher-order semantic features; 3x3Conv+ReLU (512), 3x3Conv+ReLU (512), 2x2Max-pooling, the size is halved (28→14), the output size is: 14x14x512.

[0052] The fifth layer (Block5): 3x3Conv+ReLU (512), 3x3Conv+ReLU (512), 3x3Conv+ReLU (512), 2x2Max-pooling, the size is halved (14→7), the output size is: 7x7x512.

[0053] The classification layer (fully connected layer): AvgPool (optional), replaced by global average pooling (7x7→1x1) in some implementations, reducing the parameter amount; FC4096+ReLU fully connected layer, flattening 7x7x512 into a 25088-dimensional vector, mapping to 4096 dimensions, ReLU activation function; FC4096+ReLU, the second fully connected layer, further abstracting features; FC1000+Softmax, outputting a 1000-dimensional vector (corresponding to the probability of 1000 classes of ImageNet), and Softmax normalizing the probability.

[0054] The specific process of the embodiment is as follows: The transparent flexible reconfigurable antenna is connected to the vector network analyzer through a coaxial cable to form a data acquisition link. The transparent flexible reconfigurable antenna radiates and receives electromagnetic waves to the eye, and the vector network analyzer collects antenna reflection coefficient data that changes with the eye movement state. The collected original reflection coefficient data is transmitted to the data processing and identification module. If the first mode is selected, the blinking frequency is directly output; if the second mode is selected, the pre-trained deep learning model is used to classify the time-frequency spectrum, and the identification result of the eye movement state is output.

[0055] Four subjects were invited to wear the device to conduct experiments 1 and 2 respectively: experiment 1 was an alternating experiment of keeping eyes open for 5 seconds and closing eyes for 5 seconds; experiment 2 was an experiment of keeping eyes blinking once per second. The measurement was made by a vector network analyzer, 5000 points were collected within 25 seconds, the sampling frequency was 200 Hz, and 8 groups of data were obtained, namely subject1, subject2, subject3, subject4, subject5, subject6, subject7, and subject8. Figure 9 The horizontal coordinate is time s, and the vertical coordinate is reflection coefficient amplitude dB. Figure 9 The horizontal coordinate is time s, and the vertical coordinate is reflection coefficient amplitude dB. Figure 10 The horizontal coordinate is time s, and the vertical coordinate is reflection coefficient amplitude dB. Figure 10 The horizontal coordinate is time s, and the vertical coordinate is reflection coefficient amplitude dB. Figure 9 , Figure 10 It can be seen that the antenna reflection coefficient data under the open eye and closed eye states have obvious differences in amplitude and phase, and the open eye, closed eye, and blinking eye states can be distinguished by the antenna reflection coefficient data.

[0056] Wavelet filtering was performed on each antenna reflection coefficient data to eliminate high-frequency noise, and the denoising level was 4. Compared with other low-pass filters, the wavelet filter can well maintain the sharp transitions in the signal and will not blur the rising / falling edges that may appear in the signal, which is crucial for eye state monitoring. Figure 11 The horizontal coordinate is time s, and the vertical coordinate is reflection coefficient amplitude dB. Fourier transform (FFT) was performed on the data to obtain the blinking frequency, and the blinking frequency was determined as the peak position of FFT. Figure 11 The horizontal coordinate is time s, and the vertical coordinate is reflection coefficient amplitude dB. Fourier transform (FFT) was performed on the data to obtain the blinking frequency, and the blinking frequency was determined as the peak position of FFT. Figure 12 The horizontal coordinate is time s, and the vertical coordinate is reflection coefficient amplitude dB. Fourier transform (FFT) was performed on the data to obtain the blinking frequency, and the blinking frequency was determined as the peak position of FFT.

[0057] Figure 13 The horizontal coordinate is time s, and the vertical coordinate is reflection coefficient amplitude dB. Fourier transform (FFT) was performed on the data to obtain the blinking frequency, and the blinking frequency was determined as the peak position of FFT. Figure 13 The left graph shows the open eye time-frequency spectrum, Figure 13 The right graph shows the closed eye time-frequency spectrum, and the horizontal coordinate is time s and the vertical coordinate is frequency Hz. It can be seen from the graph that the time-frequency spectrum corresponding to different movements has significantly different characteristics, and the spectrum can be classified and recognized based on the characteristics using a deep learning method.

[0058] Those skilled in the art know that, in addition to implementing the system provided by the present application and each device, module and unit thereof in the form of pure computer readable program code, the system provided by the present application and each device, module and unit thereof can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. by logically programming the method steps to achieve the same functions. Therefore, the system provided by the present application and each device, module and unit thereof can be considered as a hardware component, and the devices, modules and units included therein for achieving various functions can also be considered as structures within the hardware component; the devices, modules and units for achieving various functions can also be considered as both software modules implementing methods and structures within hardware components.

[0059] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be combined with each other in any manner without conflict.

Claims

1. An eye movement state recognition system based on a transparent reconfigurable lens integrated antenna, characterized in that, include: A transparent flexible reconfigurable antenna employs a coplanar waveguide structure. The transparent flexible reconfigurable antenna includes a radiator, a feed line, a director, a ground plane, and a switching diode. The radiator, the feed line, the director, the ground plane, and the switching diode are all placed on the same side of a transparent flexible dielectric substrate. The transparent flexible reconfigurable antenna is made of a transparent conductive material. The radiation direction of the radiator is dynamically adjusted by controlling the on / off state of the switching diode. The data acquisition module is used to collect the reflection coefficient data of the transparent flexible reconfigurable antenna as it changes due to eye movements after the human body wears a lens integrating the transparent flexible reconfigurable antenna. The data processing and recognition module includes two working modes: In the first mode, the reflection coefficient data is filtered and denoised, and the spectrum analysis is performed. Based on the analysis results, the temporal and frequency domain features that characterize eye movement are extracted, and the recognition results of eye movement state are output. The second mode involves filtering and denoising the reflection coefficient data and performing short-time Fourier transform to generate a time-frequency map containing the time-frequency characteristics of eye movements. A pre-trained deep learning model is then used to classify the time-frequency map and output the recognition result of the eye movement state.

2. The eye movement state recognition system based on a transparent reconfigurable lens integrated antenna according to claim 1, characterized in that, The radiator is a planar dipole, and the shape of the planar dipole is rectangular, square, trapezoidal, triangular, circular, elliptical, polygonal or fractal structure; the radiator is connected to the signal source through a transmission line, and the transmission line is in the form of microstrip line, coplanar waveguide or parallel double line.

3. The eye movement state recognition system based on a transparent reconfigurable lens integrated antenna according to claim 1, characterized in that, The transparent flexible reconfiguration antenna is a rectangular monopole antenna. A transparent conductive film is fabricated on the transparent flexible dielectric substrate. The antenna structure formed on the transparent conductive film includes a radiator, a feed line, and ground planes located on the left and right sides of the feed line. The radiator and the feed line are connected by a trapezoidal transition module, which serves as a gradient structure to achieve broadband impedance matching. On the left and right sides of the radiator, a parasitic rectangular dipole serving as a director is provided. A gap is opened in the middle of each director, and a switching diode is connected across the gap. By controlling the on / off state of the switching diode, the electrical length of the director is changed, thereby dynamically adjusting the radiation direction of the radiator and enabling the antenna to operate in different modes.

4. The eye movement state recognition system based on a transparent reconfigurable lens integrated antenna according to claim 1, characterized in that, The radiator, the feed line, the director, and the ground plane are all made of transparent conductive material and printed on a transparent flexible dielectric substrate. The transparent conductive material includes indium tin oxide, fluorine-doped tin oxide, tungsten-doped indium oxide, aluminum-doped zinc oxide, silver metal mesh, or copper metal mesh. The thickness of the transparent conductive material is less than 1 μm, the surface resistance is less than 3 ohms / unit area, and the light transmittance is greater than 70%.

5. The eye movement state recognition system based on a transparent reconfigurable lens integrated antenna according to claim 1, characterized in that, The material of the transparent flexible dielectric substrate includes polyvinylnaphthol, polydimethylsiloxane, polyimide, or polyethylene terephthalate.

6. The eye movement state recognition system based on a transparent reconfigurable lens integrated antenna according to claim 1, characterized in that, In the data acquisition module, the transparent flexible reconfigurable antenna is integrated onto the lens in the following ways: directly printed on the lens surface or first printed on a transparent flexible film and then attached to the lens surface; the transparent flexible reconfigurable antenna is positioned directly opposite the center of the eye.

7. The eye movement state recognition system based on a transparent reconfigurable lens integrated antenna according to claim 1, characterized in that, The data acquisition module constructs a data acquisition link, which begins with a transparent flexible reconfigurable antenna integrated on the lens. The feed line of the transparent flexible reconfigurable antenna is connected to the conductive line of the coaxial cable through conductive silver paste. The two ground planes of the transparent flexible reconfigurable antenna are connected to the ground line of the coaxial cable through conductive silver paste. The coaxial cable is led out from inside the frame and temples, and the other end of the coaxial cable is connected to the signal transceiver through an SMA connector. The data acquisition module acquires and records the antenna reflection coefficient data through the signal transceiver.

8. The eye movement state recognition system based on a transparent reconfigurable lens integrated antenna according to claim 1, characterized in that, In the data processing and recognition module, wavelet filtering, moving average filtering, or Kalman filtering is used to filter and denoise the reflection coefficient data.

9. The eye movement state recognition system based on a transparent reconfigurable lens integrated antenna according to claim 1, characterized in that, In the first mode of the data processing and recognition module, the time-domain features include eye-opening duration, eye-closing duration, and blinking frequency, and the frequency-domain features include the dominant blinking frequency. In the second mode of the data processing and recognition module, the time-frequency spectrum includes energy distribution and instantaneous frequency changes under different states.

10. The eye movement state recognition system based on a transparent reconfigurable lens integrated antenna according to claim 1, characterized in that, In the second mode of the data processing and recognition module, the deep learning model adopts the Vgg16 deep neural network model, which uses multiple layers of small-sized convolutional kernels stacked together. The network depth is increased through a repetitive modular structure. The Vgg16 deep neural network model is trained using the time-frequency spectrum as a training set until convergence, thus obtaining the pre-trained deep learning model.

Images