Glasses type eye movement tracking device and eye movement tracking method and system

Abstract

The embodiment of the invention provides a glasses type eye movement tracking device, method and system, and relates to the technical field of eye movement tracking. The glasses type eye movement tracking device comprises a glasses frame, two display screens, an eye movement acquisition assembly and a processor. The processor is arranged on the glasses frame and connected with the eye movement collection assembly, and the processor is used for controlling the two display screens to display a virtual image when it is determined that the target object wears the glasses frame, and controlling the eye movement collection assembly to start and obtain eye movement data of the target object; the eye movement data is eye movement information obtained by the target object based on a preset eye movement normal form task corresponding to the virtual image and is used for evaluating the probability that the target object suffers from the preset brain function disease. According to the embodiment of the invention, the miniaturized and lightweight glasses type eye movement tracking device with a brand new design is provided, the application and carrying portability of the eye movement tracking technology can be improved, meanwhile, the glasses type design is adopted, the wearing is comfortable, and the application range and application scene of eye movement tracking are increased.

Description

Technical Field

[0001] This application relates to the field of eye tracking technology. Specifically, this application relates to a glasses-type eye tracking device, its method, and system. Background Art

[0002] As an important human-computer interaction technology, traditional eye tracking technology, although having great potential and application prospects, still faces some problems in practical applications as follows: 1. Poor portability: An eye tracker with a large volume and weight is difficult to carry around, restricting its application in outdoor or mobile scenarios.

[0003] 2. Uncomfortable to wear: For an eye tracker that needs to be worn on the head, excessive weight may cause discomfort to the subject and affect the accuracy of experimental results. Summary of the Invention

[0004] Embodiments of this application provide a glasses-type eye tracking device, its method, and system, which are used to solve the technical problems of poor portability or uncomfortable wearing of the existing glasses-type eye tracking devices.

[0005] In the first aspect, embodiments of this application provide a glasses-type eye tracking device, including: A glasses frame and two display screens, the two display screens are installed on the glasses frame and are used to display virtual images in front of the eyes of the target object; An eye movement acquisition component, provided on the glasses frame, for acquiring the eye movement data of the target object; A processor, provided on the glasses frame, connected to the eye movement acquisition component, and is used to control the two display screens to display virtual images when it is determined that the target object wears the glasses frame, control the eye movement acquisition component to start and obtain the eye movement data of the target object; the eye movement data is the eye movement information obtained by the target object based on a preset eye movement paradigm task corresponding to the virtual image and is used to evaluate the probability that the target object suffers from a preset brain function disease.

[0006] In a possible implementation, the eye movement acquisition component includes: At least one light source, used to irradiate light onto the cornea of the target object to form a reflected light spot on the cornea; At least one camera module, used to collect the pupil change information and the change information of the reflected light spot of the target object's eyes, and generate eye movement data based on the pupil change information and the change information of the reflected light spot.

[0007] In a possible implementation, the glasses-type eye tracking device further includes: An integrated sensor module is provided on the glasses frame, which is used to collect the status information of the target object wearing the glasses frame and send the status information to the processor, so that the processor can perform corresponding operations based on the status information.

[0008] In a possible implementation, the integrated sensor module includes at least one of the following: An inertial measurement unit sensor, including an accelerometer, a gyroscope, and a magnetometer, which is used to collect the motion information of the target object; the status information includes the motion information. A proximity light sensor, which is used to generate wearing information and send the wearing information to the processor when it detects that the distance between the target object and the proximity light sensor is not greater than a predetermined distance, so that the processor can determine that the target object wears the glasses frame based on the wearing information.

[0009] In a possible implementation, at least one light source is two near-infrared light sources, and at least one camera module is two near-infrared cameras; The glasses frame includes two edge parts respectively provided below the two display screens; One near-infrared light source and one near-infrared camera are set as a group and arranged on one edge part.

[0010] In a possible implementation, the glasses-type eye tracking device further includes: A refractive adjustment button is provided on the glasses frame and above the display screen, which is used to change the distance and / or angle between the optical elements of the display screen when the refractive adjustment button is rotated, so as to adjust the diopter.

[0011] In a possible implementation, the glasses-type eye tracking device further includes at least one of the following: A scene camera is provided on the glasses frame and in the middle area of the glasses frame; the middle area is located between the two display screens; At least one speaker and a volume control button are provided on the temple of the glasses frame.

[0012] Two electrochromic lenses and a light transmission button, each electrochromic lens is correspondingly provided on one side of the display screen close to the target object, and the light transmission button is provided on the temple of the glasses frame.

[0013] In a possible implementation, the glasses-type eye tracking device further includes at least one of the following: Two nose pads are detachably mounted on the glasses frame; A light-shielding member is detachably mounted on the glasses frame and is used to be correspondingly provided in front of the display screen.

[0014] In a possible implementation, the glasses-type eye tracking device further includes: A power supply and data interface is provided at the end of the temple of the glasses frame, for connecting to an external power supply or a terminal device.

[0015] In a possible implementation, the preset brain functional diseases include at least one of the following: cognitive dysfunction, Parkinson's disease, schizophrenia, anxiety disorder, and depression.

[0016] In a second aspect, an embodiment of the present application provides an eye movement tracking system, including a terminal device and the glasses-type eye movement tracking device in the first aspect; The terminal device is connected to the glasses-type eye movement tracking device, and the terminal device is used to evaluate the probability that the target object has a preset brain functional disease based on the eye movement data.

[0017] In a third aspect, an embodiment of the present application provides an eye movement tracking method, which is applied to the glasses-type eye movement tracking device in the first aspect. The method includes: When it is determined that the target object wears the glasses frame, controlling the two display screens to display virtual images, controlling the eye movement acquisition component to start and obtaining the eye movement data of the target object collected by the eye movement acquisition component; the eye movement data is the eye movement information obtained by the target object based on a preset eye movement paradigm task corresponding to the virtual image; Sending the eye movement data to the terminal device so that the terminal device evaluates the probability that the target object has a preset brain functional disease based on the eye movement data.

[0018] In a possible implementation, when a light-shielding member is installed on the glasses frame of the glasses-type eye movement tracking device, controlling the two display screens to display virtual images includes: in the virtual reality mode, when the two display screens are shielded by the light-shielding member, controlling the two display screens to display virtual images; or, When a light-shielding member is not installed on the glasses frame of the glasses-type eye movement tracking device and the glasses frame is provided with a scene camera, controlling the two display screens to display virtual images includes: in the augmented reality mode, starting the scene camera and controlling the two display screens to display virtual images.

[0019] The beneficial effects brought by the technical solution provided by the embodiment of the present application are: The glasses-type eye movement tracking device according to the embodiment of the present application includes a glasses frame and two display screens. The eye movement acquisition component and the processor are arranged on the glasses frame. The eye movement data of the target object is acquired through the eye movement acquisition component. When it is determined that the target object wears the glasses frame, the processor controls the two display screens to display virtual images, controls the eye movement acquisition component to start and obtain the eye movement data of the target object, and further obtains the eye movement data. Since the eye movement data is the eye movement information obtained by the target object based on the preset eye movement paradigm task corresponding to the virtual image, the probability that the target object suffers from the preset brain function disease can be evaluated. Therefore, the embodiment of the present application provides a glasses-type eye movement tracking device with a brand-new design that is miniaturized and lightweight. By using this glasses-type eye movement tracking device, eye movement tracking can be realized, the portability of the application and carrying of the eye movement tracking technology can be improved, and at the same time, the glasses-type design is comfortable to wear, the applicable range and applicable scenarios of eye movement tracking are increased, the difficulty of equipment installation and deployment is reduced, and the auxiliary diagnosis and screening of brain physiology, cognitive function and neurodegenerative diseases are made more convenient. Description of the Drawings

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description in the embodiments of the present application.

[0021] Figure 1 It is a schematic structural diagram of the first glasses-type eye movement tracking device provided by the embodiment of the present application; Figure 2 It is a schematic structural diagram of the second glasses-type eye movement tracking device provided by the embodiment of the present application; Figure 3 It is a hardware framework diagram of a glasses-type eye movement tracking device provided by the embodiment of the present application; Figure 4 It is a framework schematic diagram of an eye movement tracking system provided by the embodiment of the present application; Figure 5 It is a flowchart of an eye movement tracking method provided by the embodiment of the present application.

[0022] Reference Signs: 10 - Glasses-type eye movement tracking device; 101 - Glasses frame, 102 - Display screen, 103 - Processor, 104 - Eye movement acquisition component, 1041 - Near-infrared light source, 1042 - Near-infrared camera, 105 - Integrated sensor module, 1051 - Inertial measurement unit sensor, 1052 - Proximity light sensor, 106 - Nose pad, 107 - Power supply and data interface, 108 - Refractive adjustment button, 109 - Scene camera, 110 - Speaker, 111 - Volume control button, 112 - Electrochromic lens, 113 - Translucent button, 114 - Flash module; 20 - Terminal device. Detailed implementation manners

[0023] The embodiments of the present application will be described below with reference to the accompanying drawings in the present application. It should be understood that the implementation manners described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0024] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the terms "comprising" and "including" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components and / or their combinations supported by the art of the present technology. It should be understood that when we say an element is "connected" or "coupled" to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" indicates being implemented as "A", or being implemented as "B", or being implemented as "A and B".

[0025] To make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0026] In the related art, eye tracking technology can monitor and analyze the movement and fixation points of the human eye in real time. Eye tracking technology is based on the analysis of eye videos and mainly uses non-invasive video eye tracking methods. By recording the positions of the pupil and the corneal reflection spot, a geometric model is used to estimate the line of sight direction. An eye tracking system usually includes one or more cameras, light sources and advanced image processing algorithms, and can convert the eye movement into a data stream, including information such as pupil position, gaze vector and fixation point.

[0027] Eye tracking technology has experienced a long-term development, from early invasive methods, such as using magnetic contact lenses or sensors that monitor the electrical discharge of eye muscles, to modern non-invasive methods, such as video-based eye tracking technology. Modern technology reflects infrared light and captures eye movements through multiple cameras. The performance is measured in hertz (frames per second), and the fidelity is judged by the resolution and accuracy.

[0028] Association between Brain Neural Circuits and Eye Movements: Eye movements are controlled by complex neural circuits in the brain. Regions such as the frontal lobe, parietal lobe, and midbrain are all involved in the regulation of eye movements. When these regions are damaged due to neurodegenerative diseases, it will affect the function of the neural circuits controlling eye movements, and then lead to abnormal eye movement patterns. For example, patients with MCI (mild cognitive impairment) may have early pathological changes in specific regions of the brain, which will interfere with the transmission and processing of neural signals, resulting in problems with eye movement control and characteristic abnormal eye movement manifestations.

[0029] Cognitive dysfunction caused by neurodegenerative diseases will be reflected in eye movements. Eye movements can serve as a window to reflect cognitive function. For example, patients with MCI have mild impairment of cognitive function. When performing eye movement tasks that require cognitive participation, such as eye movement tests that require memory, attention, executive function, etc., due to insufficient cognitive function, they cannot complete the tasks as effectively as normal people, thus showing specific abnormal eye movements. For example, in an eye movement task that requires memorizing images, patients with MCI may have problems with image recognition and memory due to impaired memory function, which will in turn affect the subsequent eye movement pattern, manifested as abnormal fixation times for familiar and novel images, etc.

[0030] Eye tracking technology can accurately record and quantify various eye movement parameters, such as saccade speed, latency, accuracy, smooth pursuit gain, fixation time, etc. By analyzing these quantified indicators, it can be compared with the eye movement data of the normal population to establish an eye movement characteristic model for the disease. When the eye movement data of the tested person deviates from the normal range, reaches a certain threshold or conforms to a specific abnormal pattern, it may indicate the risk of neurodegenerative diseases such as MCI. Moreover, as the disease progresses, the eye movement abnormalities may become more obvious, and these quantified indicators can also be used to monitor the development process of the disease.

[0031] Traditional eye movement systems are usually limited by the technical level during the design and manufacturing process, and it is difficult to achieve miniaturization and lightweight while ensuring performance. Therefore, the following three problems arise: 1. Poor portability: Eye trackers with large volume and weight are difficult to carry around, restricting their application in outdoor or mobile scenarios.

[0032] 2. Uncomfortable wearing: For eye trackers that need to be worn on the head, excessive weight may cause discomfort to the subjects and affect the accuracy of the experimental results.

[0033] 3. Limited application scenarios: Due to the limitations of volume and weight, traditional eye trackers are difficult to be applied in some special scenarios, such as virtual reality (VR), augmented reality (AR), etc.

[0034] Further research has found that, as an important human-computer interaction technology, although traditional eye-tracking technology has great potential and application prospects, it still faces some challenges and problems in practical applications: 1. Large volume and weight: To achieve high-precision eye movement tracking, traditional eye trackers usually come with complex optical systems. These optical systems consist of multiple high-quality lenses, mirrors and other components. For example, in order to ensure clear capture of eye details at different distances, a lens group with variable focal length is required, which makes the optical part bulky. Moreover, to ensure accurate transmission and focusing of light, these optical components have high size and precision requirements. To protect the internal delicate optical and electronic components, the housing of traditional eye trackers is generally made of sturdy materials such as metal or high-strength plastic. The thickness of the housing needs to be thick enough to prevent damage to internal components caused by external collisions, vibrations and other factors. This increases the volume and weight of the entire device.

[0035] 2. Complex structure: To accurately position in front of the user's eyes, traditional eye trackers require a stable mechanical bracket. This bracket should be able to adjust the position and angle of the eye tracker to adapt to different users' facial features and usage postures. Generally, a metal material is used to make the bracket to ensure sufficient strength and stability. This bracket structure is usually relatively heavy, and in order to achieve flexible adjustment functions, multiple adjustable joints and knobs are designed, and these components also increase the volume and weight of the device.

[0036] 3. Poor portability: Since numerous optical and electronic components inside traditional eye trackers consume a large amount of electrical energy, especially the complex optical systems and data processing units have high power during operation. To ensure the long-term stable operation of the device, either a large-capacity battery needs to be equipped, or it needs to be powered by an external power supply. The large-capacity battery itself has a certain volume and weight, and the external power supply also increases the overall weight of the device. Coupled with the weight of the device itself, it greatly limits the portability of the device.

[0037] 4. Complicated installation and debugging of the device: The heavier and larger eye trackers are usually more complicated to install. For example, they need to be installed on special brackets or fixing devices, and it may require multiple people to cooperate to complete the installation process. And during the installation process, precise adjustment of the position and angle is required, which will consume a lot of time and energy.

[0038] 5. Poor comfort: During long-term experiments or monitoring processes, the weight of the eye tracker directly affects the comfort of the user. If the eye tracker is too heavy, for example, a head-mounted eye tracker is too heavy, it will cause greater pressure on the user's head and face, and easily cause fatigue and pain in the head and neck. This may make it difficult for the user to concentrate on participating in the experiment or normal activities for a long time, thus affecting the accuracy and reliability of the data.

[0039] 6. High cost: The device has many components and a relatively large volume, with a large structural housing. Eventually, it is difficult to control the costs of electronic components and structural parts.

[0040] 7. Great limitation in eye movement tracking scenarios: Usually, it needs to be placed at a fixed position, such as on a desktop, and connected to a computer. The subject must sit at a specific position and within a certain distance range in front of the table to accurately record eye movement information. This greatly restricts the subject's activity range, making it impossible to move and observe in a natural and free state, and it is difficult to obtain eye movement data in a real situation.

[0041] The glasses-type eye movement tracking device, its method, and system provided by this application aim to solve the above technical problems in the prior art.

[0042] The following uses specific embodiments to elaborate in detail on the technical solutions of this application and how the technical solutions of this application solve the above technical problems. It should be noted that the following embodiments can refer to, draw on, or combine with each other. For the same terms, similar features, and similar implementation steps in different embodiments, they will not be described repeatedly.

[0043] See Figure 1 As shown, the structural schematic diagram of the first glasses-type eye movement tracking device 10 provided by the embodiment of this application is shown. See Figure 2 As shown, the structural schematic diagram of the second glasses-type eye movement tracking device 10 provided by the embodiment of this application is shown.

[0044] Combined with Figure 1 and Figure 2 As shown, the glasses-type eye movement tracking device 10 includes: a glasses frame 101, two display screens 102, an eye movement acquisition component 104, and a processor 103. The two display screens 102 are installed on the glasses frame 101 and are used to display virtual images in front of the eyes of the target object.

[0045] The eye movement acquisition component 104 is provided on the glasses frame 101, and the eye movement acquisition component 104 is used to acquire the eye movement data of the target object.

[0046] The processor 103 is provided on the glasses frame 101 and is connected to the eye movement acquisition component 104. The processor 103 is used to control the two display screens 102 to display virtual images when it is determined that the target object wears the glasses frame 101, control the eye movement acquisition component 104 to start and obtain the eye movement data of the target object; the eye movement data is the eye movement information obtained by the target object based on a preset eye movement paradigm task corresponding to the virtual image and is used to evaluate the probability that the target object suffers from a preset brain function disease.

[0047] The eyeglass-type eye movement tracking device 10 according to the embodiment of the present application includes an eyeglass frame 101 and two display screens 102. The eye movement acquisition component 104 and the processor 103 are arranged on the eyeglass frame 101. The eye movement data of the target object is collected through the eye movement acquisition component 104. When it is determined that the target object wears the eyeglass frame 101 through the processor 103, the two display screens 102 are controlled to display virtual images, and the eye movement acquisition component 104 is controlled to start and obtain the eye movement data of the target object, and then the eye movement data is obtained. Since the eye movement data is the eye movement information obtained by the target object based on the preset eye movement paradigm task corresponding to the virtual image, the probability that the target object suffers from the preset brain function disease can be evaluated.

[0048] Therefore, the embodiment of the present application provides a newly designed miniaturized and lightweight eyeglass-type eye movement tracking device 10. By using the eyeglass-type eye movement tracking device 10, eye movement tracking can be realized, the portability of the application and carrying of the eye movement tracking technology can be improved. At the same time, the eyeglass-type design is comfortable to wear, the applicable range and applicable scenarios of eye movement tracking are increased, the difficulty of equipment installation and deployment is reduced, and the auxiliary diagnosis and screening of brain physiology, cognitive function and neurodegenerative diseases are made more convenient.

[0049] Optionally, the eyeglass frame 101 mainly uses a special high-strength engineering plastic frame, and the core components of eye movement tracking are ingeniously integrated into the eyeglass frame 101. The design of the eyeglass frame 101 ensures sufficient strength to support each component while reducing the weight as much as possible to improve the comfort of the wearer.

[0050] Optionally, the preset eye movement paradigm task can be a traditional saccade test task, such as: forward saccade task, reverse saccade task, memory-guided saccade task, etc.

[0051] In some embodiments, the eye movement acquisition component 104 includes: at least one light source and at least one camera module.

[0052] At least one light source is used to irradiate light onto the cornea of the target object to form a reflected light spot on the cornea. At least one camera module is used to collect the pupil change information and the change information of the reflected light spot of the target object's eye, and generate eye movement data based on the pupil change information and the change information of the reflected light spot.

[0053] See Figure 2 As shown, at least one light source is two near-infrared light sources 1041, and at least one camera module is two near-infrared cameras 1042. The eyeglass frame 101 includes two edge parts respectively arranged below the two display screens 102; one near-infrared light source 1041 and one near-infrared camera 1042 are set as a group and arranged on one edge part.

[0054] See Figure 2As shown, the eye movement acquisition component 104 is usually arranged in a combined structure on the left and right, and is located at the lower edge position of the glasses frame 101. The near-infrared light source 1041 uses 940nm invisible light to irradiate the display screen 102 to form a reflected light spot. Since it is invisible light, it will not interfere with the eye movement process. The near-infrared camera 1042 takes pictures and analyzes the user's eyes from bottom to top, and records the pupil movement information of the eyes and the change information of the reflected light spots of the eyes.

[0055] Optionally, the glasses-type eye movement tracking device 10 may be a VR (Virtual Reality) glasses or an AR (Augmented Reality) glasses.

[0056] In some embodiments, the glasses-type eye movement tracking device 10 further includes: an integrated sensor module 105.

[0057] The integrated sensor module 105 is disposed on the glasses frame 101. The integrated sensor module 105 is used to collect the state information of the target object wearing the glasses frame 101, and send the state information to the processor 103, so that the processor 103 performs corresponding operations based on the state information.

[0058] Optionally, the processor 103 is used to collect, package and forward all sensor data and camera images, and finally transmit the data to a terminal device such as a mobile phone, a tablet computer, or a computer through a data transmission line for analysis.

[0059] Optionally, the processor 103 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in combination with the disclosure of the present application. The processor 103 may also be a combination that implements a computing function, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0060] See Figure 3 As shown, the embodiment of the present application provides a hardware framework diagram of a glasses-type eye movement tracking device 10. As Figure 3 As shown, the integrated sensor module 105 includes at least one of the following: an inertial measurement unit sensor 1051, a proximity light sensor 1052.

[0061] The inertial measurement unit sensor 1051 includes an accelerometer, a gyroscope, and a magnetometer. The inertial measurement unit sensor 1051 is used to collect the motion information of the target object; the status information includes the motion information.

[0062] The proximity light sensor 1052 is used to generate wearing information and send the wearing information to the processor 103 when it detects that the distance between the target object and the proximity light sensor 1052 is not greater than a predetermined distance, so that the processor 103 determines that the target object wears the glasses frame 101 based on the wearing information.

[0063] Wherein, the wearing information is used to indicate that the target object wears the glasses frame 101. The processor 103 is used to determine that the target object wears the glasses frame 101 in response to obtaining the wearing information, control the two display screens 102 to display virtual images, and control the eye movement acquisition component 104 to start and obtain the eye movement data of the target object.

[0064] Optionally, the inertial measurement unit sensor 1051 is used to monitor the movement and direction adjustment during the use of the glasses-type eye movement tracking device 10 by the user.

[0065] Optionally, the proximity light sensor 1052 estimates the proximity distance of an object by using the reflection of light. After the glasses-type eye movement tracking device 10 is connected to an external device, it is necessary for the proximity light sensor 1052 to notify components such as the near-infrared camera 1042, the display screen 102, and the near-infrared light source 1041 to work when it senses the approach of an object, that is, some components of the glasses work after the user wears the glasses, and some components enter the sleep state to control the energy consumption when the user takes them off.

[0066] See Figure 3 As shown, the glasses-type eye movement tracking device 10 further includes: a refractive adjustment button 108.

[0067] The refractive adjustment button 108 is provided on the glasses frame 101 and above the display screen 102. The refractive adjustment button 108 is used to change the distance and / or angle between the optical elements of the display screen 102 when the refractive adjustment button 108 is rotated, so as to adjust the diopter.

[0068] See Figure 3 As shown, the glasses-type eye movement tracking device 10 further includes at least one of the following: a scene camera 109, at least one speaker 110 and a volume control button 111, two electrochromic lenses 112, and a light transmission button 113.

[0069] See Figure 1As shown, the scene camera 109 is disposed on the glasses frame 101 and located in the middle area of the glasses frame 101; the middle area is located between the two display screens 102. The scene camera 109 is used to record the environmental scene information when the target object wears the glasses.

[0070] At least one speaker 110 and the volume control button 111 are both disposed on the temple of the glasses frame 101. Each electrochromic lens 112 is correspondingly disposed on one side of a display screen 102 close to the target object, and the light-transmitting button 113 is disposed on the temple of the glasses frame 101.

[0071] Optionally, at least one speaker 110 includes a left speaker and a right speaker. The left speaker and the right speaker are respectively located on the left and right temples of the glasses frame 101, and the sound adjustment is realized by the volume control button 111 located under the left temple.

[0072] In some embodiments, the glasses-type eye tracking device 10 further includes at least one of the following: two nose pads 106, a light-shielding member.

[0073] See Figure 2 As shown, the two nose pads 106 are detachably mounted on the glasses frame 101. The light-shielding member is detachably mounted on the glasses frame 101 and is used to be correspondingly disposed in front of the display screen 102.

[0074] Optionally, when the light-shielding member is mounted on the glasses frame 101 of the glasses-type eye tracking device 10, in the virtual reality mode, when the two display screens 102 are shielded by the light-shielding member, the processor 103 is used to control the two display screens 102 to display virtual images; or, when the light-shielding member is not mounted on the glasses frame 101 of the glasses-type eye tracking device 10 and the glasses frame 101 is provided with the scene camera 109, in the augmented reality mode, the scene camera 109 is started, and the processor 103 is used to control the two display screens 102 to display virtual images.

[0075] Optionally, the processor 103 is further used to determine the visual focus points of the target object in different scenarios based on the environmental scene information and eye movement data collected by the scene camera 109.

[0076] Optionally, the nose pads 106 are optimized ergonomically to adapt to different facial contours, reduce the pressure feeling during long-term wearing, and are compatible with nose pads 106 of different sizes according to different user facial features and usage habits.

[0077] Optionally, the light-shielding member is a detachable structure, which can be a light-shielding cover. The light-shielding member is installed directly in front of the display screen 102. When the light-shielding member shields the two display screens 102, the display screens 102 are opaque. The light-shielding member can be installed when the user views the stimulation points on the screen for the eye movement tracking scenario, so as to prevent the ambient light from passing through the screen and the display screen 102 and interfering with the user's eye movement tracking process. The light-shielding member can be removed when performing eye movement tracking in an open environment.

[0078] Optionally, the electrochromic lens 112 can accurately adjust the light transmittance and color of the display screen 102 according to different environments and requirements. For example, in a strong light environment, the lens can be dimmed to reduce the light entering the eyes; in a weak light environment, the display screen 102 can be brightened to ensure a clear field of vision, which is suitable for various environments. The electrochromic lens 112 is located behind the display screen 102, and the adjustment of the light transmittance of the electrochromic lens 112 is achieved through the light transmission button 113 at the temple of the glasses frame 101.

[0079] Optionally, the display screen 102 can be a high-definition, low-latency OLED (Organic Light-Emitting Diode) micro display screen. Combined with advanced optical projection technology, the virtual image is clearly presented in front of the user's eyes. It is composed of a left screen and a right screen, and the displayed image transmits the output signal of the external device through a data transmission line (Type-c). The top of the display screen 102 is also equipped with a refractive power adjustment button 108. When rotating the refractive power adjustment button 108, the distance or angle between the optical elements is changed to achieve the adjustment of the refractive power.

[0080] See Figure 2 As shown, the glasses-type eye movement tracking device 10 further includes: a power supply and data interface 107.

[0081] The power supply and data interface 107 is provided at the end of the temple of the glasses frame 101, and the power supply and data interface 107 is used to connect to an external power supply or connect to the terminal device 20.

[0082] See Figure 3 As shown, the glasses-type eye movement tracking device 10 further includes a flash module 114, and the flash module 114 is connected to the processor 103.

[0083] The flash module 114 is used to store program storage and important data storage. The processor 103 combines the high-speed read and write capabilities of the built-in DDR with the non-volatile storage of the external Flash, which can give full play to their respective advantages and improve the performance of the entire glasses-type eye movement tracking device 10. The DDR can quickly respond to the user's operations and process various real-time data, while the Flash can provide stable program and data storage support for the glasses-type eye movement tracking device 10, enabling the glasses-type eye movement tracking device 10 to achieve better performance in terms of startup, operation, and data processing.

[0084] The flash module 114 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, which is not limited here.

[0085] Optionally, the flash module 114 is used to store the computer program for implementing the embodiments of the present application and is controlled by the processor 103 to execute. The processor 103 is used to execute the computer program stored in the flash module 114 to implement the steps shown in the foregoing method embodiments.

[0086] The glasses-type eye movement tracking device 10 of the embodiments of the present application has functions of brain cognition and physiological function diagnosis based on eye movement tracking, supports eye movement tracking, supports binocular synchronous acquisition, supports eye movement trajectory tracking and processing, supports pupil data change acquisition and analysis, and is used for the assessment of neuropsychological diseases such as cognitive dysfunction, Parkinson's disease, schizophrenia, anxiety disorder, and depression. As an example, the process of the glasses-type eye movement tracking device 10 of the embodiments of the present application for eye movement tracking includes: (1) Install a light shield. The application scenario is the VR virtual reality mode for an immersive experience. The display screen 102 shows different behavioral paradigm contents and gives voice guidance according to the external device program for different indications, and the user performs eye movement tasks as required. Among them, the behavioral paradigms include: saccade, antisaccade, memory saccade, double-step saccade, smooth pursuit, working memory capacity, visual acuity, visual field, vergence eye movement, etc. The near-infrared light source 1041 irradiates the eyes, and the high-precision near-infrared camera 1042 captures the light reflected by the eyes, so as to obtain information such as the position of the eyes, pupil size, and eyeball rotation angle. Advanced image recognition algorithms and deep learning models are used to perform real-time analysis on the collected eye images to achieve precise tracking of the user's eye movement trajectory.

[0087] (2) By analyzing the eye movement patterns of the user when looking at different contents, such as fixation point distribution, fixation time, saccade frequency, etc., infer the user's attention concentration, cognitive load, information comprehension ability, inhibition ability, learning ability, etc., and obtain the user scores under each paradigm by referring to the weights of each characteristic parameter under different diseases, and finally evaluate and output a conclusion report.

[0088] (3) Eye movement tracking function based on visual field research Remove the light shield. The application scenario is the AR augmented reality mode, which can collect data of the real scene through the scene and sensors, and combine the real world and the virtual world. In the embodiment of the present application, by performing correlation analysis on the environmental scene information collected by the scene camera 109 and the eye movement data, it is possible to understand the user's visual focus points in different scenes. The device has an automatic real-time calibration function. After the user wears the glasses-type eye movement tracking device 10, the system will automatically guide the user to perform a simple calibration operation, and quickly establish an eye feature model of the user by looking at several specific points. During use, the algorithm will adaptively adjust the tracking parameters according to the environmental light, the user's eye state, and the scene changes to ensure accurate eye movement tracking and scene perception in various complex environments.

[0089] In some embodiments, the preset brain functional diseases include at least one of the following: cognitive dysfunction, Parkinson's disease, schizophrenia, anxiety disorder, and depression.

[0090] The glasses-type eye movement tracking device 10 in the embodiment of the present application can achieve the function of eye movement tracking through the following embodiments.

[0091] Embodiment 1: Structural design Combine Figure 2 and Figure 3As shown in the figure, the device structure design of the embodiment of the present application includes a glasses frame 101, a processor 103, an inertial measurement unit sensor 1051, a proximity light sensor 1052, a flash module 114, a near-infrared camera 1042, a scene camera 109, a near-infrared light source 1041, a speaker 110, a nose pad 106, a light-shielding member, an electrochromic lens 112, a display screen 102, a refractive adjustment button 108, a volume control button 111, a light-transmitting button 113, and a data transmission line.

[0092] Among them, the glasses frame 101 is mainly made of lightweight and high-strength materials. This material not only ensures sufficient strength to support the various components of the eye tracker but also significantly reduces the overall weight of the glasses, improving the wearing comfort. The shape design of the glasses frame 101 is based on ergonomic principles. It is compatible with nose pads 106 of different sizes according to the nasal bridge and facial features of different users, reducing the sense of pressure and increasing the immersion of eye tracking. Moreover, the nose pad 106 supports mounting refractive lenses, and most myopic users can achieve eye tracking without wearing refractive lenses.

[0093] On the inner side of the front end of the glasses frame 101, a set of micro infrared light-emitting diode arrays is arranged on each side along the frame edge. These LEDs (light-emitting diodes) use low-power and high-brightness near-infrared light-emitting chips to emit infrared light. To improve the emission efficiency and uniformity of infrared light, a small condenser lens is equipped above each LED. These lenses are made of optical plastic and are precision injection molded, which can focus the light emitted by the LED and project it evenly onto the surface of the eyeball, providing clear and stable lighting conditions for subsequent eye tracking. The integrated sensor module 105 uses a micro near-infrared camera module, which can quickly and accurately capture the motion images of the eyeball. The near-infrared camera 1042 is connected to the data processing units of a computer, a mobile phone, and a tablet through a high-speed data transmission line. The speaker 110 is located on both sides of the glasses temple near the ears and gives a sound reminder when a voice reminder is needed.

[0094] Embodiment 2: Diagnosis of Brain Physiology, Cognitive Function, and Neurodegenerative Diseases First, the glasses-type eye tracker 10 is connected to a supported mobile phone, tablet, or computer. The user wears the glasses-type eye tracker 10 to ensure that both eyes can see all the content on the screen. If not, replace the nose pad 106 of different sizes as needed. Then, calibrate the glasses-type eye tracker 10. After calibration, start initial data collection. The user starts to operate or observe the stimulus material according to the preset eye movement paradigm task, and the near-infrared camera 1042 collects images.

[0095] External devices precisely record and quantify various eye movement parameters during the user's task execution, such as saccade speed, latency, accuracy, smooth pursuit gain, fixation duration, etc. By analyzing these quantified metrics, it is possible to compare with the eye movement data of the normal population and establish an eye movement feature model for diseases. When the eye movement data of the tested person deviates from the normal range, reaches a certain threshold or conforms to a specific abnormal pattern, it may indicate the risk of neurodegenerative diseases such as MCI. These data include information such as the position of the fixation point (X and Y coordinates in the screen coordinates), fixation duration (the duration of each fixation point), saccade path (the trajectory of the eye rapidly moving from one fixation point to another), pupil size change (reflecting the cognitive load, emotional state, etc. of the subject), etc.

[0096] Finally, the collected raw eye movement data is cleaned and preprocessed. This may include removing invalid data points caused by user blinking, head movement, etc.; smoothing the fixation point data to reduce noise interference in the data; standardizing the data of different users to make it comparable, for example, normalizing the fixation duration according to the total task duration. This is to facilitate researchers to more intuitively understand and interpret the data, discover the laws and trends in the data, and thus draw conclusions about the visual behavior of the subjects.

[0097] Example 3: Eye movement tracking based on visual field research A scene camera 109 is adopted in the front of the glasses-type eye movement tracking device 10 to collect visual field data. The near-infrared camera 1042 captures the eye fixation points. The electrochromic lens 112 can adjust the light transmittance as needed to facilitate the user to observe the scene. By shooting the scene with the scene camera 109 and collecting the eye fixation points with the near-infrared camera 1042, the fixation situation of the human eye on the scene camera 109 can be calculated, which can be used for scientific research to analyze the eye movement tracking situation of humans in multiple fields such as consumption, education, medical treatment, and aviation.

[0098] The main problem solved by the portable glasses-type eye movement tracking device 10 provided by the embodiment of the present application is: to increase the portability of the eye movement tracking device, control the volume and weight of the device, and further facilitate the auxiliary diagnosis and screening of brain physiology, cognitive function, and neurodegenerative diseases. The problem is mainly solved from two aspects: hardware facilities and supporting software systems.

[0099] The embodiment of the present application aims to overcome the many drawbacks brought by the large volume and weight of traditional eye trackers, and provide an innovative eye tracker design scheme to achieve the miniaturization and light weight of the device. Traditional eye trackers have played an important role in the diagnosis of nervous system diseases, cognitive disorder diseases, and psychological diseases. A more convenient glasses-type eye movement device can significantly improve its use experience and application scope.

[0100] In the context of the continuous development of modern science and technology, eye trackers are playing an increasingly important role in many fields such as psychology research, human-computer interaction, and market research. However, due to factors such as its complex and large internal optical system, data processing unit, mechanical structure, and relatively inefficient power supply system, traditional eye trackers are bloated and overweight. This not only limits its portability in field research and mobile application scenarios, making it difficult for researchers to work flexibly in a variety of environments, but also causes great pressure on the user's head and face during long-term use, seriously affecting the comfort of use. It also increases the difficulty and complexity of installation and debugging, and reduces work efficiency.

[0101] The embodiment of the present application adopts highly integrated micro-optical sensing technology to miniaturize the traditional large optical components and optimize their layout, thereby greatly reducing the space occupied by the optical system. In terms of mechanical structure, the innovative XR (Extended Reality) glasses design is adopted, which is lightweight and sturdy as a whole, which not only ensures the stability and accuracy of the device during use, but also greatly reduces the volume and weight of the bracket. In addition, considering the facial features and usage postures of different users, multiple sets of detachable nose pads 106 are designed for selection. When experiencing immersive eye movement, it may be affected by the ambient light. In the embodiment of the present application, a shading piece can be detachably installed on the glasses frame 101, and the shading piece is used for shading to avoid interference of ambient light and reflected light on eye tracking as much as possible.

[0102] Therefore, the embodiments of the present application are committed to creating an eye tracker with a compact size, lightweight and excellent performance, so that it can be widely used in various complex and changeable scenarios, whether it is rigorous research in the laboratory, or auxiliary diagnosis of neurological diseases, cognitive disorders, and psychological diseases, or integrated applications in outdoor field research. It can provide users with convenient, comfortable and efficient eye tracking services, promote the popularization and development of eye tracker technology in more fields, and open up new broad prospects for research and application in related fields.

[0103] The eye-tracking device 10 of the glasses used in the embodiment of the present application can at least achieve the following technical effects: 1. High comfort level. Glasses-type eye trackers are more in line with the natural state of human beings and will not cause excessive discomfort to the subjects when worn.

[0104] 2. Low cost. The overall hardware supporting cost is low and easy to promote.

[0105] 3. High portability. The glasses-type eye tracker is lightweight and can be carried around like ordinary glasses, making it convenient to use flexibly in fixed and mobile scenes.

[0106] 4. Low energy efficiency. The glasses-type eye tracker consumes less power and is more economical to use.

[0107] 5. Easy to install. The wearing method of the glasses-type eye tracker is simple and fast, just like wearing ordinary glasses, and almost no additional installation steps are required.

[0108] 6. The auxiliary diagnosis function of brain cognition and physiological functions based on eye movement tracking technology provides users with a new, portable, and non-invasive way of health monitoring and cognitive assessment.

[0109] See Figure 4 As shown, the embodiment of the present application provides a schematic framework diagram of an eye movement tracking system. As Figure 4 shown, the eye movement tracking system includes a terminal device 20 and the glasses-type eye movement tracking device 10 of the embodiment of the present application.

[0110] The terminal device 20 is connected to the glasses-type eye movement tracking device 10, and the terminal device 20 is used to evaluate the probability that the target object has a preset brain function disease based on the eye movement data.

[0111] See Figure 4 As shown, the terminal device 20, as an external device, can be at least one of a smart phone 201, a notebook computer 202, and a desktop computer 203.

[0112] See Figure 5 As shown, the embodiment of the present application provides a flowchart of an eye movement tracking method. The eye movement tracking method is applied to the glasses-type eye movement tracking device 10 of the embodiment of the present application, and the method includes: step S501 to step S502.

[0113] S501. When it is determined that the target object wears the glasses frame 101, control the two display screens 102 to display virtual images, and control the eye movement acquisition component 104 to start and obtain the eye movement data of the target object collected by the eye movement acquisition component 104; the eye movement data is the eye movement information obtained by the target object based on a preset eye movement paradigm task corresponding to the virtual image.

[0114] S502. Send the eye movement data to the terminal device 20 so that the terminal device 20 can evaluate the probability that the target object has a preset brain function disease based on the eye movement data.

[0115] In some embodiments, when a light-shielding member is installed on the glasses frame 101 of the glasses-type eye movement tracking device 10, controlling the two display screens 102 to display virtual images includes: in the virtual reality mode, when the two display screens 102 are shielded by the light-shielding member, controlling the two display screens 102 to display virtual images; or, When the spectacle frame 101 of the glasses-type eye tracking device 10 is not equipped with a light-shielding member and the spectacle frame 101 is provided with a scene camera 109, controlling the two display screens 102 to display virtual images includes: in the augmented reality mode, starting the scene camera 109 and controlling the two display screens 102 to display virtual images.

[0116] Optionally, an embodiment of the present application further provides an eye tracking method, including the following steps: (1) When it is detected that the target object wears the spectacle frame 101, start the display screen 102, the near-infrared light source 1041, and the near-infrared camera 1042.

[0117] (2) Calibrate the eye movement of the target object, for example: start the 5-point or 9-point eye movement calibration.

[0118] (3) Control the target object to perform a preset eye movement paradigm task, and collect eye movement data through the near-infrared camera 1042. The eye movement data includes various eye movement parameters and parameter values corresponding to the eye movement parameters.

[0119] (4) Based on the eye movement data, evaluate the probability that the target object suffers from a preset brain function disease, and output an auxiliary diagnosis conclusion of various neurodegenerative diseases of the target object based on the preset eye movement paradigm task.

[0120] It should be understood that although the flowchart of the embodiment of the present application indicates each operation step by an arrow, the execution order of these steps is not limited to the order indicated by the arrow. Unless there is a clear description in this article, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage among these sub-steps or stages can also be executed at different times respectively. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiment of the present application does not limit this.

[0121] In the embodiment of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit including the function of the module or unit.

[0122] The above are only optional implementation manners of some implementation scenarios of the present application. It should be noted that for those of ordinary skill in the art, without departing from the technical concept of the solution of the present application, adopting other similar implementation means based on the technical idea of the present application also belongs to the protection scope of the embodiments of the present application.

Claims

1. A glasses-type eye tracking device, characterized in that: include: A glasses frame and two display screens, wherein the two display screens are mounted on the glasses frame and are used to display virtual images in front of the eyes of the target object; An eye movement acquisition component, provided on the glasses frame, for acquiring eye movement data of the target object; A processor is provided in the eyeglass frame and connected to the eye movement acquisition component. When it is determined that the target object wears the eyeglass frame, the processor controls the two display screens to display the virtual image and controls the eye movement acquisition component to start and obtain the eye movement data of the target object. The eye movement data is the eye movement information obtained by the target object based on a preset eye movement paradigm task corresponding to the virtual image and is used to evaluate the probability that the target object suffers from a preset brain functional disease.

2. The eye tracking device of claim 1, wherein: The eye movement acquisition component comprises: at least one light source, used to irradiate light onto the cornea of ​​the target object to form a reflective light spot on the cornea; At least one camera module is used to collect pupil change information of the target object's eyes and change information of the reflective light spot, and generate the eye movement data based on the pupil change information and change information of the reflective light spot.

3. The eye tracking device of claim 1, wherein: Also includes: The integrated sensor module is arranged on the eyeglass frame, and is used for collecting the state information of the target object wearing the eyeglass frame, and sending the state information to the processor, so that the processor performs corresponding operations based on the state information.

4. The eye tracking device of claim 3, wherein: The integrated sensor module includes at least one of the following: An inertial measurement unit sensor, including an accelerometer, a gyroscope and a magnetometer, for collecting motion information of the target object; the state information includes the motion information; The proximity light sensor is used to generate wearing information and send the wearing information to the processor when detecting that the distance between the target object and the proximity light sensor is not greater than a predetermined distance, so that the processor determines that the target object wears the eyeglass frame based on the wearing information.

5. The eye tracking device of claim 2, wherein: At least one of the light sources is two near-infrared light sources, and at least one of the camera modules is two near-infrared cameras; The glasses frame comprises two edge portions respectively arranged below the two display screens; One of the near-infrared light sources and one of the near-infrared cameras are arranged as a group on one of the edge portions.

6. The eye-tracking device of claim 1, wherein: Also includes: A diopter adjustment button is provided on the eyeglass frame and located above the display screen, and is used to change the distance and / or angle between the optical elements of the display screen when the diopter adjustment button is rotated, so as to adjust the diopter.

7. The eye-tracking device of claim 1, wherein: Also includes at least one of the following: A scene camera, provided on the glasses frame and located in a middle area of ​​the glasses frame; the middle area is located between the two display screens; at least one speaker and a volume control button, both disposed at the temples of the eyeglass frame; Two electro-optical lenses and a light-transmitting button, each of the electro-optical lenses is correspondingly arranged on a side of the display screen close to the target object, and the light-transmitting button is arranged on the temple of the glasses frame.

8. The eye tracking device of claim 1, wherein: Also includes at least one of the following: two nose pads, detachably mounted on the eyeglass frame; The shading member is detachably mounted on the eyeglass frame and is used to be arranged in front of the display screen.

9. The eye-tracking device of claim 1, wherein: Also includes: The power supply and data interface is arranged at the end of the temple of the glasses frame and is used for connecting to an external power supply or a terminal device.

10. The eye tracking device of claim 1, wherein: The predetermined brain functional disease includes at least one of the following: Cognitive dysfunction, Parkinson's disease, schizophrenia, anxiety, depression.

11. An eye tracking system, characterized in that: It comprises a terminal device and a glasses-type eye tracking device as claimed in any one of claims 1 to 10; The terminal device is connected to the eyeglass-type eye tracking apparatus, and the terminal device is used to evaluate the probability that the target object suffers from a preset brain functional disease based on eye movement data.

12. An eye tracking method, characterized in that: Applied to the eyeglass-type eye tracking device according to any one of claims 1 to 10, the method comprising: When it is determined that the target object wears the glasses frame, the two display screens are controlled to display the virtual image, and the eye movement acquisition component is controlled to start and acquire the eye movement data of the target object acquired by the eye movement acquisition component; the eye movement data is the eye movement information of the target object obtained based on a preset eye movement paradigm task corresponding to the virtual image; The eye movement data is sent to a terminal device, so that the terminal device evaluates the probability that the target object suffers from a preset brain functional disease based on the eye movement data.

13. The eye tracking method according to claim 12, characterized in that: In the case where the glasses frame of the glasses-type eye tracking device is equipped with a shading member, controlling the two display screens to display virtual images includes: in a virtual reality mode, when the two display screens are shielded by the shading member, controlling the two display screens to display virtual images; or When a visor frame of the spectacles-type eye tracking device is not equipped with a shading piece and the spectacles frame is provided with a scene camera, two display screens are controlled to display a virtual image, including: in an augmented reality mode, starting the scene camera and controlling the two display screens to display a virtual image.

Images