Positioning and vision expansion method based on eye tracking principle and smart glasses

Abstract

The present invention discloses a positioning and visual field expansion method based on the eye tracking principle and smart glasses. The smart glasses include a glasses body worn on the head and an intelligent visual aid system integrated in the glasses body. The intelligent visual aid system includes an environment picture capturing module, an imaging module, an eye tracking module, a data processing module, a human-computer interaction module and a power supply module. The environment picture capturing module includes a wide-angle camera for capturing environment pictures; the imaging module includes a graphics processor and a waveguide display screen, and the waveguide display screen is used to display the environment pictures; the eye tracking module realizes line of sight capture through eye tracking technology, captures the environment pictures in front of the eyes through the wide-angle camera, compresses the pictures to a corresponding degree according to the wearer's field of view and displays them in front of the patient, and selects and enlarges a specific area through the eye tracking technology so that the wearer can accurately observe the details of each part in the picture.

Description

Technical Field

[0001] The present invention relates to the field of vision assistance technology, and in particular to a positioning and visual field expansion method based on the eye tracking principle and smart glasses. Background Art

[0002] Low vision refers to the severe impairment of the best corrected visual acuity of the better eye of both eyes, which is generally between 0.3-0.05. When the best corrected visual acuity of the better eye of both eyes is ≤0.05, it is called blindness; when the patient's visual field is impaired, resulting in a visual field of ≤10°, it also becomes blind. The visual function of low vision patients and blind patients is severely impaired, but there is still the potential to use the residual visual function to live and work. In order to help low vision patients and blind patients, people in this field are committed to the research of visual function assistive devices, among which a visual aid is a device or equipment that can improve or enhance the functional vision of low vision patients. Similar to a hearing aid, a hearing aid can enable a person with poor hearing to hear sounds that he could not hear before, while a visual aid can enable low vision and blind patients to see things that they could not see or see clearly before, so that they can better return to normal society.

[0003] In terms of design principles, existing visual aids generally include optical visual aids, non-optical visual aids and some non-visual auxiliary equipment. Optical visual aids use the magnifying effect of the convex lens or optical system of this type of visual aid to enlarge the image of the object, so that patients with severe visual impairment (or low vision and blindness) can see clearly very easily; or compress the field of view, so that patients with severe narrowing of the field of vision can expand their field of vision and easily see the surrounding environment. Electronic visual aids are typical representatives of non-optical visual aids and are also the future development direction of visual aids. The main components of electronic visual aids are cameras, processors and displays. The camera captures the image in front of the patient's eyes, and then it is magnified or reduced and displayed on the display in front of the patient's eyes, solving the problem of patients not being able to see clearly.

[0004] Functionally, visual aids are divided into those with a "magnification" function and those with a "field of vision expansion" function. However, the majority of visual aids on the market currently have a "magnification" function, while there are very few visual aids with a field of vision expansion function. In addition, existing electronic field of vision expansion equipment has an inherent defect, that is, the image clarity after widening the field of vision is significantly reduced. In the case of a severely narrowed field of vision, it is often difficult for patients to distinguish the details of a certain part of the image after the field of vision is expanded. On this basis, a magnification function needs to be incorporated to meet the requirements of seeing details. However, for the specific location that the patient needs to magnify, very accurate positioning is required to meet the requirements. Otherwise, after magnification, the field of vision is narrowed again, and the target is often unable to be found. This makes the precise positioning function of the electronic equipment for field of vision expansion particularly important.

[0005] The existing positioning methods used in visual aids include electrode method and contact lens method. The electrode method is to attach electrodes around the eyeball to obtain eye movement data information. However, this method has the defect of large data variability resulting in poor positioning accuracy; and the contact lens method is to obtain the electrooculogram through a contact lens directly attached to the cornea, but this method still has the defects of large variability and invasiveness, as well as a high risk of infection and damage to the cornea; in addition, the Chinese invention patent published on September 14, 2021 (publication number: CN113391449A) discloses a smart field of view expansion visual aid and a smart field of view expansion visual aid method, which adopts a positioning method that is not based on the eyeball. This method selects the desired enlarged area by manual or voice control selection. It is a rough positioning method and has the problem of low accuracy.

[0006] Therefore, it is necessary for the field to solve the positioning defects and the conflict between expanding the field of view and maintaining the resolution of the visual aid technology. Summary of the invention

[0007] The present invention proposes a pair of eye-tracking, positioning and field-expanding intelligent vision-aiding glasses, which capture the environmental image in front of the eyes through a wide-angle camera, compress it to a corresponding degree according to the wearer's field of view and display it in front of the patient's eyes, and select and magnify a specific area through eye-tracking technology so that the wearer can accurately observe the details of each part of the image.

[0008] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions:

[0009] A pair of eye-tracking, positioning and field-expanding intelligent vision-aiding glasses, comprising a glasses body worn on the head and an intelligent vision-aiding system integrated in the glasses body, wherein the intelligent vision-aiding system comprises an environment picture capturing module, an imaging module, an eye-tracking module, a data processing module, a human-computer interaction module and a power supply module, wherein the glasses body comprises a wearing mechanism and a main body located in front of the eyes, wherein the main body comprises an integrated left eye part and a right eye part, wherein:

[0010] Environmental image capture module: includes at least one wide-angle camera, which is fixedly arranged on the main body or the wearing mechanism, and faces directly in front of the main body, which is consistent with the direction in which the eyeballs look forward when wearing the glasses.

[0011] Imaging module: including a graphics processor and a waveguide display screen, wherein the waveguide display screen is arranged inside the main body, and an observation window is provided on the side of the main body facing the eyeball for the line of sight to pass through and observe the image on the waveguide display screen;

[0012] Eye tracking module: including an endoscope camera and several LED near-infrared light sources, wherein the endoscope camera is arranged in the main body, and the shooting light path of the endoscope camera passes through the observation window to shoot the pupil of the eyeball, and the LED near-infrared light source is also arranged in the main body, and the LED near-infrared light source illuminates the eyeball through the observation window so that the endoscope camera can shoot the pupil grayscale image;

[0013] Data processing module: a control circuit board built into the main body of the glasses, on which a central processing unit is integrated, and the graphics processor is also integrated on the control circuit board, and the wide-angle camera, waveguide display, endoscope camera and LED near-infrared light source are all connected to the control circuit board circuit;

[0014] Human-computer interaction module: comprising a control terminal for inputting a control signal to the control circuit board, wherein the control terminal may be a remote controller or a voice controller;

[0015] Power supply module: uses a rechargeable battery, which is integrated into the glasses body and electrically connected to the control circuit board to power each module.

[0016] Preferably, the environmental image capturing module includes two wide-angle cameras, which are respectively arranged at the left eye and the right eye, and the two wide-angle cameras are symmetrically arranged about the midline of the left eye and the right eye.

[0017] Preferably, the endoscope camera is a medical-grade endoscope camera, and six LED near-infrared light sources are distributed at equal angles on the periphery of the endoscope camera, and the LED near-infrared light sources use near-infrared light with a wavelength of 750nm.

[0018] The present invention also discloses a method for positioning and enlarging a gaze area based on the eye tracking principle. The method is based on the eye tracking positioning and field of view expansion smart vision-aiding glasses of any of the above-mentioned solutions, and comprises the following steps:

[0019] 7) Wearing the glasses body, fixing the main body in front of the eyes, making the observation window face the wearer's pupils, and making the waveguide display screen be located in front of the wearer's observation angle;

[0020] 8) Preparatory work for establishing eye tracking and positioning function:

[0021] 2.0) Learning: Pre-learn a large number of eye images through the convolutional neural network algorithm. Eye images can be obtained from some public data sets, such as the human pupil contour images published by SWIRSKI and the ImageNet database;

[0022] 2.1) Calibration: Only performed when used for the first time, nine calibration marks at fixed positions are displayed on the waveguide display screen. Each calibration mark consists of three black concentric circles, and the nine calibration marks will flash randomly on the screen. During the calibration process, each calibration mark will appear on the screen for two seconds, and the user needs to stare at the mark until it disappears. At the same time, the endoscope camera captures the eye image during the calibration process and calibrates the eye movement related parameters, including the angle between the optical axis and the visual axis, the distance k from the pupil to the center of the eyeball, the corneal curvature, the corneal diameter, and the axial length of the eye.

[0023] 9) Obtaining real-time observation images: using a wide-angle camera to capture real-time environmental images, and according to the narrowing of the wearer's field of vision, compressing the captured real-time environmental images and displaying them on the waveguide display screen;

[0024] 10) Select the zoomed area through eye tracking

[0025] 4.0) Obtaining grayscale images of the eyeball: Using an endoscope camera to capture the wearer's eyeball in real time to obtain grayscale images and obtain eye movement related parameters, including the angle between the optical axis and the visual axis, the distance k from the pupil to the center of the eyeball, corneal curvature, corneal diameter, and eye axis length;

[0026] 4.1) Pupil center calculation: The grayscale image acquired in real time in step 3) is used to obtain the pupil contour and pupil center through a convolutional neural network calculation method, and the erroneous position caused by blinking is removed based on the data obtained by calibration;

[0027] 4.2) Eyeball center calculation: The eyeball shape is simulated and calculated through the eye movement related parameters obtained during calibration and the convolutional neural network calculation of the grayscale image, thereby obtaining the eyeball center;

[0028] 4.3) Obtaining the gaze position: A three-dimensional gaze estimation model is established according to the geometric position between the eye and the waveguide display screen, and a geometric mapping between the human eye and the gaze position of the human eye on the waveguide display screen is realized through the established three-dimensional gaze estimation model. The establishment of the three-dimensional gaze estimation model includes obtaining the optical axis and obtaining the gaze, wherein:

[0029] a) Optical axis acquisition: The line connecting the center of pupil and the center of eyeball is the optical axis of eyeball;

[0030] b) Obtaining sight line: Through kappa angle correction, the optical axis plus the kappa angle is the visual axis direction. Before use, the kappa angle is detected and input into the data processing module, and the visual axis direction is automatically calculated. The intersection of the visual axis and the image displayed by the waveguide display is the gaze position of the human eye;

[0031] 4.4) In the waveguide display, a regional selection mark is formed with the wearer's visual axis as the center. The selection mark moves synchronously with the wearer's visual axis, and the selection mark reflects the gaze position of the human eye in real time;

[0032] 11) Confirm that the selected area is marked as the enlarged area

[0033] 5.0) Input the amplification signal by manual remote control or voice control. After the control circuit board receives the amplification signal, the image within the selected area mark is locked;

[0034] 5.1) The image within the selected area is enlarged;

[0035] 12) Input the end amplification signal by manual remote control or voice control. After the control circuit board receives the end amplification signal, the amplified image area disappears, the image within the selection mark is unlocked, and the selection mark continues to move synchronously with the visual axis. At this time, the area can be reselected.

[0036] Preferably, in the above step 3), the captured real-time environment picture is compressed in different levels, specifically:

[0037] Level 1: compressed to 10 degrees of visual field;

[0038] Level 2: compressed to 15 degrees of visual field;

[0039] Level 3: Compressed to 20 degrees of field of view.

[0040] Preferably, in the above step 5.0), after the image in the selection area mark is locked, the magnification ratio can be selected by manual remote control or voice control, and the magnification ratio selection is designed to be 2-6 times.

[0041] Compared with the prior art, the eye tracking positioning and visual field expansion smart vision glasses using the above technical solution have the following beneficial effects:

[0042] 1. Wear the eyeglass body 1 on the head, capture the environment in front of the wearer through a wide-angle camera, and compress the captured image through a graphics processor to adapt to the wearer's field of view, so that patients with symptoms of narrowed field of view can also easily see the surrounding environment clearly. The wearer can actively select the local area he wants to observe to enlarge it. Since there is no need to enlarge the image as a whole, the basic image captured by the camera (i.e., the image before enlargement) does not need to be compressed to a large extent, and the clarity of the image can be guaranteed to a large extent, so that the image of the enlarged area can also ensure sufficient clarity, which is conducive to patients seeing and recognizing clearly.

[0043] Second, the real-time positioning of the target magnification area is achieved by adopting eye tracking technology. The target magnification area is the area where the eyeball is looking at the waveguide display screen. Through eye tracking, the waveguide display screen only displays the regional selection mark in the looked area. The selection mark can move in real time with the movement of the gaze position, so as to locate the gaze position of the human eye in real time and accurately, so as to carry out magnified observation of the target position.

[0044] 3. Through neural convolutional deep learning, data is obtained from public data sets (such as the human pupil contour images published by SWIRSKI) to pre-learn a large number of glasses images, and then the wearer's eye grayscale images are obtained through the endoscope camera. The pupil center and eye center are calculated by the central processor, and a three-dimensional line of sight estimation model is established to finally obtain accurate line of sight, which has the advantages of more accurate data and fewer influencing confounding factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic diagram of the structure of an embodiment of the eye-tracking, positioning and field-expanding intelligent vision-aiding glasses of the present invention.

[0046] Figure 2 Schematic diagram of the structure of the glasses body in this embodiment (inner perspective).

[0047] Figure 3 It is a side view of the main body of the glasses in this embodiment.

[0048] Figure 4 This is a diagram of a method for positioning and enlarging a gaze area based on the eye tracking principle in this embodiment.

[0049] Figure 5 and Figure 6 It is an enlarged schematic diagram of the positioning area of ​​this embodiment.

[0050] Figure numerals: 1. Glasses main body; 10. Wearing mechanism; 11. Main body; 110. Left eye; 111. Right eye; 12. Observation window; 2. Wide-angle camera; 3. Waveguide display; 4. Endoscopic camera; 5. LED near-infrared light source. DETAILED DESCRIPTION

[0051] The present invention is further described below in conjunction with the accompanying drawings.

[0052] like Figures 1 to 3The eye tracking positioning and field of vision expansion intelligent vision glasses shown in the figure include a glasses body 1 worn on the head and an intelligent vision system integrated in the glasses body 1. The intelligent vision system includes an environment picture capture module, an imaging module, an eye tracking module, a data processing module, a human-computer interaction module and a power supply module. The glasses body 1 includes a wearing mechanism 10 and a main body 11 located in front of the eyes. The main body 11 includes an integrated left eye part 110 and a right eye part 111.

[0053] Environmental image capture module: includes two wide-angle cameras 2, which are fixedly mounted on the main body 11 or the wearing mechanism 10. The two wide-angle cameras 2 are both facing the front of the main body 11 and are respectively arranged at the left eye 110 and the right eye 111, which is consistent with the direction of the eyeballs looking forward when wearing the glasses body 1. The two wide-angle cameras 2 are symmetrically arranged about the midline of the left eye 110 and the right eye 111.

[0054] The imaging module includes a graphics processor and a waveguide display screen 3 . The waveguide display screen 3 is arranged inside a main body 11 . An observation window 12 is provided on a side of the main body 11 facing the eyeball for the line of sight to pass through and observe the image on the waveguide display screen 3 .

[0055] Eye tracking module: includes an endoscope camera 4 and six LED near-infrared light sources 5. The endoscope camera 4 adopts a medical-grade endoscope camera. The endoscope camera 4 is arranged in the main body 11, and the shooting light path of the endoscope camera 4 passes through the observation window 12 to shoot the pupil of the eyeball. The LED near-infrared light source 5 adopts near-infrared light with a wavelength of 750nm. The LED near-infrared light source 5 is also arranged in the main body 11. The six LED near-infrared light sources 5 are distributed at equal angles on the periphery of the endoscope camera 4. The LED near-infrared light source 5 irradiates the eyeball through the observation window 12 so that the endoscope camera 4 can shoot the pupil grayscale image.

[0056] Data processing module: It is a control circuit board built into the eyeglass body 1. The control circuit board is integrated with a central processing unit, and a graphics processor is also integrated on the control circuit board. The control circuit board is a programmable control circuit board. As the carrier of the convolutional neural network algorithm, the wide-angle camera 2, the waveguide display screen 3, the endoscope camera 4 and the LED near-infrared light source 5 are all connected to the control circuit board circuit.

[0057] Human-computer interaction module: includes a control terminal for inputting control signals to the control circuit board. The control terminal can be a remote controller or a voice controller.

[0058] Power supply module: A rechargeable battery is used, which is integrated in the eyeglass body 1 and electrically connected to the control circuit board to realize power supply for each module.

[0059] like Figures 4 to 6The method for positioning and enlarging the gaze area based on the eye tracking principle is shown. The method is based on the above-mentioned eye tracking positioning and expanding the field of vision smart glasses, and includes the following steps:

[0060] 1) Wear the glasses body 1, fix the main body 11 in front of the eyes, make the observation window 12 face the wearer's pupils, and make the waveguide display screen 3 be located in front of the wearer's viewing angle;

[0061] 2) Preparatory work for establishing eye tracking and positioning function:

[0062] 2.0) Learning: Pre-learn a large number of eye images through the convolutional neural network algorithm. Eye images can be obtained from some public data sets, such as the human pupil contour images published by SWIRSKI and the ImageNet database;

[0063] 2.1) Calibration: Only performed when used for the first time, nine calibration marks at fixed positions are displayed on the waveguide display screen 3, each calibration mark consists of three black concentric circles, and the nine calibration marks will flash randomly on the screen; during the calibration process, each calibration mark will appear on the screen for two seconds, and the user needs to stare at the mark until it disappears; at the same time, the endoscope camera 4 captures the eye image during the calibration process and calibrates the eye movement related parameters, including the angle between the optical axis and the visual axis, the distance k from the pupil to the center of the eyeball, the corneal curvature, the corneal diameter, and the axial length of the eye;

[0064] 3) Obtaining real-time observation images: The real-time environment images are captured by the wide-angle camera 2. According to the narrowing of the wearer's field of vision, the captured real-time environment images are compressed in different levels and displayed on the waveguide display screen 3. The control signal is input through manual remote control or voice control to select the following compression levels:

[0065] Level 1: compressed to 10 degrees of visual field;

[0066] Level 2: compressed to 15 degrees of visual field;

[0067] Level 3: Compressed to 20 degrees of field of view.

[0068] 4) Select the zoomed area through eye tracking

[0069] 4.0) Obtaining a grayscale image of the eyeball: The wearer's eyeball is photographed in real time by the endoscope camera 4 to obtain a grayscale image, and obtain eye movement related parameters, including the angle between the optical axis and the visual axis, the distance k from the pupil to the center of the eyeball, the corneal curvature, the corneal diameter, and the axial length of the eyeball;

[0070] 4.1) Pupil center calculation: The grayscale image acquired in real time in step 3) is used to obtain the pupil contour and pupil center through a convolutional neural network calculation method, and the erroneous position caused by blinking is removed based on the data obtained by calibration;

[0071] 4.2) Eyeball center calculation: The eyeball shape is simulated and calculated through the eye movement related parameters obtained during calibration and the convolutional neural network calculation of the grayscale image, thereby obtaining the eyeball center;

[0072] 4.3) Obtaining the gaze position: A three-dimensional gaze estimation model is established according to the geometric position between the eye and the waveguide display screen 3. The geometric mapping between the human eye and the gaze position of the human eye on the waveguide display screen 3 is realized by the established three-dimensional gaze estimation model. The establishment of the three-dimensional gaze estimation model includes obtaining the optical axis and obtaining the gaze, wherein:

[0073] a) Optical axis acquisition: The line connecting the center of pupil and the center of eyeball is the optical axis of eyeball;

[0074] b) Obtaining sight line: Through kappa angle correction, the optical axis plus the kappa angle is the visual axis direction. Before use, the kappa angle is detected and input into the data processing module, and the visual axis direction is automatically calculated. The intersection of the visual axis and the image displayed by the waveguide display screen 3 is the gaze position of the human eye;

[0075] 4.4) In the waveguide display screen 3, a regional selection mark is formed with the wearer's visual axis as the center. The selection mark moves synchronously with the wearer's visual axis, and the selection mark reflects the gaze position of the human eye in real time;

[0076] 5) Confirm that the selected area is marked as the enlarged area

[0077] 5.0) Input the amplification signal by manual remote control or voice control. After the control circuit board receives the amplification signal, the image within the selected area mark is locked;

[0078] 5.1) Continue to select the magnification through manual remote control or voice control, designed to be 2-6 times magnification selection, the image within the selected area mark is magnified;

[0079] 6) Input the end amplification signal by manual remote control or voice control. After the control circuit board receives the end amplification signal, the enlarged image area disappears, the image within the selection mark is unlocked, and the selection mark continues to move synchronously with the visual axis. At this time, you can reselect the area.

[0080] In this paper, the following two papers are used as technical references for pre-learning a large number of eye images and establishing a 3D line of sight estimation model through a convolutional neural network algorithm:

[0081] [1]Fuhl W,Santini T,Kasneci G,et al.PupilNet: Convolutional NeuralNetworks for Robust Pupil Detection[J].Revista De Odontologia Da Unesp,2016,19(1):806-821.

[0082] [2]Ji,Jcq. "3D Gaze Estimation with a Single Camera without IRIllumination." International Conference on Pattern Recognition IEEE, 2008.

[0083] The above is a preferred embodiment of the present invention. A person skilled in the art may make several modifications and improvements without departing from the principle of the present invention, and these modifications and improvements shall also be deemed as the protection scope of the present invention.

Claims

1. A positioning and visual field expansion method based on the eye tracking principle, using intelligent vision-aiding glasses, the intelligent vision-aiding glasses comprising a glasses body (1) worn on the head and an intelligent vision-aiding system integrated in the glasses body (1), the intelligent vision-aiding system comprising an environment picture capturing module, an imaging module, an eye tracking module, a data processing module, a human-computer interaction module and a power supply module, the glasses body (1) comprising a wearing mechanism (10) and a main body (11) located in front of the eyes, the main body (11) comprising an integrated left eye portion (110) and a right eye portion (111), The environment image capturing module comprises at least one wide-angle camera (2), the wide-angle camera (2) being fixedly arranged on the main body (11) or the wearing mechanism (10), the wide-angle camera (2) facing directly in front of the main body (11), which is consistent with the direction in which the eyeball looks forward when the eyeglass body (1) is worn; An imaging module comprising a graphics processor and a waveguide display screen (3), wherein the waveguide display screen (3) is arranged inside a main body (11), and an observation window (12) is provided on a side of the main body (11) facing an eyeball, through which a line of sight passes to observe an image on the waveguide display screen (3); The eye tracking module comprises an endoscope camera (4) and a plurality of LED near-infrared light sources (5), wherein the endoscope camera (4) is arranged in a main body (11), and a shooting light path of the endoscope camera (4) passes through an observation window (12) to shoot an eyeball pupil; the LED near-infrared light source (5) is also arranged in the main body (11), and the LED near-infrared light source (5) illuminates the eyeball through the observation window (12) so that the endoscope camera (4) can shoot a pupil grayscale image; the endoscope camera (4) adopts a medical-grade endoscope camera, and six LED near-infrared light sources (5) are distributed at equal angles on the periphery of the endoscope camera (4), and the LED near-infrared light source (5) adopts near-infrared light with a wavelength of 750 nm; The data processing module is a control circuit board built into the eyeglass body (1), the control circuit board is integrated with a central processing unit, the graphics processor is also integrated on the control circuit board, and the wide-angle camera (2), the waveguide display screen (3), the endoscope camera (4) and the LED near-infrared light source (5) are all connected to the control circuit board circuit; Human-computer interaction module: comprising a control terminal for inputting control signals to the control circuit board, wherein the control terminal is a remote controller or a voice controller; A power supply module: using a rechargeable battery, which is integrated into the eyeglass body (1) and electrically connected to the control circuit board to realize power supply for each module; Features: Using the above intelligent visual aid system to locate and enlarge the gaze area includes the following steps: 1) Wearing the eyeglass body (1), fixing the main body (11) in front of the eyes, making the observation window (12) face the pupils of the wearer, and making the waveguide display screen (3) be located in front of the wearer's viewing angle; 2) Preparatory work for establishing eye tracking positioning function: 2.0) Learning: Pre-learning a large number of eye images through a convolutional neural network algorithm. The eye images are obtained from some public data sets, including human pupil contour images published by SWIRSKI; 2.1) Calibration: This is only performed when used for the first time. Nine calibration marks are displayed at fixed positions on the waveguide display (3). Each calibration mark consists of three black concentric circles. The nine calibration marks will flash randomly on the screen. During the calibration process, each calibration mark will appear on the screen for two seconds. The user needs to stare at the mark until it disappears. At the same time, the endoscope camera (4) captures the eye image during the calibration process to obtain eye movement related parameters, including the angle between the optical axis and the visual axis, the distance k from the pupil to the center of the eyeball, the corneal curvature, the corneal diameter, and the axial length of the eye. 3) Obtaining a real-time observation picture: Using a wide-angle camera (2) to capture a real-time environment picture, and according to the narrowing of the wearer's field of vision, compressing the captured real-time environment picture and displaying it on the waveguide display screen (3); compressing the captured real-time environment picture in different levels, specifically: Level 1: compressed to 10 degrees of visual field; Level 2: compressed to 15 degrees of visual field; Level 3: compressed to 20 degrees of visual field; 4) Select the zoomed area through eye tracking 4.0) Obtaining a grayscale image of the eyeball: Using the endoscope camera (4) to photograph the wearer's eyeball in real time to obtain a grayscale image, and obtain eye movement related parameters, including the angle between the optical axis and the visual axis, the distance k from the pupil to the center of the eyeball, the corneal curvature, the corneal diameter, and the axial length of the eyeball; 4.1) Pupil center calculation: The grayscale image acquired in real time in step 3) is used to obtain the pupil contour and pupil center through a convolutional neural network calculation method, and the erroneous position caused by blinking is removed; 4.2) Eyeball center calculation: The eyeball shape is simulated and calculated through the eye movement related parameters obtained during calibration and the convolutional neural network calculation of the grayscale image, thereby obtaining the eyeball center; 4.3) Obtaining the gaze position: A three-dimensional gaze estimation model is established according to the geometric position between the eye and the waveguide display screen (3), and a geometric mapping between the human eye and the gaze position of the human eye on the waveguide display screen (3) is realized through the established three-dimensional gaze estimation model. The establishment of the three-dimensional gaze estimation model includes obtaining the optical axis and obtaining the gaze, wherein: a) Optical axis acquisition: The line connecting the center of pupil and the center of eyeball is the optical axis of eyeball; b) Obtaining sight line: Through kappa angle correction, the optical axis plus the kappa angle is the visual axis direction. Before use, the kappa angle is detected and input into the data processing module, and the visual axis direction is automatically calculated. The intersection of the visual axis and the image displayed by the waveguide display screen (3) is the gaze position of the human eye; 4.4) In the waveguide display screen (3), a regional selection mark is formed with the wearer's visual axis as the center. The selection mark moves synchronously with the wearer's visual axis, and the selection mark reflects the gaze position of the human eye in real time; 5) Confirm that the selected area is marked as the enlarged area 5.0) Input the amplification signal by manual remote control or voice control. After the control circuit board receives the amplification signal, the image in the selection area mark is locked. After the image in the selection area mark is locked, the amplification ratio can be selected by manual remote control or voice control, and the design is 2-6 times amplification ratio selection; 5.1) The image within the selected area logo is enlarged; 6) Input the end amplification signal through manual remote control or voice control. After the control circuit board receives the end amplification signal, the enlarged image area disappears, the image within the selection mark is unlocked, and the selection mark continues to move synchronously with the visual axis. At this time, you can reselect the area.

2. The method for positioning and expanding visual field based on the eye tracking principle according to claim 1, characterized in that: The environment image capturing module comprises two wide-angle cameras (2), the two wide-angle cameras (2) being respectively arranged at the left eye (110) and the right eye (111), and the two wide-angle cameras (2) being symmetrically arranged about the midline of the left eye (110) and the right eye (111).

Images