Dynamic virtual image plane adjustment device and method based on multi-image-plane eye movement tracking

Abstract

A dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking, belonging to the technical field of visual training. The multi-image plane eye movement tracking system uses an infrared camera to capture the user's eye image, processes it to obtain accurate fixation point coordinates, and transmits the data to the control system; the control system receives the fixation point coordinates, analyzes the required image plane depth at present, generates a lens adjustment instruction, and sends it to the lens-screen adjustment system; the lens-screen adjustment system drives the lens and the screen to make physical adjustments, and at the same time uses a sensing feedback mechanism to monitor the adjustment accuracy in real time to ensure that the virtual image position matches the user's fixation focus; the display system combines the depth information provided by the control system to dynamically adjust the animation screen so that it changes synchronously with the user's fixation depth. And a dynamic virtual image plane adjustment method based on multi-image plane eye movement tracking is provided. The present invention realizes the real-time tracking and dynamic adjustment of the user's visual focus, and meets the accurate visual interaction requirements in multi-depth scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual training, and particularly provides a dynamic virtual image plane adjustment device and method based on multi-image plane eye movement tracking. Background Art

[0002] In recent years, virtual reality (VR) and augmented reality (AR) technologies have developed rapidly and have been widely applied in many fields such as medical treatment, education, and entertainment. However, the technical limitations of existing VR / AR devices in the display method are still relatively significant, especially there are many bottlenecks in the focal plane adjustment. Most VR / AR devices on the current market adopt a fixed focal plane design, resulting in the image plane that the user gazes at in the virtual environment always being at the same depth and unable to dynamically adjust the image plane position according to the change of the user's fixation point. This fixed focal length design not only limits the authenticity of the immersive experience but may also cause visual fatigue after long-term use and affect the user's visual health.

[0003] Although some devices have a zoom function, their adjustment methods often rely on manual operation by the user, with a cumbersome process and it is difficult to achieve real-time and intelligent focal length adjustment. For example, some zoom devices require the user to adjust the lens position through a mechanical knob or button, while a few devices with electronic focusing functions rely on preset focusing modes, lacking not only an active feedback mechanism but also a dynamic adjustment function, and it is difficult to meet the personalized needs of users in different scenarios.

[0004] Current eye movement tracking technologies are mainly used in fields such as line-of-sight position analysis and interactive control and have not been applied to dynamic focal length adjustment. The mainstream eye movement tracking methods include pupil positioning technology based on an infrared camera and line-of-sight prediction methods based on machine learning, and both have made great progress in terms of accuracy and real-time performance. However, these technologies have not been effectively combined and applied to optimize the visual experience of virtual reality devices, especially the intelligent application in focal length adjustment is still in its infancy.

[0005] On the other hand, multi-image plane display technology is a method proposed in recent years to relieve visual fatigue. Its core principle is to generate multiple virtual image planes at different depths, enabling users to obtain a more realistic visual experience when viewing virtual objects at different depths. However, the implementation of this technology often relies on complex optical designs, such as multi-layer display screens, optical stratification devices, or dynamic reconstruction systems based on light fields. The application of these solutions in existing devices is limited, mainly because of their complex hardware structure, high computational cost, and there is still much room for optimization in terms of synchronization with the change of the user's line of sight and coordination with the dynamic adjustment of the image plane.

[0006] Regarding the existing related patents, the analysis is as follows:

[0007] Compared with the patented technology CN209014753U (variable-focus lens and VR device): It adjusts the refractive index by changing the thickness of the liquid in the elastic optical cavity to achieve the zoom function. However, its zoom method still relies on passive manual adjustment, requires presetting fixed focal lengths and adjusting the focus through an analog circuit, and lacks active feedback based on the user's real-time gaze information.

[0008] Compared with the patented technology CN208823365U (a variable-focus VR eye vision instrument): It uses a motor drive method to adjust the distance between two lenses to change the optical path between the virtual image plane and the human eye. Although this method can achieve near and far vision adjustment to a certain extent, it still requires manual operation or relies on fixed time interval adjustment and lacks adaptive adjustment to the user's real-time gaze requirements.

[0009] Compared with the patented technology CN114895793A (an active adaptive eye movement tracking method and AR glasses): It uses eye movement tracking to capture the user's fixation point and combines a ToF (Time-of-Flight) sensor to detect the distance of the user's actual observation point to adjust the imaging position and focusing distance of the image module, avoiding visual discomfort caused by frequent changes in the virtual scene. However, this method mainly focuses on the adjustment of the image projection position and does not optimize the focal length adjustment by combining multi-image plane display technology.

[0010] Most of the devices on the market currently adopt a display design with a fixed focal plane. This method is prone to causing visual fatigue and discomfort when the user uses it for a long time. The main reason is that existing VR / AR devices usually only support a single fixed focal plane, cannot dynamically adjust the position of the image plane according to the depth of the user's fixation point, or adjust the focus manually by the user. In this way, the image plane that the user gazes at in the virtual scene is always at the same depth, unable to meet the natural focusing requirements of the line of sight. Prolonged use will cause the lens to remain in a fixed state continuously, increasing the visual burden, unable to meet the user's usage experience, thus causing visual fatigue and having an adverse impact on eye health. Summary of the Invention

[0011] In order to overcome the deficiencies of the prior art, the present invention provides a dynamic virtual image plane adjustment device and method based on multi-image plane eye movement tracking. Through the cooperation of multiple systems, it realizes the real-time tracking and dynamic adjustment of the user's visual focus, meets the accurate visual interaction requirements in multi-depth scenarios, and thus dynamically adjusts the position of the image plane according to the change of the user's fixation point.

[0012] The technical solution adopted by the present invention to solve its technical problems is:

[0013] A dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking includes a multi-image plane eye movement tracking system, a lens-screen adjustment system, a control system, and a display system.

[0014] The multi-image-plane eye movement tracking system captures the user's eye images using an infrared camera. Through calculations by the image acquisition module, pupil detection module, ellipse fitting module, calibration module, and polynomial fitting module, accurate fixation point coordinates are obtained and the data is transmitted to the control system;

[0015] The control system receives the fixation point information, analyzes the required image plane depth at present, and generates a lens adjustment instruction, which is sent to the lens-screen adjustment system;

[0016] The lens-screen adjustment system physically adjusts the lens and the screen according to the control instruction, and at the same time uses the sensing feedback mechanism to monitor the adjustment accuracy in real time to ensure that the virtual image position matches the user's fixation focus;

[0017] The display system combines the depth information provided by the control system to dynamically adjust the animation screen, making it synchronously change with the user's fixation depth to enhance the visual immersion.

[0018] Furthermore, in the multi-image-plane eye movement tracking system, the image acquisition module obtains the user's eye pupil image by calling an active infrared camera. The pupil detection module is used for calibrating the pupil contour. The ellipse fitting module is used for detecting the pupil center position with the highest confidence. The calibration module performs coordinate transformation in combination with known reference points. The polynomial fitting module optimizes the data and finally converts the pupil data into fixation point coordinates.

[0019] Preferably, the implementation process of the image acquisition module is as follows:

[0020] Step 111, select a suitable short-focus camera such as 940nm and infrared lamp beads;

[0021] Step 112, connect the 940nm short-focus camera and the infrared lamp beads for subsequent acquisition;

[0022] Step 113, adjust the positions of the 940nm short-focus camera and the infrared lamp beads to ensure that the eyes are fully illuminated and avoid pupil detection failure.

[0023] More preferably, the implementation process of the pupil detection module is as follows:

[0024] Step 121, judge whether the pupil detection fails according to the pupil threshold algorithm. If the detection fails, step 113 needs to be repeated to ensure the success of the pupil detection;

[0025] Step 122, convert the input video into video frames and process them frame by frame:

[0026] Crop and scale the input original image to unify the image size and facilitate subsequent processing. By calculating the aspect ratio Current_ratio (current_width / current_height) of the current image and the target aspect ratio Desire_ratio (preset as width / height, for example, width = 580, height = 480), if Current_ratio > Desire_ratio, it indicates that the image is too wide. Then calculate the new width New_width = int(Desire_ratio * Current_ratio), and crop the image at the position offset from the center by loc_w = (current_width - New_width) / / 2 in the width direction of the image to obtain the cropped image cropped_img. If Current_ratio <= Desire_ratio, that is, the image is too high, calculate the new height New_height = int(current_width / Desire_ratio), and crop at the position offset from the center by loc_h = (current_height - New_height) / / 2 in the height direction of the image to get cropped_img = image[loc_h: loc_h + New_height, :]; finally, use the cv2.resize function to resize the cropped image to the target size (width, height);

[0027] Step 123, store binary images similar to three different regions A, B, and C in the image:

[0028] Convert the cropped and scaled image to a grayscale image and apply a strict binary threshold to the grayscale image; by adding the threshold increment to the darkest pixel value Darkest_value , calculate the threshold Threshold = Darkest_value + ; use the cv2.threshold function to binarize the image, set the pixels less than this threshold to white (255), and the rest to black (0) to obtain the image Thresholded_image_1 after strict threshold processing; then, with the darkest point Darkest_point as the center and a side length of create a square mask, and set the pixel values outside this square region to , and perform mask processing on Thresholded_image_1;

[0029] Add the threshold increment to the darkest pixel value Darkest_value , and calculate the threshold Threshold = Darkest_value + . Perform binarization on the grayscale image to obtain Thresholded_image_2, and use the darkest point as the center with a side length of for masking;

[0030] Add the threshold increment to the darkest pixel value Darkest_value , and calculate the threshold Threshold = Darkest_value + . Perform binarization on the grayscale image to obtain Thresholded_image_3; then use the darkest point as the center with a side length of for masking;

[0031] And save Thresholded_image_1, Thresholded_image_2, and Thresholded_image_3;

[0032] Step 124, detect the color threshold ranges of three different regions A, B, and C;

[0033] Step 125, select the one with a smaller average color threshold fluctuation as the most reliable pupil image.

[0034] More preferably, the implementation process of the ellipse fitting module is as follows:

[0035] Step 131, perform dilation on each binary image to enhance the target region:

[0036] For the image after multi-threshold processing and masking, first use morphological dilation operations (such as the cv2.dilate function) to enhance the contour features in the image; then, find the contours in the image through the cv2.findContours function; next, filter the found contours, traverse all contours, calculate the area of each contour area = cv2.contourArea(contour), if area >= pixel (for example, pixel = 10), then further calculate the width w and height h of the bounding rectangle of the contour (through the cv2.boundingRect function), and calculate the aspect ratio length = max(w, h), width = min(w, h), Current_ratio = max(length / width, width / length). If Current_ratio <= ratio_thresh (for example, ratio_thresh = 5), then the contour meets the conditions, and return the contour with the largest area that meets the conditions;

[0037] Step 132, extract the external contours in the dilated image;

[0038] Step 133, filter the contours by area and quantity limits and return the largest contour;

[0039] Step 134, do not perform searches within D pixels of the image edge;

[0040] Step 135, check the brightness every E pixels within the area;

[0041] Step 136, along axis and axis, sample every F pixels, ignoring the boundaries;

[0042] Step 137, update the pixel points with the largest threshold between the 0-255 color channels;

[0043] More preferably, the implementation process of the calibration module is as follows:

[0044] Step 141, select the screen size as the calibration screen, input the screen parameters into the control system and project them into the human eye through the lens;

[0045] Step 142, select 9-12 points evenly covering the screen as the calibration points;

[0046] Step 143, the user wears the device and adjusts the size of the eye image in the picture;

[0047] Step 144, perform autonomous calibration through peripherals such as a keyboard or a handle to ensure that the point being stared at in the current state is the calibration point:

[0048] In the calibration mode, images are obtained in real time through the camera, and the central coordinates (x_eye, y_eye) of the pupil are obtained using the above-mentioned pupil detection algorithm; the current calibration target points are drawn on each frame of the image (displayed in red); when the user presses the space bar, the current pupil coordinates (x_eye, y_eye) and the corresponding screen coordinates (i.e., the screen coordinates (x_screen, y_screen) of the current calibration target point) are recorded into the calibration data dictionary calibration_data. After all the calibration points are recorded, calibration_data is saved to a file (such as calibration_data.json);

[0049] Step 145, adjust the distance between the lens and the screen, so as to change the distance between the image plane and the human eye, and repeat steps 142 to 144;

[0050] Step 146, save the calibrated weight information at different depths.

[0051] Preferably, the implementation process of the polynomial fitting module is as follows:

[0052] Step 151, obtain the pixel coordinates of 9 - 12 calibration points;

[0053] Step 152, output the pixel coordinates of the pupil center from the video frame;

[0054] Step 153, the pixel coordinates in steps 151 and 152 and are split into one-dimensional data;

[0055] Step 154, select a polynomial to perform one-dimensional data fitting on the coordinate points;

[0056] Step 155, the polynomial coefficients in the fitting function;

[0057] Step 156, the polynomial coefficients in the fitting function;

[0058] Step 157, save the fitted axis and the coefficients of the one-dimensional polynomials of the axis:

[0059] Load the pupil coordinate data eye_coords and the screen coordinate data screen_coords from the saved calibration data file. Extract the x component eye_x and y component eye_y of the pupil coordinates, and the x component screen_x and y component screen_y of the screen coordinates. Use functions to perform polynomial fitting on the pupil x coordinates and the screen x coordinates, and the pupil y coordinates and the screen y coordinates respectively, to obtain the fitting coefficients coefficients_x and coefficients_y;

[0060] Step 158, start predicting the fixation point coordinates by inputting the coordinates of the pupil center point;

[0061] Furthermore, the lens-screen adjustment system includes a lens, a screen, a linear slide rail, a micro stepping motor, and a drive circuit. The drive circuit is connected to the micro stepping motor. The moving end of the micro stepping motor is linked with the screen. The screen is slidably mounted on the linear slide rail. The screen and the lens are respectively arranged on the front and back sides of the linear slide rail. Each component of this system is highly integrated. When the fixation point enters the target area, the drive circuit sends a control signal to the micro stepping motor, and the micro drive motor drives the screen to perform a smooth and precise linear displacement along the linear slide rail, thereby adjusting the geometric distance between the lens and the screen. In this solution, the introduction of the micro stepping motor realizes the lightweight design of the structure, and at the same time has an accurate feedback adjustment ability. After receiving the data from the multi-image-plane eye movement tracking system, the system can drive the lens-screen unit to perform dynamic adjustment through the micro stepping motor, so as to ensure the real-time matching of the user's visual focus and the image plane depth.

[0062] The lens-screen adjustment system further includes an adjustment module and a sensing feedback module. The adjustment module is used to select the distance to be adjusted in real time according to the predicted fixation point coordinates. The sensing feedback module is used to drive the micro stepping motor to adjust the distance between the lens and the screen.

[0063] Preferably, the implementation process of the adjustment module is as follows:

[0064] Step 211, select the picture area according to the predicted fixation point coordinates output in step 158;

[0065] Step 212, adjust the pre-set travel range after this area is marked.

[0066] More preferably, the implementation process of the sensing feedback module is as follows:

[0067] Step 221, select a micro stepping motor with an 8mm travel;

[0068] Step 222, apply a PWM wave to control the travel of the micro stepping motor according to different fixation areas.

[0069] The described control system includes a data processing module, a calculation module, and a control instruction module. The data processing module is used to receive the gaze point coordinates from the eye tracking system. The calculation module is used to analyze the required image plane depth at present. The control instruction module is used to send motion instructions to the lens-screen adjustment system.

[0070] Preferably, the implementation process of the data processing module is as follows:

[0071] Step 311: Receive the pupil center coordinates output by the eye tracking system;

[0072] Step 312: Receive and analyze the calibrated gaze point coordinates;

[0073] Step 313: Based on the pupil center coordinates and the calibrated gaze point coordinates, perform coordinate correction and filtering to remove noise and improve data accuracy;

[0074] Step 314: Store the corrected gaze point coordinates and transmit the data to the calculation module;

[0075] More preferably, the implementation process of the calculation module is as follows:

[0076] Step 321: Receive the gaze point coordinate information from the data processing module;

[0077] Step 322: Based on the gaze point-depth mapping model preset in the system, calculate the required image plane depth at present;

[0078] Step 323: Detect outliers in the calculation result and perform interpolation processing to optimize data smoothness;

[0079] Step 324: Store the calculated depth information and send it to the control instruction module;

[0080] Even more preferably, the implementation process of the control instruction module is as follows:

[0081] Step 331: Receive the image plane depth data transmitted by the calculation module;

[0082] Step 332: Generate control instructions for the stepper motor according to the depth requirement, including forward, backward, and stroke length;

[0083] Step 333: Use PWM (pulse width modulation) signals to control the moving step size and speed of the micro stepper motor;

[0084] Step 334: Real-time monitor the stepper motor status to ensure accurate adjustment of the lens position;

[0085] Step 335: Feedback the execution result and make secondary adjustments if necessary to improve the system response accuracy.

[0086] The display system includes an animation preprocessing module and a fixation point sensing module. The animation preprocessing module is used to preset the virtual image depth corresponding to different regions in the picture in advance; the fixation point sensing module is used to ensure that the picture update is synchronized with the lens adjustment.

[0087] Preferably, the implementation process of the animation preprocessing module is as follows:

[0088] Step 411: Set the depth information in the picture according to the depth information in real life;

[0089] Step 412: Preset the depth information for different regions of each frame of the animation;

[0090] Step 413: Preset different depth information for the pictures at different depths;

[0091] Step 414: Set the preset depth information as the feedback terminal;

[0092] Preferably, the implementation process of the fixation point sensing module is as follows:

[0093] Step 421: Set the area threshold range;

[0094] Step 422: Preset different area threshold ranges for the pictures at different depths;

[0095] Step 423: When the fixation point moves to the area threshold, generate feedback information to drive the micro stepping motor in Step 222 to achieve feeding.

[0096] A dynamic virtual image plane adjustment method based on multi-image plane eye movement tracking includes the following steps:

[0097] Step 1: The user wears the device: The user correctly wears the eye movement tracking device, ensures that the device fits firmly on the face, and adjusts the position so that the camera can clearly capture the user's eye area; Self-check whether the user's pupils are completely visible and adjust the infrared light intensity;

[0098] Step 2: Eye movement calibration: Enter the calibration mode, guide the user to fixate on multiple preset calibration points in sequence to establish the user's eye movement model; By calculating the mapping relationship between the pupil center offset and the screen coordinates, accurately fit the user's personalized eye movement parameters; During the calibration process, dynamically adjust the parameters according to the sampling error;

[0099] Step 3, Fixation Point Prediction: In the normal operation mode, continuously collect the user's eye movement data, including pupil position and line of sight direction; based on the established eye movement model, calculate and predict the user's current fixation point coordinates in real time, and determine the specific target area of the line of sight on the screen; this predicted data is used to drive the subsequent adaptive vision adjustment mechanism;

[0100] Step 4, Fixation Point Sensing: Further accurately measure the user's real-time fixation point and correct the error of the predicted data; detect the minute movements of the user's eyeballs and dynamically adjust the calculation accuracy of the fixation point to cope with the eye movement characteristics of different users;

[0101] Step 5, Lens-Screen Distance Adjustment: When it is confirmed that the user's fixation point is within the threshold range of the area set by the fixation point sensing module, the drive circuit generates a drive signal and transmits it to the micro stepping motor, and the micro stepping motor precisely moves the screen along the linear slide rail to change the geometric distance between the screen and the lens, so as to adjust the visual focal length and thus match the user's current line of sight requirements.

[0102] The technical concept of the present invention is as follows: By combining eye tracking technology with a multi-image plane dynamic adjustment scheme, the present invention proposes an intelligent and real-time responsive virtual image plane adjustment device. This device can dynamically adjust the lens position according to the depth information of the user's fixation point, enabling intelligent focusing of the virtual image plane at different depths, thereby effectively alleviating visual fatigue, enhancing the immersive experience, and providing a more advanced solution for the focal length adjustment of future VR / AR devices. Compared with the prior art CN208823365U, the present invention combines eye tracking with multi-image plane dynamic adjustment technology, making the focal length adjustment process more natural and intelligent, and can effectively alleviate visual fatigue; compared with the prior art CN114895793A, on the basis of the existing technology, the present invention further proposes a virtual image plane dynamic adjustment scheme combining multi-image plane eye tracking technology, enabling the system to dynamically adjust the relative position between the lens and the screen according to the depth information of the user's fixation point, achieving more accurate focal length control.

[0103] The present invention can achieve the effect of dynamically adjusting the depth of the virtual image plane by the multi-image plane eye tracking module to continuously sense the user's line of sight position, combining the lens-screen adjustment module to dynamically adjust the optical path length, and cooperating with the efficient data processing of the control system module and the depth image presentation of the display system module, providing a more comfortable and natural visual experience for the user.

[0104] The beneficial effects of the present invention are mainly manifested in:

[0105] 1) Fusion innovation of optics and ophthalmology: Based on the accommodation principle of the lens, this invention integrates with eye tracking. According to the requirements of ophthalmic medicine and basic optical knowledge, it ingeniously combines optics and ophthalmology efficiently, and independently designs a dynamic virtual image plane adjustment device and method based on multi-image plane eye tracking.

[0106] 2) Multi-image plane eye tracking: Multi-image plane eye tracking can meet the user's eye tracking function at different image plane depths. Compared with traditional VR / AR devices that usually perform eye tracking only on a single image plane, it is impossible to achieve multi-image plane eye tracking.

[0107] 3) Animation depth setting: Set the depth information corresponding to different positions in advance in each frame of the animation, and jointly implement a feedback mechanism with the multi-image plane eye tracking system to help users achieve a more realistic interaction scenario.

[0108] 4) Active feedback: By judging the position of the eye fixation point, the physical distance between the lens and the screen is changed in real time to change the distance between the virtual image plane and the human eye, without manual adjustment, to achieve active eye movement feedback. Description of the Drawings

[0109] Figure 1 is a principle block diagram of a dynamic virtual image plane adjustment device based on multi-image plane eye tracking;

[0110] Figure 2 is a flow schematic diagram of a dynamic virtual image plane adjustment method based on multi-image plane eye tracking;

[0111] Figure 3 is an optical path schematic diagram of a multi-image plane eye tracking system;

[0112] Figure 4 is a preset depth schematic diagram of the display system;

[0113] Figure 5 is a front structure schematic diagram of a dynamic virtual image plane adjustment device based on multi-image plane eye tracking in the present invention;

[0114] Figure 6 is a back structure schematic diagram of a dynamic virtual image plane adjustment device based on multi-image plane eye tracking in the present invention.

[0115] Among them, 101 is the screen, 102 is the lens 102, 103 is the human eye, 104 is the active infrared camera, 105 is the partition, 106 is the micro stepping motor, 201 is the near vision cylindrical obstacle, 202 is the far vision cylindrical obstacle, 203 is the sun, 204 is the cloud, 301 is the housing, 302 is the linear slide rail, 303 is the lens frame, 304 is the card slot, 305 is the active infrared light source, 401 is the screen frame. Detailed implementation mode

[0116] The present invention will be further described below with reference to the accompanying drawings.

[0117] Refer to Figures 1 to 6 , a dynamic virtual image plane adjustment device combining multi-image plane eye movement tracking technology, which can detect the position of the user's fixation point in real time by presetting the depth information at different positions in the display animation, dynamically adjust the distance between the lens and the screen, change the position of the virtual image plane, and achieve active feedback without the need for the user to manually adjust, thereby relieving visual fatigue and improving the user experience.

[0118] Figure 1 is the principle block diagram of the dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking in the present invention, as Figure 1 shown, the dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking includes a multi-image plane eye movement tracking system, a control system, a lens-screen adjustment system, and a display system.

[0119] The optical path of the multi-image plane eye movement tracking system is as Figure 3 shown. The light emitted by the screen 101 passes through the lens 102 and then enters the human eye 103, and the active infrared camera 104 collects the eye image for multi-image plane eye movement tracking. Among them, the partition 105 is used to ensure binocular dichoptic vision, and the distance between the screen and the lens is changed by the micro stepping motor 106 according to the predicted position of the fixation point, so as to change the distance between the image generated by the light emitted by the screen and the human eye, and can more realistically simulate the real scene.

[0120] The multi-image plane eye movement tracking system includes an image acquisition module, a pupil detection module, an ellipse fitting module, a calibration module, and a polynomial fitting module, where:[[]]

[0121] The image acquisition module is used to obtain high-resolution images of the user's eyes to ensure the accuracy of subsequent processing. The implementation process is as follows:[[]]

[0122] Step 111, select a suitable 940nm short-focus camera and infrared light source as the data acquisition device to enhance the contrast of pupil detection;[[]]

[0123] Step 112, correctly connect the 940nm short-focus camera and the infrared light source to ensure the stable operation of the acquisition system;[[]]

[0124] Step 113, optimize the positions of the camera and the infrared light source to fully illuminate the eye area, avoid pupil detection failure, and improve the quality of data acquisition;[[]]

[0125] The pupil detection module is responsible for identifying the pupil area from the collected images, extracting the pupil edge information, and initially determining the pupil center position. The implementation process is as follows:[[]]

[0126] Step 121: Evaluate the validity of the current detection result based on the pupil threshold algorithm. If the detection fails, the camera and light source need to be adjusted (repeat the environmental adjustment step 113) to ensure a stable pupil image is obtained;

[0127] Step 122: Convert the input video stream into independent video frames and perform frame-by-frame analysis on each frame;

[0128] Step 123: Binarize three different regions A, B, and C in the image and store the binary image to enhance pupil features;

[0129] Step 124: Detect the color threshold ranges of regions A, B, and C and analyze their changing trends;

[0130] Step 125: Select the region with the smallest fluctuation in color threshold among regions A, B, and C as the most reliable pupil image to improve the stability of detection;

[0131] Ellipse fitting module: Based on the pupil detection result, perform ellipse fitting on the pupil region to optimize the positioning accuracy of the pupil center. The implementation process is as follows:

[0132] Step 131: Perform dilation processing on the binarized image to enhance the visibility of the target region;

[0133] Step 132: Extract the external contour from the dilated image to obtain the candidate pupil region;

[0134] Step 133: Based on the contour area and quantity features, select the largest contour that best matches the pupil shape;

[0135] Step 134: Do not perform pupil search in the region within D pixels of the image edge to reduce noise interference;

[0136] Step 135: Perform brightness detection every E pixels in a specific region to enhance the robustness of pupil detection;

[0137] Step 136: Sample every F pixels along the X-axis and Y-axis, ignoring boundary noise to improve calculation efficiency;

[0138] Step 137: Update the pixel points with the largest threshold change within the 0 - 255 color range to improve the accuracy of pupil detection;

[0139] Calibration module: Used to establish the mapping relationship between the pupil center and the screen fixation point to ensure the accuracy of eye movement tracking. The implementation process is as follows:

[0140] Step 141: Select the screen size as the calibration screen and input its parameters into the control system to ensure that the lens projection imaging meets the user's visual requirements;

[0141] Step 142: Uniformly arrange 9 - 12 calibration points on the screen to ensure uniform data distribution and improve the calibration accuracy;

[0142] Step 143: After the user wears the device, adjust the size of the eye image on the screen to meet the calibration requirements;

[0143] Step 144: The user performs calibration through an external device (keyboard or joystick) to ensure that the current fixation point corresponds correctly to the calibration points;

[0144] Step 145: Adjust the distance between the lens and the screen, thereby changing the distance between the image plane and the human eye, and repeat steps 142 to 144;

[0145] Step 146: Record the calibration weight information under different depth conditions to provide reference data for subsequent eye movement tracking;

[0146] The polynomial fitting module is responsible for fitting the mathematical relationship between the pupil center coordinates and the screen fixation points, constructing a mathematical model for predicting the user's fixation position, and the implementation process is as follows:

[0147] Step 151: Obtain the pixel coordinates of 9 - 12 calibration points ;

[0148] Step 152: Output the pixel coordinates of the pupil center from the video frame ;

[0149] Step 153: Split the pixel coordinates in steps 151 and 152 and into one - dimensional data;

[0150] Step 154: Select the polynomial order and fit the coordinate point data to construct an optimal mapping function;

[0151] Step 155: Based on the polynomial coefficients trained with the data, ensure the conversion accuracy in the X - axis direction; ;

[0152] Step 156: Calculate the polynomial coefficients to ensure the mapping accuracy in the Y - axis direction;

[0153] Step 157: Save the coefficients of the one - dimensional polynomials of the axis and axis that are fitted;

[0154] Step 158: During actual operation, input the real-time detected pupil center coordinates, and use the fitting function to predict the fixation point coordinates on the screen to achieve high-precision eye movement tracking.

[0155] The display system includes an animation preprocessing module and a fixation point sensing module. Figure 4 As shown in the schematic diagram of the preset depth screen in the display system, the scene simulated in this screen is a road, which includes a near-view cylindrical obstacle 201, a far-view cylindrical obstacle 202, the sun 203, and clouds 204. According to the geometric perspective relationship, the human eye will spontaneously think that the near-view cylindrical obstacle 201 is closer than the far-view cylindrical obstacle 202, and the sun 203 and clouds 204 can be regarded as infinitely far away. Therefore, when the predicted fixation point reaches a certain position among 201 - 204, a signal will be sent to the screen-lens adjustment system to change the distance between the two through a micro stepping motor.

[0156] The described display system includes an animation preprocessing module and a fixation point sensing module, where:

[0157] The animation preprocessing module is used to preset the virtual image depths corresponding to different regions in the screen. The implementation process is as follows:

[0158] Step 211: Set the depth parameters in the virtual screen according to the depth information of the real world.

[0159] Step 212: Assign depth information of different regions to each frame of the animation.

[0160] Step 213: Set corresponding depth parameters for different depth of field ranges.

[0161] Step 214: Transmit the calculated depth information to the feedback terminal for dynamic adjustment in visual display.

[0162] The fixation point sensing module is used to ensure the synchronization of screen update and lens adjustment. The implementation process is as follows:

[0163] Step 221: Set the regional threshold range of the screen for fixation detection.

[0164] Step 222: Preset different regional thresholds according to different depth of field ranges.

[0165] Step 223: When the user's fixation point enters the set regional range, trigger feedback information to drive the micro stepping motor to adjust the focal length to optimize the viewing experience.

[0166] According to the Figure 2 flowchart shown, the tester needs to Figure 5The housing 301 shown in the figure is correctly worn on the head and can see through the lens 102 Figure 6 the screen 101 shown in the figure. The screen 101 is installed in the screen frame 401. The screen frame 401 is slidably installed on one side of the linear slide rail 302. The lens 102 is fixedly installed on the other side of the linear slide rail 302 through the lens frame 303 and is connected to the partition 105 through the card slot 304. During the process of gazing at the screen 101, the two eyes are separated by the partition 105 to ensure binocular dissociation, so that the images seen by the two eyes are relatively independent. After the test adjustment is completed, by Figure 5 the active infrared light source 305 in it illuminates the pupil, and the active infrared camera 104 takes a pupil image for eye movement calibration. When the calibration is completed, through Figure 6 the micro stepping motor 106 in it drives the screen frame 401 to move, thereby changing the distance between the screen and the lens. During the distance change, since the partition 105 and the lens frame 303 are connected through the card slot 304, the two eyes are still independent of each other. After the distance change ends, according to Figure 2 the flowchart shown in it is looped until the final training ends.

[0167] Furthermore, the movement of the micro stepping motor 106 driving the screen frame 401 is completed by the lens-screen adjustment system.

[0168] The lens-screen adjustment system includes a lens 102, a screen 101, a linear slide rail 302, a micro stepping motor 106 and a drive circuit. The drive circuit is connected to the micro stepping motor 106. The moving end of the micro stepping motor 106 is linked with the screen 101. The screen 101 is slidably installed on the linear slide rail 302. The screen 101 and the lens 102 are respectively arranged on the front and back sides of the linear slide rail 302; the components of this system are highly integrated. When the fixation point enters the target area, the drive circuit sends a control signal to the micro stepping motor 106, and the drive motor drives the screen 101 to perform a smooth and precise linear displacement along the linear slide rail 302, thereby adjusting the geometric distance between the lens 102 and the screen 101. In this solution, the introduction of the micro stepping motor 106 realizes the lightweight design of the structure and at the same time has an accurate feedback adjustment ability. After receiving the data from the multi-image plane eye movement tracking system, the system can drive the lens-screen unit to perform dynamic adjustment through the micro stepping motor, so as to ensure the real-time matching of the user's visual focus and the image plane depth.

[0169] The lens-screen adjustment system includes an adjustment module and a sensing feedback module, where:

[0170] The adjustment module is used to select the distance to be adjusted in real time according to the predicted fixation point coordinates. The implementation process is as follows:

[0171] Step 311: Select a screen area according to the predicted fixation point coordinates outputted.

[0172] Step 312: When the fixation point falls into this area, the system adjusts the focal length according to the preset travel range to match the corresponding visual requirements.

[0173] The sensing feedback module is used to drive a micro stepping motor to adjust the distance between the lens and the screen. The implementation process is as follows:

[0174] Step 321: Use a high-precision micro stepping motor with a travel range of 8 mm to ensure the accuracy and stability of the adjustment.

[0175] Step 322: According to different user fixation areas, the system generates a PWM (pulse width modulation) control signal to drive the micro stepping motor to perform corresponding displacement adjustments to achieve focal length optimization.

[0176] The loop process is implemented by a control system. The control system includes a data processing module, a calculation module, and a control instruction module. Among them:

[0177] The data processing module is used to receive the fixation point coordinates from the eye tracking system. The implementation process is as follows:

[0178] Step 411: Receive the pupil center coordinates outputted by the eye tracking system.

[0179] Step 412: Receive and parse the calibrated fixation point coordinates.

[0180] Step 413: Based on the pupil center coordinates and the calibrated fixation point coordinates, perform coordinate correction and filtering to remove noise and improve data accuracy.

[0181] Step 414: Store the corrected fixation point coordinates and transmit the data to the calculation module.

[0182] The calculation module is used to analyze the required image plane depth. The implementation process is as follows:

[0183] Step 421: Receive the fixation point coordinate information from the data processing module.

[0184] Step 422: Based on the preset fixation point-depth mapping model of the system, calculate the required image plane depth.

[0185] Step 423: Detect outliers in the calculation results and perform interpolation processing to optimize data smoothness.

[0186] Step 424: Store the calculated depth information and send it to the control instruction module.

[0187] The control instruction module is used to send motion instructions to the lens-screen adjustment system, and the implementation process is as follows:

[0188] Step 431: Receive the image plane depth data transmitted by the calculation module;

[0189] Step 432: Generate control instructions for the stepper motor according to the depth requirement, including forward, backward and stroke length;

[0190] Step 433: Use PWM (pulse width modulation) signals to control the moving step size and speed of the micro stepper motor;

[0191] Step 434: Monitor the stepper motor status in real time to ensure accurate adjustment of the lens position;

[0192] Step 435: Feedback the execution result and make secondary adjustment when necessary to improve the system response accuracy.

[0193] Refer to Figure 2 , a dynamic virtual image plane adjustment method based on multi-image plane eye movement tracking. When the system starts, the system initializes and starts all core systems. In this stage, it is ensured that all hardware works properly and the software enters the standby state to prepare for subsequent operations;

[0194] The adjustment method includes the following steps:

[0195] Step 1: The user wears the device: The user correctly wears the eye movement tracking device, ensures that the device fits firmly on the face, and adjusts it to a suitable position so that the camera can clearly capture the user's eye area. The system self-checks whether the user's pupils are fully visible and adjusts the infrared light intensity to optimize the pupil imaging quality and ensure the accuracy of subsequent data collection;

[0196] Step 2: Eye movement calibration: The system enters the calibration mode and guides the user to fixate on multiple preset calibration points in sequence to establish the user's eye movement model; by calculating the mapping relationship between the pupil center offset and the screen coordinates, the system accurately fits the user's personalized eye movement parameters to ensure high accuracy in subsequent fixation point prediction; during the calibration process, the parameters are dynamically adjusted according to the sampling error to improve the tracking stability;

[0197] Step 3: Fixation point prediction: In the normal operation mode, the system continuously collects the user's eye movement data, including feature information such as pupil position and line of sight direction; based on the established eye movement model, the system calculates and predicts the user's current fixation point coordinates in real time to determine the specific target area of the line of sight on the screen; this predicted data is used to drive the subsequent adaptive vision adjustment mechanism;

[0198] Step 4, Fixation Point Sensing: The system further precisely measures the user's real-time fixation point and corrects the errors in the predicted data; the system can detect the minute movements of the user's eyeballs and dynamically adjust the calculation accuracy of the fixation point to cope with the eye movement characteristics of different users; this stage ensures that the system has high stability and robustness in identifying the fixation target.

[0199] Step 5, Lens-Screen Distance Adjustment: When the system confirms that the user's fixation point is within the threshold range of the area set by the fixation point sensing module, the drive circuit generates a drive signal and transmits it to the micro stepping motor. The micro stepping motor precisely moves the screen along the linear slide rail, changing the geometric distance between the screen and the lens to adjust the visual focal length, so as to match the user's current line-of-sight requirements; this adjustment process adopts a closed-loop control mechanism to ensure the smoothness and accuracy of the screen movement and dynamically adjusts according to the real-time changes of the user's fixation point.

[0200] The system performs the final operation, records key data, and decides whether to enter the next operation cycle or safely shut down all devices according to the user's needs; when exiting, the system releases the hardware resources and saves the user's personalized calibration parameters so that they can be quickly loaded during subsequent use to improve the system response efficiency.

[0201] The content described in the embodiments of this specification is only a list of the implementation forms of the inventive concept and is only for illustrative purposes. The protection scope of the present invention should not be regarded as limited to the specific forms stated in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be conceived by those of ordinary skill in the art based on the inventive concept of the present invention.

Claims

1. A dynamic virtual image plane adjustment device based on multi-image-plane eye movement tracking, characterized in that The adjustment device includes a multi-image-plane eye movement tracking system, a lens-screen adjustment system, a control system, and a display system. The multi-image-plane eye movement tracking system uses an infrared camera to capture the user's eye image. After calculations by the image acquisition module, pupil detection module, ellipse fitting module, calibration module, and polynomial fitting module, the accurate fixation point coordinates are obtained and the data is transmitted to the control system. The control system receives the fixation point information, analyzes the required image plane depth at present, generates a lens adjustment instruction, and sends it to the lens-screen adjustment system. The lens-screen adjustment system physically adjusts the lens and the screen according to the control instruction, and at the same time uses a sensing feedback mechanism to monitor the adjustment accuracy in real time to ensure that the virtual image position matches the user's fixation focus. The display system combines the depth information provided by the control system to dynamically adjust the animation screen, making it synchronously change with the user's fixation depth to enhance the visual immersion.

2. The dynamic virtual image plane adjusting device based on multi-image plane eye movement tracking according to claim 1, characterized in that, In the multi-image-plane eye movement tracking system, the image acquisition module obtains the user's eye pupil image by calling an active infrared camera. The pupil detection module is used for calibrating the pupil contour. The ellipse fitting module is used for detecting the pupil center position with the highest confidence. The calibration module performs coordinate conversion in combination with known reference points. The polynomial fitting module optimizes the data and finally converts the pupil data into fixation point coordinates.

3. The dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking according to claim 2, characterized in that The implementation process of the image acquisition module is as follows: Step 111, select a short-focus camera and infrared lamp beads. Step 112, connect the short-focus camera and infrared lamp beads. Step 113, adjust the positions of the short-focus camera and infrared lamp beads to ensure that the eyes are fully illuminated. The implementation process of the pupil detection module is as follows: Step 121, judge whether the pupil detection fails according to the pupil threshold algorithm. If the detection fails, step 113 needs to be repeated to ensure the success of the pupil detection. Step 122, convert the input video into video frames and process them frame by frame: Crop and scale the input original image to unify the image size and facilitate subsequent processing. By calculating the aspect ratio Current_ratio of the current image and the target aspect ratio Desire_ratio, if Current_ratio > Desire_ratio, it indicates that the image is too wide. Then calculate the new width New_width = int(Desire_ratio * Current_ratio), and crop the image at the position offset from the center by loc_w = (current_width - New_width) / / 2 in the width direction of the image to obtain the cropped image cropped_img. If Current_ratio <= Desire_ratio, that is, the image is too high, calculate the new height New_height = int(current_width / Desire_ratio), and crop at the position offset from the center by loc_h = (current_height - New_height) / / 2 in the height direction of the image to get cropped_img = image[loc_h: loc_h + New_height, :]; Finally, use the cv2.resize function to adjust the cropped image to the target size (width, height); Step 123, store binary images similar to three different regions A, B, and C in the image: Convert the cropped and scaled image to a grayscale image and apply a strict binary threshold to the grayscale image; by adding the threshold increment to the darkest pixel value Darkest_value , calculate the threshold Threshold = Darkest_value + ; Use the cv2.threshold function to binarize the image, set the pixels less than the threshold to 255 and the rest to 0, to obtain the strictly thresholded image Thresholded_image_1, that is, A; then, with the darkest point Darkest_point as the center and a side length of create a square mask, set the pixel values outside this square area to , and perform masking on Thresholded_image_1; Add the threshold increment to the darkest pixel value Darkest_value , and calculate the threshold Threshold = Darkest_value + , perform binarization on the grayscale image to obtain Thresholded_image_2, that is, B, with the darkest point as the center and side length Perform masking; Add the threshold increment to the darkest pixel value Darkest_value , and calculate the threshold Threshold = Darkest_value + . Perform binarization on the grayscale image to obtain Thresholded_image_3, which is C; then, with the darkest point as the center and side length perform masking; And save Thresholded_image_1, Thresholded_image_2, and Thresholded_image_3; Step 124, detect the color threshold ranges of three different regions A, B, and C; Step 125, select the one with relatively small average color threshold fluctuation as the most reliable pupil image.

4. The dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking according to claim 2, characterized in that, The implementation process of the ellipse fitting module is as follows: Step 131, perform dilation processing on each binary image to enhance the target region: For the image after multi-threshold processing and masking, first use morphological dilation operation to enhance the contour features in the image; then, find the contours in the image through the cv2.findContours function; next, filter the found contours, traverse all contours, calculate the area of each contour area = cv2.contourArea(contour), if area >= pixel, where pixel is a preset threshold, then further calculate the width w and height h of the bounding rectangle of the contour, and calculate the aspect ratio length = max(w, h), width = min(w, h), Current_ratio = max(length / width, width / length); if Current_ratio <= ratio_thresh, where ratio_thresh is a threshold, then this contour meets the condition, and return the contour with the largest area that meets the condition; Step 132, extract the external contours in the dilated image; Step 133, filter the contours by area and quantity limits, and return the largest contour; Step 134, do not perform searches within D pixels of the image edge; Step 135, check the brightness every E pixels within the area; Step 136, sample every F pixels along the axis and the axis, ignoring the boundaries; Step 137, update the pixel point with the largest threshold between the 0 - 255 color channels.

5. The dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking according to claim 2, characterized in that The implementation process of the calibration module is as follows: Step 141, select the screen size as the calibration screen, input the screen parameters into the control system and project them into the human eye through the lens; Step 142, select 9 - 12 points evenly covering the screen as calibration points; Step 143, the user wears the device and adjusts the size of the eye image in the picture; Step 144, perform autonomous calibration through peripherals such as a keyboard or a handle to ensure that the point being stared at in the current state is the calibration point: In the calibration mode, obtain the image in real time through the camera, and use the above pupil detection algorithm to get the center coordinates (x_eye, y_eye) of the pupil; draw the current calibration target point on each frame of the image; when the user presses the space bar, record the current pupil coordinates (x_eye, y_eye) and the corresponding screen coordinates into the calibration data dictionary calibration_data; when all the calibration points are recorded, save calibration_data to a file; Step 145, adjust the distance between the lens and the screen, thereby changing the distance between the image plane and the human eye, and repeat steps 142 to 144; Step 146, save the weight information calibrated at different depths.

6. The dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking according to claim 2, wherein, The implementation process of the polynomial fitting module is as follows: Step 151, obtain the pixel point coordinates of 9 - 12 calibration points; Step 152, output the pixel point coordinates of the pupil center from the video frame; Step 153, split the pixel point coordinates in Step 151 and Step 152 and into one-dimensional data; Step 154, select a polynomial to perform one-dimensional data fitting on the coordinate points; Step 155, fitting the polynomial coefficients in the function; Step 156, fitting the polynomial coefficients in the function; Step 157, save the axis and the coefficients of the one-dimensional polynomials of the axis: Load the pupil coordinate data eye_coords and the screen coordinate data screen_coords from the saved calibration data file, extract the x component eye_x and y component eye_y of the pupil coordinates, and the x component screen_x and y component screen_y of the screen coordinates. Use functions to perform polynomial fitting on the pupil x coordinate and the screen x coordinate, and the pupil y coordinate and the screen y coordinate respectively, to obtain the fitting coefficients coefficients_x and coefficients_y; Step 158, start predicting the fixation point coordinates by inputting the pupil center coordinates.

7. The dynamic virtual image plane adjustment device based on multi-image-plane eye movement tracking according to any one of claims 1 to 6, characterized in that, The lens-screen adjustment system includes a lens, a screen, a linear slide rail, a micro stepping motor, and a drive circuit. The drive circuit is connected to the micro stepping motor. The moving end of the micro stepping motor is linked with the screen. The screen is slidably mounted on the linear slide rail. The screen and the lens are arranged on the front and rear sides of the linear slide rail respectively; The lens-screen adjustment system further includes an adjustment module and a sensing feedback module. The adjustment module is used to select the distance to be adjusted in real time according to the predicted fixation point coordinates; the sensing feedback module is used to drive the micro stepping motor to adjust the distance between the lens and the screen; The implementation process of the adjustment module is as follows: Step 211, select the picture area according to the predicted fixation point coordinates output by the polynomial fitting module; Step 212, adjust the pre-set travel range when this area is marked; The implementation process of the sensing feedback module is as follows: Step 221, select a micro stepping motor with an 8mm travel; Step 222, apply a PWM wave to control the travel of the micro stepping motor according to different fixation areas.

8. The dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking according to any one of claims 1 to 6, characterized in that, The control system includes a data processing module, a calculation module, and a control instruction module. The data processing module is used to receive the fixation point coordinates from the eye tracking system. The calculation module is used to analyze the required image plane depth at present. The control instruction module is used to send a motion instruction to the lens-screen adjustment system; The implementation process of the data processing module is as follows: Step 311, receive the pupil center coordinates output by the eye tracking system; Step 312, receive and analyze the calibrated fixation point coordinates; Step 313, based on the pupil center coordinates and the calibrated fixation point coordinates, perform coordinate correction and filtering to remove noise and improve data accuracy; Step 314, store the corrected fixation point coordinates and transmit the data to the calculation module; The implementation process of the calculation module is as follows: Step 321, receive the fixation point coordinate information from the data processing module; Step 322, calculate the required image plane depth at present based on the pre-set fixation point-depth mapping model of the system; Step 323, perform outlier detection on the calculation result and perform interpolation processing to optimize data smoothness; Step 324, store the calculated depth information and send it to the control instruction module; The implementation process of the control instruction module is as follows: Step 331, receive the image plane depth data transmitted by the calculation module; Step 332: Generate control instructions for the stepper motor according to the depth requirement, including forward, backward, and stroke length; Step 333: Use Pulse Width Modulation (PWM) signals to control the movement step and speed of the micro stepper motor; Step 334: Monitor the status of the stepper motor in real time to ensure accurate adjustment of the lens position; Step 335: Feed back the execution result and make secondary adjustments if necessary to improve the system response accuracy.

9. The dynamic virtual image plane adjustment device based on multi-image-plane eye movement tracking according to any one of claims 1 to 6, characterized in that The display system includes an animation preprocessing module and a fixation point sensing module. The animation preprocessing module is used to preset the virtual image depth corresponding to different regions in the picture in advance; the fixation point sensing module is used to ensure that the picture update is synchronized with the lens adjustment; The implementation process of the animation preprocessing module is as follows: Step 411: Set the depth information in the picture according to the depth information in real life; Step 412: Preset the depth information for different regions of each frame of the animation; Step 413: Preset different depth information for the pictures at different depths; Step 414: Set the preset depth information as the feedback terminal; The implementation process of the fixation point sensing module is as follows: Step 421: Set the regional threshold range; Step 422: Preset different regional threshold ranges for the pictures at different depths; Step 423: When the fixation point moves to the regional threshold, generate feedback information to drive the micro stepper motor of the lens-screen adjustment system to achieve feeding.

10. A method for implementing a dynamic virtual image plane adjustment device based on multi-image plane eye movement tracking as described in claim 1, characterized in that, The method includes the following steps: Step 1: User wears the device: The user correctly wears the eye tracking device, ensures that the device fits firmly on the face, and adjusts the position so that the camera can clearly capture the user's eye area; Self-check whether the user's pupils are fully visible and adjust the infrared light intensity; Step 2: Eye movement calibration: Enter the calibration mode, guide the user to fixate on multiple preset calibration points in sequence to establish the user's eye movement model; By calculating the mapping relationship between the pupil center offset and the screen coordinates, accurately fit the user's personalized eye movement parameters; During the calibration process, dynamically adjust the parameters according to the sampling error; Step 3: Fixation point prediction: In the normal operation mode, continuously collect the user's eye movement data, including pupil position and line of sight direction; Based on the established eye movement model, calculate and predict the user's current fixation point coordinates in real time to determine the specific target area of the line of sight on the screen; This predicted data is used to drive the subsequent adaptive visual adjustment mechanism; Step 4: Fixation point sensing: Further accurately measure the user's real-time fixation point and correct the error of the predicted data; Detect the small movements of the user's eyeballs and dynamically adjust the calculation accuracy of the fixation point to cope with the eye movement characteristics of different users; Step 5, lens-screen distance adjustment: The lens-screen adjustment system includes a lens, a screen, a linear slide rail, a micro stepping motor, and a drive circuit. The drive circuit is connected to the micro stepping motor. The moving end of the micro stepping motor is linked to the screen. The screen is slidably mounted on the linear slide rail. The screen and the lens are respectively arranged on the front and rear sides of the linear slide rail. When it is confirmed that the user's fixation point is within the threshold range of the area set by the fixation point sensing module, the drive circuit generates a drive signal and transmits it to the micro stepping motor. The micro stepping motor precisely moves the screen along the linear slide rail, changing the geometric distance between the screen and the lens to adjust the visual focal length, so as to match the user's current line-of-sight requirements.

Images