An Image Processing Algorithm for Real-Time Extraction of Closed-Eye Pupil Center and Trajectory Tracking

Abstract

The present invention discloses an image processing algorithm for real-time extraction of pupil center and trajectory tracking in the closed-eye state. From the perspective of digital image processing, this algorithm first performs grayscale conversion, contrast enhancement, and smoothing processing on each frame of the input eye video. Then, by using adaptive threshold segmentation technology and combining morphological operations and the Canny edge detection operator, it can accurately extract the boundary of the pupil region. Subsequently, the least squares method based on geometric distance is used to accurately fit the elliptical parameters of the pupil contour, thereby obtaining the center coordinates and area information of the pupil, significantly improving the accuracy and reliability of pupil monitoring in the closed-eye state. By recording and analyzing the changes in the pupil center position in the time series, real-time tracking of the pupil movement trajectory is achieved. Finally, baseline correction and smoothing filtering processing are adopted to evaluate the pupil light reflex (PLR) in the closed-eye state before and after white light excitation. This method has the advantages of strong robustness, adaptability to light changes, high real-time performance, and high precision.

Description

Technical Field

[0001] The present invention relates to the technical field of pupil monitoring, and specifically to an algorithm for real-time extraction and trajectory tracking of the pupil center in an eye movement video under closed eyelids based on image processing. Background Art

[0002] Monitoring of pupil changes is of crucial significance in the fields of clinical care and neuroscience research. The information it conveys can not only assist clinicians in predicting and diagnosing potential disease deterioration trends in patients, but also provide key reference indicators for various neurological function assessments. Especially in fields such as neurology, intensive care, and anesthesia that highly rely on rapid and accurate physiological signal assessments, monitoring of pupil size and its dynamic response to light stimulation has become a key part of the comprehensive evaluation of patients' vital signs. When patients are in relatively special physiological states, such as with closed eyelids, limited consciousness, or in a state of deep sedation and anesthesia, conventional visual pupil measurement methods become very difficult. Due to the relatively frequent changes in the visible light environment and the high requirements for the patient's physiological state and environmental adaptability, traditional visible light-based pupil measurement means often struggle to provide accurate and stable pupil parameters in the case of closed eyelids, low-light environments, or the presence of eye injuries. On the other hand, in clinical practice, when eye drops, sedatives, anesthetics, or other intervention means are sometimes used, the change in pupil diameter can directly or indirectly reflect the degree of nervous system damage and drug effects. Therefore, it is necessary to achieve precise monitoring of the pupil state of patients with closed eyes in a non-invasive or low-burden state.

[0003] However, current traditional pupil measurement methods are all intermittent, qualitative, manual, and limited to open-eye cases. For example, directly using a flashlight to irradiate and observe, manually comparing pupil scales, and using professional instruments for lid-opening measurement, etc., all have certain limitations. These traditional means often require the patient's eyelids to be open and have high requirements for the operation proficiency of medical staff and the on-site lighting conditions. At the same time, when faced with patients who are unable or unsuitable to open their eyes (such as patients with severe injuries resulting in the inability to open their eyelids independently, or patients in a state of deep sedation and anesthesia), the traditional means cannot be carried out normally, making pupil measurement under such special physiological or pathological conditions a clinical problem. In addition, the solutions relying on visible light measurement have deficiencies in accuracy and consistency in strong light, weak light, and variable light scenarios, which is not conducive to continuous and quantitative tracking of the severity and progression trend of diseases.

[0004] In the face of the above challenges, some researchers have proposed a method for detecting and measuring the pupil under closed eyelids using a near-infrared light source and an infrared camera. For example, according to the technical solution reported in the paper "Measuring pupil size and light response through closed eyelids" published by Yousef Farraj et al. in the 10th issue of Volume 12 of "Biomedical Optics Express" in 2021, near-infrared light is irradiated on the eyelids and the transcutaneous reflection signal is captured by a camera, and then the pupil diameter is estimated using an image algorithm. Such methods reduce the influence of visible light environment changes to a certain extent, making it possible to evaluate the size and dynamic characteristics of the pupil under closed eyelids. Through this infrared imaging method, pupil information can be obtained under relatively more stable and low-contrast image conditions, and the stress response of patients caused by strong light stimulation can be reduced. However, such methods still face some problems when implemented: First, during the human eye image acquisition process, the captured human eye images do not always contain a complete pupil. The human eye pupil can vary to a certain extent or even greatly in terms of shape, size, opening and closing state, etc. In a noisy image, the pupil is not necessarily a complete circular pattern. Since IR scatters through the eyelids and the pupil image has no obvious boundary, the low-intensity interference region in the binary image may surround the pupil, reducing the accuracy of pupil localization. This method assumes that the pupil is circular, which will cause errors in fitting the pupil diameter; Second, this method only calculates the pupil diameter through static images and cannot extract the pupil trajectory and achieve real-time tracking. Therefore, it is inconvenient to study the complex shape and movement of the pupil under dynamic conditions, and the flexibility is relatively low.

[0005] Meanwhile, some scholars have attempted to use intelligent algorithms or machine learning methods to assist in pupil detection and size quantification. They automatically segment and identify eye features in video data through deep learning models. For example, in the paper "Touchless short-wave infrared imaging for dynamic rapid pupillometry and gaze estimation in closed eyes" published by Omer Ben Barak-Dror, Barak Hadad, etc. in 2024, it is reported in the literature that the combination of short-wave infrared (SWIR, ~0.9 - 1.7 microns) imaging and image processing algorithms enables neural network-based analysis to successfully estimate and reconstruct open-eye images using closed-eye SWIR data, and evaluate pupil size, achieving rapid pupillometry and eye movement tracking in the closed-eye state. However, this method requires a large amount of open-eye annotation data to be established through experiments for model training, has a relatively large demand for computing resources, requires specific SWIR cameras and light sources, and is relatively complex. At the same time, this method relies on the assumption of binocular symmetry in healthy individuals, and the analysis depends on the training of individual open-eye data of the subjects. Some literature points out that more robust image enhancement and adaptive segmentation strategies are needed to stably locate the pupil boundary under conditions of noise interference, low local contrast, and frequent light changes. Summary of the Invention

[0006] In view of the above technical problems faced in the process of pupil detection and trajectory tracking under closed eyelids in the prior art, the present invention proposes an image processing algorithm for real-time extraction of the pupil center and trajectory tracking with closed eyes. Based on traditional digital image processing, this algorithm adopts a least-squares ellipse fitting algorithm based on geometric distance, utilizes the geometric characteristics of the ellipse, determines the ellipse parameters by minimizing the geometric distance from given points to the ellipse surface, and can gradually approach the optimal solution, that is, the ellipse shape that best fits the image data, through fminsearch iteration, ultimately realizing the real-time extraction and trajectory description of the pupil in the eye movement video with closed eyelids.

[0007] The image processing algorithm for real-time extraction of the pupil center and trajectory tracking with closed eyes of the present invention includes the following steps:

[0008] S1: Acquisition of original video data and image preprocessing:

[0009] The present invention first initializes the reading and writing of the video, then initializes the coordinate array and creates a fixed graphics window to accelerate the processing of the effective area of the video. Then, it reads the original eye images in the video sequence frame by frame, and first performs grayscale conversion on each frame of the image to eliminate the interference of color information on pupil recognition. Then, according to the illumination conditions in the actual scene, the average value of the image brightness is calculated. The global brightness of the grayscale image is adjusted using methods of brightness enhancement and contrast improvement (such as histogram equalization) to highlight the contrast between the pupil area and the surrounding tissues. On this basis, Gaussian smoothing filtering is used to suppress high-frequency noise and local texture stray information, providing better basic image data for subsequent segmentation and fitting.

[0010] S2: Adaptive threshold segmentation and morphological operation processing:

[0011] For the preprocessed grayscale image, the present invention uses the adaptive threshold method for binary segmentation, dynamically adjusts the segmentation threshold according to the local area brightness characteristics, and effectively deals with the low-contrast areas caused by non-uniform illumination conditions and closed eyelid occlusion. In the obtained preliminary binary image, there may be edge defects or internal holes in the pupil area. The broken edges are connected through morphological closing operations, the internal holes are repaired using filling operations, and then morphological opening operations are used to eliminate discrete noise points to obtain a binary image of the pupil candidate area that is continuous, complete, and has clear boundaries.

[0012] S3: Connected component analysis and pupil candidate area screening:

[0013] In the binary image after morphological processing, the present invention performs connected component labeling to obtain morphological features such as the area, eccentricity, roundness, elongation, and centroid of all independent regions. By setting reasonable threshold conditions (such as the minimum area lower limit, eccentricity upper limit, and roundness lower limit, etc.), each connected component is screened one by one, and the regions that do not conform to the physiological characteristics of the pupil are eliminated. Finally, only the candidate regions that match the morphological, size, and distribution characteristics of the real pupil are retained, providing a reliable basis for the precise positioning of the pupil center.

[0014] S4: Edge extraction of pupil contour features:

[0015] After obtaining the pupil candidate area, the present invention uses the Canny operator to perform edge detection on the effective area, thereby extracting the precise pupil contour. The coordinates of these boundary points are statistically formed into a point set for subsequent ellipse fitting.

[0016] S5: Least squares ellipse fitting based on geometric distance:

[0017] Extract the pupil contour point set from the regional boundary, and use the least squares method based on geometric distance to perform ellipse fitting according to the extracted boundary points. The specific steps include calculating the covariance matrix and performing eigenvalue decomposition to preliminarily estimate the major axis and minor axis directions of the pupil. Subsequently, the least squares fitting method (such as using the weighted residual sum of squares minimization optimization algorithm) is used to iteratively solve the ellipse parameters (the present invention uses the fminsearch iterative algorithm). Through iteration, the optimal solution can be gradually approximated, that is, the ellipse shape that best conforms to the image data, realizing high-precision ellipse fitting of the pupil contour.

[0018] The ellipse fitting method based on the least squares method used in the present invention considers five parameters such as the ellipse center position, semi-axis length, and ellipse attitude angle, forming a parameter vector. The specific implementation steps of the algorithm are as follows:

[0019] 1) Initialize the ellipse parameters:

[0020]

[0021] Calculate the center point:

[0022]

[0023] Calculate the covariance matrix:

[0024] Center the boundary point coordinates:

[0025]

[0026] Calculate the covariance matrix C:

[0027]

[0028] Eigenvalue decomposition:

[0029] Perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvectors v1 and v2 of the eigenvalues λ1 and λ2:

[0030] Cv = λv

[0031] Calculate the major and minor axes:

[0032] Calculate the major semi-axis a0 and minor semi-axis b0 through the eigenvalues:

[0033]

[0034] Calculate the ellipse rotation angle φ:

[0035]

[0036] where v 11 and v 21They are the x and y components of the eigenvector v1.

[0037] Ensure that the major axis and minor axis are correct:

[0038] if a < b

[0039] a, b = b, a

[0040] φ = φ + π / 2

[0041] 2) Calculate the weights:

[0042]

[0043] where: n is the number of data points, (x i , y i ) are the coordinates of each data point.

[0044] 3) Define the objective function:

[0045]

[0046] where: P = [p1, p2, p3, p4, p5] are the ellipse parameters to be optimized

[0047] 4) Minimize the objective function:

[0048]

[0049] Iterate through fminsearch, and terminate when the error ε < 10 -7 for termination.

[0050] 5) Extract the fitting results:

[0051] Center:

[0052] Major semi - axis:

[0053] Minor semi - axis:

[0054] Rotation angle:

[0055] S6: Pupil center extraction and real - time trajectory drawing:

[0056] The central coordinates, major and minor axis parameters, and rotation angle obtained by fitting in step S5) can not only clarify the position of the pupil center point, but also calculate the pupil area to quantify the change in pupil size. After obtaining the position of the pupil center point by fitting, the center point coordinates are stored in the coordinate array, and the video is processed frame by frame through a loop to obtain a set of pupil center coordinate arrays. The movement trajectory of its center coordinates is depicted in a fixed window with a smooth curve, and the image display is updated during each loop, so that the central movement trajectory of the pupil can be depicted in real time. When all the center point trajectories are depicted, the video is saved.

[0057] S7: Plotting the curve of the change in pupil area over time before and after white light excitation:

[0058] Based on the processing of multiple consecutive frames of video, the present invention performs time series analysis on the pupil area data. First, the pupil area data of the initial several frames is selected to calculate the baseline, and at the same time, a relatively large window size is used to smooth the ellipse area data to ensure the continuity and smoothness of the time series and reduce the influence of noise, so as to obtain a more stable curve that can effectively reflect the dynamic response of the pupil over time.

[0059] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0060] Starting from the perspective of digital image processing, based on traditional image processing algorithms, the present invention adopts morphological operations and is optimized by the least squares fitting and iterative algorithms, and proposes a real-time pupil center extraction and trajectory tracking algorithm during the closed eyelid process based on video image processing, which has the following advantages:

[0061] 1) The adaptive threshold segmentation method is adopted, which significantly reduces the problem of uneven illumination caused by light source excitation, provides more accurate segmentation, and has strong adaptability.

[0062] 2) The present invention adopts a filtering algorithm and an edge extraction algorithm, both of which belong to convolution algorithms, significantly reducing the number of parameters of the method. The present invention only needs several convolution operations to extract the pupil edge point set parameters, with simple calculation and high efficiency.

[0063] 3) The least squares ellipse fitting algorithm based on geometric distance adopted by the present invention utilizes the geometric characteristics of the ellipse to determine the ellipse parameters by minimizing the geometric distance from a given point to the ellipse surface. This algorithm considers five parameters such as the center position of the ellipse, the semi-axis length, and the attitude angle of the ellipse, and is optimized by an iterative algorithm, with high fitting accuracy and improved robustness of the algorithm. Description of the Drawings

[0064] Figure 1A detailed flowchart of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention;

[0065] Figure 2 A schematic diagram of the original image of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention;

[0066] Figure 3 A schematic diagram of the image after grayscale conversion and contrast enhancement of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention;

[0067] Figure 4 A schematic diagram of the image after Gaussian smoothing filtering of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention;

[0068] Figure 5 A schematic diagram of the locally adaptive threshold-binarized image of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention;

[0069] Figure 6 A schematic diagram of the image after morphological processing of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention;

[0070] Figure 7 A schematic diagram of the image after deleting small areas and some interference regions of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention;

[0071] Figure 8 A schematic diagram of the image after Canny operator edge extraction of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention.

[0072] Figure 9 A schematic diagram of the elliptical fitting result image based on the geometric distance least squares method of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention.

[0073] Figure 10 A schematic diagram of the pupil center motion trajectory tracking image of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention.

[0074] Figure 11 A schematic diagram of the image of the change of the pupil area over time of an embodiment of the real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of the present invention. Detailed implementation manners

[0075] To more clearly understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be further elaborated in detail below in conjunction with the accompanying drawings and specific embodiments.

[0076] The real-time extraction of the closed-eye pupil center and the trajectory tracking image processing algorithm of this embodiment, as Figure 1 shown, gives the detailed flowchart of this algorithm, including the following steps:

[0077] 1) Input video acquisition and preprocessing:

[0078] First, use the video reading module to read the eye video to be processed frame by frame. The excitation of white light under the closed eyelids in the input video is used to verify the reactivity of the pupil, resulting in problems such as uneven brightness, noise interference, and low contrast in the original video frames. To improve the distinguishability between the pupil area and the surrounding eye structures, it is necessary to preprocess the video frames. In this example, one of the frames is selected for demonstration. The original schematic diagram of the image is as Figure 2 shown.

[0079] The frame image is grayscale processed to remove the interference of color information and make the image features more simple. Then calculate the average brightness of this frame image. Since the subsequent morphological processing will set different parameters according to different average brightnesses, according to the overall brightness level and contrast characteristics of the image, means such as brightness enhancement and histogram equalization are used to globally enhance the grayscale image, so that the details in the dark part are improved, and the image after enhanced contrast is obtained, as Figure 3 shown.

[0080] There may still be high-frequency noise and local uneven illumination in the enhanced image. To further improve the accuracy of subsequent segmentation, Gaussian smoothing filtering is performed on the enhanced grayscale image. At this time, random noise can be effectively suppressed and the overall structural features of the pupil boundary can be retained, as Figure 4 shown. It can be seen that the overall illumination of the preprocessed image is uniform and the noise is significantly reduced.

[0081] 2) Adaptive threshold segmentation and morphological processing:

[0082] After obtaining a relatively clear preprocessed grayscale image, the image is binarized by the local adaptive threshold method to obtain a local adaptive threshold-binarized image, as Figure 5 shown. This threshold segmentation method can automatically adapt to the brightness changes in the periorbital skin and pupil areas in each local area, so that the pupil and the background can still be effectively distinguished under uneven illumination and partial eyelid closure conditions. In this example, after the white light excitation, the average brightness of the image is relatively large. At this time, the adaptive threshold is set to 0.5, while when there is no white light excitation, the adaptive threshold is set to 0.53.

[0083] In the obtained binary image, edge breaks and small internal holes may appear in the pupil region. Morphological closing operation is used to connect the pupil edge and repair the broken curve; the region filling operation is used to fill the holes inside the pupil, so that the pupil presents a complete solid region; then, morphological opening operation is used to remove the small noise adhered to the outside of the pupil edge, and the clean pupil body is retained. After the above processing, the image after morphological operation is obtained, as Figure 6 shown. It can be clearly seen that the pupil region in the binary image after the above morphological processing presents a relatively smooth and coherent regional shape. In this example, both the opening operation and the closing operation use circular structuring elements. For frame images with a relatively high average brightness, first perform a closing operation with a circular structuring element with a radius of 5, and then perform an opening operation with a circular structuring element with a radius of 40; for frame images with a relatively low average brightness, first perform a closing operation with a circular structuring element with a radius of 5, and then perform an opening operation with a circular structuring element with a radius of 25.

[0084] 3) Connected component labeling and pupil candidate region screening:

[0085] Perform connected component analysis on the processed binary image to label all connected regions in the image. At this time, in addition to the pupil, there may still be several pseudo-target regions formed by uneven illumination, noise, eyelid edges, eyelash shadows, etc.

[0086] To eliminate these interferences, the regionprops function is used to extract various features of each connected component, including area, eccentricity, roundness (calculated by the relationship between area and perimeter), filling ratio of the circumscribed rectangle, and region centroid, etc. By setting appropriate thresholds, such as minimum area, upper limit of eccentricity, lower limit of roundness, etc., regions that do not match the physiological characteristics of the pupil can be effectively filtered. The parameters set in this example are as follows: eccentricity less than 0.9, area ratio greater than 0.5, minimum area of the connected component greater than 8500 pixels, and roundness greater than 0.85. By screening the connected components under the above conditions, invalid non-pupil regions and small noises can be effectively removed. When there is no valid region or the pupil region is not obvious, skip the processing of this frame and proceed to the next frame.

[0087] When the connected components that obviously do not meet the requirements are removed, the remaining regions are the pupil candidate regions. As Figure 7 shown. After the above feature screening, the retained connected components are relatively close to the human eye pupil in terms of shape and size.

[0088] 4) Pupil contour feature edge extraction:

[0089] For the determined pupil candidate region, extract its boundary point set, and use the edge function to perform Canny edge detection on the binary region. The convolution kernel size of the Canny operator is 3×3, so as to extract the precise pupil contour and obtain the binary time-frequency diagram after edge extraction, as shown in Figure 8 shown. Statistically form the point cloud data from these boundary point coordinates for subsequent ellipse fitting.

[0090] 5) Least squares ellipse fitting based on geometric distance:

[0091] This algorithm utilizes the geometric characteristics of the ellipse to determine the ellipse parameters by minimizing the geometric distance from the given points to the ellipse surface. This algorithm considers five parameters such as the center position of the ellipse, the semi-axis lengths, and the attitude angle of the ellipse to form a parameter vector. An iterative algorithm is used to find the optimal solution. The specific calculation steps are as follows: First, initialize the ellipse parameter vector, then calculate the center and covariance matrix of the pupil boundary point set, and obtain the initial ellipse parameter estimates through eigenvalue decomposition, including the long axis and short axis directions and proportional relationships. Subsequently, use the least squares error fitting method. In this example, the fminsearch function is used to iteratively optimize the defined error function to optimize the ellipse parameters (including the center coordinates (x, y), the major semi-axis a, the minor semi-axis b, and the rotation angle ).

[0092] During the optimization process, a weighted residual minimization strategy is adopted, and smaller weights are assigned to the boundary points farther from the center to reduce the influence of outliers on the fitting result. Finally, it is also necessary to ensure that the obtained long axis and short axis are reasonable, that is, if the obtained eigenvalues (a, b) are both positive and reasonable, it indicates that the fitting is successful. Extract the fitting result to obtain the precise parameters of the pupil ellipse, thereby obtaining the ellipse fitting result optimized based on weights, as shown in Figure 9 shown. The optimal parameter values are: the pupil center coordinates are (174.2391, 757.5408), the major semi-axis a = 79.9237, the minor semi-axis b = 66.2307, the rotation angle (in radians) = 1.547, the rotation angle (in degrees) = 88.6352; the red solid line ellipse in the figure is fitted to the pupil contour, and the center point is marked with a green star mark, which is the pupil center coordinate of the current frame. It can be seen from the figure that the result obtained after the least squares ellipse fitting based on geometric distance and the optimization of the iterative algorithm coincides with the effective pupil region actually observed under closed eyelids, indicating that this algorithm can accurately identify and locate the position of the pupil (including the center coordinates), thus verifying the beneficial effects of the present invention.

[0093] 6) Pupil center extraction and real-time trajectory drawing:

[0094] The pupil center coordinates can be obtained from step 5). By recording the pupil center coordinates obtained from multiple consecutive frames, the variation characteristics of the pupil movement trajectory over time can be obtained. Finally, the movement trajectory of the pupil is plotted in real time, and the pupil center coordinate trajectory is superimposed and displayed on the original video frame to visually show the movement of the pupil under the condition of closed eyelids in an animated way, realizing the visual analysis of the dynamic changes of the pupil, such as Figure 10 shown.

[0095] 7) Plotting the curve of the pupil area change over time before and after white light excitation:

[0096] First, perform time series analysis on the pupil area (πab) calculated for each frame. Calculate the baseline value using the area data of the first several frames, calculate the corresponding time points for each frame, and apply moving average filtering to smooth the data. Then, plot the curve of the pupil area change over time, as Figure 11 shown, demonstrating the dynamic change trend of the pupil area before and after white light excitation. By analyzing this trend, evaluate the influence of white light stimulation on the pupil response, and provide quantitative and visual analysis results for the study of pupil behavior under light stimulation.

[0097] The above embodiments are the preferred specific embodiments of the present invention. Obviously, the above embodiments of the present invention are only examples for more clearly explaining the present invention, rather than limiting the embodiments of the present invention. Any modifications, equivalent replacements, and improvements within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A real-time image processing method for extracting the closed-eye pupil center and tracking its trajectory, characterized in that, The following steps are involved: S1: Raw video data acquisition and image preprocessing: First, the video sequence is acquired by irradiating the side of the temple with near-infrared light and optical imaging of closed eyelids. The video sequence includes a white light excitation process to test the pupil light reflex (PLR). The PLR shows a typical time process, which includes the following parameters: latency, contraction time, and recovery baseline. Then, the eye video sequence to be processed is acquired, and a video write object is created. Next, the input original video is extracted frame by frame, and each frame is grayscaled. The average brightness of the image is calculated, and the overall brightness is globally enhanced to achieve a preliminary improvement in the original low-contrast image. Based on the global enhancement, adaptive smoothing filtering is used in the local area to suppress high-frequency noise, so as to reduce the interference of background stray information while keeping the pupil edge relatively clear, and obtain an optimized grayscale image sequence. After the above processing, the eye grayscale image sequence after preliminary preprocessing is obtained. S2: Adaptive threshold segmentation and morphological operation processing: An adaptive threshold segmentation algorithm is used to analyze local brightness features of the preprocessed eye grayscale image sequence. A sliding window is selected and the local brightness features are counted. The local mean, standard deviation, or Gaussian filtering result is used to determine a threshold level that is suitable for the current lighting conditions, thereby generating a binary image with appropriate contrast in a scene with uneven brightness and darkness. and The noise and pseudo-target areas are suppressed and cleaned by combining morphological closing, filling and opening operations, thus obtaining a binary image with a relatively complete pupil area outline. S3: Connected domain analysis and pupil candidate region screening: The connected domain analysis method is used to mark regions and screen targets in the binary image after morphological processing. For each connected domain, the comprehensive features of area, perimeter, eccentricity, roundness, area, and occupancy ratio of the circumscribed rectangle are extracted and matched with the pre-established pupil physiological characteristic model. By setting the eccentricity, roundness, and minimum area thresholds, interfering regions that obviously deviate from the pupil morphological characteristics are filtered out. For regions that meet the preliminary screening criteria, the area and position change trends of the connected domain are further analyzed in combination with multi-frame time series information. Suspicious regions with severe continuous deformation or abnormal position jumps are eliminated, thereby screening out candidate regions with higher credibility and consistent with pupil characteristics, providing reliable data support for subsequent pupil boundary fitting and trajectory tracking. S4: Pupil contour feature edge extraction: For the retained pupil candidate area, the Canny edge detection operator is used to extract the edge point set of the pupil contour; S5: Least squares ellipse fitting based on geometric minimum distance: First, extract contour points with sufficient density from the point set, calculate the covariance matrix of these points relative to the contour centroid, perform eigenvalue decomposition on the covariance matrix, and use the maximum and minimum eigenvalues and their corresponding eigenvectors to preliminarily estimate the major axis, minor axis direction, and rotation angle of the ellipse. Subsequently, substitute the initial estimation results into the weighted residual minimization fitting model, and accurately calculate the ellipse parameters through iterative optimization to ensure the error between the fitted ellipse and the actual pupil boundary point set is minimized. At the same time, record the pupil center position in the time series to achieve real-time extraction and visual output of the pupil trajectory; S6: Real-time extraction of pupil center and trajectory description: Record the finally determined ellipse center coordinates as the pupil center position, and perform time series recording on the calculation results of multiple consecutive frames. Superimpose the pupil center trajectory on the original video frame in real time and output it, enabling the observer to intuitively view the pupil movement trajectory in real time, thereby realizing dynamic trajectory tracking and visual plotting of the pupil center in the video sequence; after all trajectories are described, the trajectory video will be saved; S7: Plotting the curve of the change in pupil area size over time before and after white light excitation: After obtaining the pupil area of each frame, select several frames at the beginning of the video as the baseline, calculate the baseline average value to evaluate the initial pupil area state, and use methods such as moving window average, weighted smoothing, or wavelet denoising to smooth and filter the area data to eliminate high-frequency noise interference, thereby plotting the curve of the change in pupil area size over time before and after white light excitation.

2. A real-time extraction and trajectory tracking image processing method for closed-eye pupil center according to claim 1, characterized in that, In step S1), the adaptive smoothing filter uses one of median filtering and Gaussian filtering.

3. A real-time extraction and trajectory tracking image processing method for closed-eye pupil center according to claim 1, characterized in that, In step S1), the average brightness will vary due to the environmental factors of the acquired image, and the average brightness under white light excitation is higher than that without white light.

4. A real-time extraction and trajectory tracking image processing method for closed-eye pupil center according to claim 1, characterized in that, In step S2), the structural element for morphological operation processing uses a disk structural element with a radius of R, and the operation sequence of morphological operations is: closing operation, filling, opening operation.

5. A real-time extraction and trajectory tracking image processing method for closed-eye pupil center according to claim 1, characterized in that, In step S3), for each connected component, the setting of the eccentricity, circularity, and minimum area threshold will vary due to the different sizes of the pupils; since the closed eyes' pupils are affected by stray light and have a quasi-circular shape, elliptical fitting is used, with a relatively high circularity, and the setting range is 0.7 - 0.

9.

6. A real-time extraction and trajectory tracking image processing method for the closed-eye pupil center according to claim 1, characterized in that, In step S4), the convolution kernel size of the Canny edge detection operator is 3×3.

7. A real-time extraction and trajectory tracking image processing method for closed-eye pupil center according to claim 1, characterized in that, In step S5), the specific implementation steps of the algorithm are as follows: 1) Initialize ellipse parameters: Calculate the center point: Center the boundary point coordinates: Calculate the covariance matrix C: Eigenvalue decomposition: Perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvectors v1, v2 of the eigenvalues λ1, λ2: Cv = λv Calculate the major and minor axes: Calculate the major semi-axis a0 and minor semi-axis b0 through the eigenvalues: Calculate the ellipse rotation angle φ: where v 11 and v 21 are the x and y components of the eigenvector v1, Ensure the major and minor axes are correct: if a < b a, b = b, a φ = φ + π / 2 2) Calculate the weights: where: n is the number of data points, (x i , y i ) are the coordinates of each data point, 3) Define the objective function: where: P = [p1, p2, p3, p4, p5] are the ellipse parameters to be optimized, 4) Minimize the objective function: Iterate through fminsearch and terminate when the error ε < 10 -7 ​ 5) Extract the fitting result: Center: Semi-major axis: Minor semi-axis: Rotation angle: .

Images