Iris acquisition method and device based on eye movement prediction, equipment and storage medium

Abstract

The invention provides an iris collection method and device based on eye movement prediction, equipment and a storage medium, and the method comprises the steps: collecting image frames and event streams in a mixed manner through an event camera and an RGB camera, obtaining eye feature point coordinates through key point detection, and converting the event streams into an event frame sequence; performing spatial-temporal feature extraction and filtering processing on the eye feature point coordinates and the event frame sequence by using a time sequence prediction network, and predicting eyeball positions and gazing directions at future moments; calculating the optimal optical axis direction of iris imaging according to the predicted eyeball position and gazing direction, and converting the optimal optical axis direction into control parameters such as a pan-tilt angle and a lens focal length of an imaging system; and synchronously adjusting the imaging system and triggering acquisition when the prediction time arrives. According to the method, active iris tracking acquisition is realized by predicting the eyeball movement track, the problem of response lag of traditional passive tracking is solved, and the success rate of iris acquisition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to an iris acquisition method and device based on eye movement prediction, equipment and a storage medium. BACKGROUND

[0002] Iris recognition technology is widely used in the field of identity authentication due to its high precision and uniqueness. Traditional iris acquisition systems usually require users to keep their eyes in a fixed position and gaze in a specific direction. This passive acquisition method has the problems of high user cooperation requirement and low acquisition success rate. With the popularity of mobile devices and long-distance recognition scenarios, users often move their heads and rotate their eyeballs during the acquisition process, causing the iris to frequently deviate from the optimal acquisition area of the imaging system, resulting in blurred images or incomplete iris information.

[0003] In the prior art, some solutions use eye movement tracking technology to track the eyeball position in real time, but these solutions only perform passive tracking and adjust the imaging system only when the eyeball movement is detected, which has the problem of response lag. Since the human eye moves quickly, especially the saccadic movement can complete a large angle rotation within tens of milliseconds, the passive tracking method cannot ensure that the iris is always in the optimal imaging position, which affects the success rate of iris acquisition and the image quality. SUMMARY

[0004] The main purpose of the present application is to solve the technical problem of low acquisition success rate caused by response lag of eyeball movement in the existing iris acquisition technology. The present application provides an iris acquisition method based on eye movement prediction, which comprises: processing the acquired image frames and event streams, obtaining eye feature point coordinates through key point detection, and converting the event streams into event frame representations to obtain eye feature point coordinates and event frame sequences; According to the eye feature point coordinates and the event frame sequences, extracting spatio-temporal features through a time series prediction network and performing filtering processing to obtain the eyeball position and gaze direction at the predicted time; According to the eyeball position at the predicted time and the gaze direction, calculating the optical axis direction of iris imaging and converting it into control parameters of the imaging system; controlling the imaging system according to the control parameters of the imaging system, triggering acquisition at the predicted time, and obtaining iris images.

[0005] The present application also provides an iris acquisition device based on eye movement prediction, which comprises: The feature extraction module is configured to process the collected image frames and event stream, obtain eye feature point coordinates through key point detection, and convert the event stream into an event frame representation to obtain eye feature point coordinates and an event frame sequence. The trajectory prediction module is configured to extract spatio-temporal features through a time series prediction network and perform filtering processing according to the eye feature point coordinates and the event frame sequence to obtain an eyeball position and a gaze direction at a predicted time. The parameter conversion module is configured to calculate an optical axis direction of iris imaging according to the eyeball position at the predicted time and the gaze direction, and convert the optical axis direction into control parameters of an imaging system. The active acquisition module is configured to control the imaging system according to the control parameters of the imaging system, trigger acquisition at the predicted time, and obtain an iris image.

[0006] The application further provides an iris acquisition device based on eye movement prediction, comprising a memory and at least one processor, the memory storing instructions, and the memory and the at least one processor being interconnected by a circuit; the at least one processor calling the instructions in the memory to enable the iris acquisition device based on eye movement prediction to perform the steps of the iris acquisition method based on eye movement prediction.

[0007] The application further provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to perform the steps of the iris acquisition method based on eye movement prediction.

[0008] The iris acquisition method, device, equipment and storage medium based on eye movement prediction described above acquire image frames and event streams through a hybrid event camera and RGB camera, obtain eye feature point coordinates through key point detection, and convert the event stream into an event frame sequence; a time series prediction network is used to extract spatio-temporal features and perform filtering processing on the eye feature point coordinates and the event frame sequence to predict an eyeball position and a gaze direction at a future time; the best optical axis direction of iris imaging is calculated according to the predicted eyeball position and gaze direction, and is converted into control parameters such as the pan-tilt angle and lens focal length of an imaging system; the imaging system is adjusted synchronously and acquisition is triggered when the predicted time arrives. This method realizes active iris tracking acquisition by predicting the eyeball movement trajectory, solves the response lag problem of traditional passive tracking, and improves the success rate of iris acquisition.

[0009] Other features and advantages of the application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application. The objects and other advantages of the application will be realized and attained by the structure particularly pointed out in the description, claims, and drawings.

[0010] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the first embodiment of the iris acquisition method based on eye movement prediction in this invention. Figure 2 This is a schematic diagram of a second embodiment of the iris acquisition method based on eye movement prediction in this invention. Figure 3 This is a schematic diagram of an embodiment of the iris acquisition device based on eye movement prediction in this invention. Figure 4 This is a schematic diagram of one embodiment of the iris acquisition device based on eye movement prediction in this invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0014] To facilitate understanding of this embodiment, a detailed description of an iris acquisition method based on eye-tracking prediction disclosed in this embodiment of the invention will be provided first. For example... Figure 1 As shown, this method includes the following steps: 101. Process the acquired image frames and event stream, obtain the coordinates of eye feature points through key point detection, and convert the event stream into an event frame representation to obtain the eye feature point coordinates and event frame sequence; In this embodiment, the process of processing the acquired image frames and event streams, obtaining eye feature point coordinates through keypoint detection, and converting the event stream into event frame representations to obtain eye feature point coordinates and an event frame sequence includes: performing eye region detection on the image frames to obtain eye image regions, locating keypoints in the eye image regions, and outputting a preset number of eye feature point coordinates; extracting corresponding image blocks from the eye image regions based on the eye feature point coordinates to obtain reference image blocks; accumulating the event stream according to a time window, mapping the accumulated events spatially into image representations to obtain an event frame sequence; and extracting event image blocks at corresponding positions from the event frame sequence based on the eye feature point coordinates to obtain an event frame sequence corresponding to the reference image blocks.

[0015] Specifically, when performing eye region detection on an image frame, a cascaded detection approach can be used. First, a face detection algorithm is used to locate the face region in the image frame. This face detection algorithm can employ a deep learning-based detector, such as MTCNN or RetinaFace. After detecting the face region, an eye image region containing both eyes is cropped based on the relative positions of facial key points. This eye image region is typically expanded outwards from the center point of both eyes by a predetermined pixel range to ensure complete coverage of the eyes and surrounding features.

[0016] For the localization of eye feature points, this embodiment employs a regression-based keypoint detection method. The acquired eye image region is input into a pre-trained keypoint detection network, which can directly regress and output the coordinates of a preset number of eye feature points. In this embodiment, the preset number can be set to 6 keypoints, including the center of the left pupil, the center of the right pupil, the inner corner of the left eye, the inner corner of the right eye, the outer feature point of the left eye, and the outer feature point of the right eye. The selection of these 6 keypoints is optimized; compared to using all 68 facial keypoints, this configuration can reduce computational complexity while maintaining tracking accuracy and reducing the cumulative propagation of errors among multiple keypoints. The output of the keypoint detection network is a normalized coordinate value, which needs to be mapped back to the actual pixel coordinate system of the eye image region to obtain the coordinates of the eye feature points.

[0017] After obtaining the coordinates of eye feature points, corresponding image patches are extracted from the eye image region based on the coordinates of each feature point. For each feature point, a rectangular region of a fixed size is cropped centered on that feature point as a reference image patch. The size of the reference image patch can be set to 31×31 pixels or larger to contain sufficient local texture information. These reference image patches will serve as templates for subsequent feature tracking, used to match corresponding positions in the event frame sequence. To improve the robustness of matching, the reference image patches can be preprocessed, such as by grayscale normalization or histogram equalization, to eliminate the influence of illumination variations.

[0018] For event stream processing, this embodiment uses a time window accumulation method to convert asynchronous event streams into synchronous event frame representations. Each pixel of the event camera outputs an event when it detects a brightness change exceeding a preset threshold. This event includes a timestamp, pixel coordinates, and polarity information. The polarity information indicates the direction of the brightness change: positive polarity indicates increased brightness, and negative polarity indicates decreased brightness. During time window accumulation, a fixed-length time window Δt is set. The length of this time window can be determined based on the speed characteristics of eye movement, typically set to 10 to 30 milliseconds. Within each time window, all events generated within that time period are collected and mapped onto a two-dimensional image plane according to the spatial coordinates of the events. For each pixel location, the number of events occurring at that location within the time window is counted, and the events are accumulated separately based on their polarity. Positive and negative polarity events can be accumulated to different channels, or distinguished by positive and negative signs and accumulated to the same channel. Through this accumulation method, discrete event streams can be converted into a two-dimensional representation similar to an image, i.e., event frames. Over time, event frames are continuously generated using a sliding window method, forming an event frame sequence.

[0019] After obtaining the event frame sequence, event image patches corresponding to the reference image patch need to be extracted from the event frames. Since there may be spatial differences between the event camera and the RGB camera, camera calibration and coordinate mapping are required first. Using pre-calibrated camera intrinsic and extrinsic parameters, a mapping relationship between the RGB image coordinate system and the event camera coordinate system can be established. Based on the position of the eye feature point coordinates in the RGB image, its corresponding position in the event frame is calculated, and an event image patch of the same size as the reference image patch is extracted using this corresponding position as the center. For each event frame in the event frame sequence, the event image patch corresponding to each feature point is extracted, forming a sequence of event image patches that corresponds to the reference image patch in both time and space. These event image patch sequences will serve as input to the temporal prediction network for subsequent eye position prediction.

[0020] It's important to note that the choice of time window length Δt during the conversion of event streams into event frames significantly impacts the final tracking performance. A shorter time window provides higher temporal resolution, capturing more subtle eye movement changes, but may result in fewer events accumulated in a single event frame, leading to a weaker image signal. A longer time window accumulates more events, enhancing the image signal, but reduces temporal resolution and may miss rapid eye movements. Therefore, an appropriate time window length must be selected based on the specific application scenario and the speed characteristics of eye movements. In practice, the time window length can be dynamically adjusted according to lighting conditions, appropriately increasing the window length under low-light conditions to accumulate sufficient events.

[0021] Furthermore, an adaptive adjustment mechanism for the event threshold can be introduced during the event stream accumulation process. The threshold θ for event generation by the event camera determines the degree of brightness change that triggers an event. This threshold can be dynamically adjusted under different lighting conditions: a standard threshold is used under normal lighting conditions; under low lighting conditions, the threshold is lowered to increase the event generation frequency, thus ensuring sufficient event information for tracking even in dark environments. The threshold adjustment can be automatic based on the measurement results of ambient light intensity or the statistical characteristics of event generation frequency.

[0022] Furthermore, the step of accumulating the event stream by time window and mapping the accumulated events spatially into an image representation to obtain an event frame sequence includes: extracting the timestamp and spatial coordinates of each event in the event stream, and marking the event polarity according to the direction of brightness change; segmenting the event stream by time according to a preset time window length, grouping events within the same time period into multiple event time periods; accumulating the events within each event time period by spatial coordinates, accumulating positive and negative polarity events to their corresponding pixel positions respectively, to obtain an event accumulation map for each time period; and arranging the event accumulation maps in chronological order to obtain an event frame sequence.

[0023] Specifically, the event camera employs an asynchronous pixel-level response mechanism, where each pixel independently monitors changes in received light intensity. When the light intensity change at a pixel location exceeds a preset threshold, that pixel immediately outputs an event. Each event contains four basic pieces of information: a timestamp, spatial coordinates, polarity, and optional auxiliary information. The timestamp records the precise moment the event occurred, typically with a microsecond-level temporal resolution; the spatial coordinates represent the position of the pixel that generated the event on the image plane; and the polarity indicates the direction of the brightness change, with positive polarity indicating an increase in brightness and negative polarity indicating a decrease in brightness.

[0024] When extracting event information, the event stream processing module receives the event data stream output in real time from the event camera. This data stream is a time-ordered sequence of events. Because the events are generated asynchronously, the time intervals between adjacent events are not uniform. Dense events occur during rapid eye movements or drastic changes in lighting, while events are sparse when the camera is stationary or under stable lighting. Therefore, these asynchronous events need to be synchronized and converted into a sequence of event frames with fixed time intervals.

[0025] For the division of time windows, this embodiment adopts a sliding time window mechanism. The time window length is set to a preset value, and the time windows are divided sequentially according to fixed time intervals, starting from the beginning of the event stream. In actual implementation, to ensure the continuity of timing information, overlapping sliding windows can also be used, i.e., there is a certain overlap between adjacent windows. This overlapping mechanism can prevent events from being truncated at the window boundaries, improving the integrity of timing information.

[0026] When segmenting an event stream over time, the event sequence is traversed, and the time window to which each event belongs is determined based on its timestamp. In this way, a continuous event stream can be divided into multiple discrete time periods, each containing all events generated within that time window. It's important to note that during rapid eye movements, a single time window may contain thousands or even tens of thousands of events; while when the eye is at rest, the number of events within a time window may only be a few dozen. This dynamic change in the number of events reflects the differences in eye movement states.

[0027] For each event time period, the events within it need to be accumulated according to spatial coordinates to generate an event accumulation map for that time period. The event accumulation map is a two-dimensional matrix with the same resolution as the event camera, and all initial values ​​are set to zero. During the accumulation process, all events within the time period are traversed, and for each event, the value at the corresponding position in the accumulation map is updated according to its spatial coordinates and polarity. This embodiment adopts a dual-channel accumulation method, that is, the event accumulation map contains two channels, the first channel accumulates positive polarity events, and the second channel accumulates negative polarity events. When a positive polarity event is encountered, the value is accumulated at the corresponding position in the first channel; when a negative polarity event is encountered, the value is accumulated at the corresponding position in the second channel. Through this separate accumulation method, the polarity information of the events can be preserved, so that the event accumulation map can simultaneously reflect the spatial distribution of brightness increases and decreases.

[0028] It's important to note that the value of each pixel in the event accumulation map reflects the activity level of brightness changes at that location within the time window. A higher value indicates more dramatic brightness changes or more frequent occurrences at that location. In eye-tracking scenarios, eye feature points such as the pupil edge and corners of the eyes generate dense events during eye movement due to significant contrast, thus appearing with higher values ​​in the event accumulation map. Conversely, the sclera (white of the eye) or background areas, lacking texture variation, generate fewer events, resulting in lower or zero values ​​in the accumulation map.

[0029] After generating event cumulative graphs for each time period, these graphs are arranged in chronological order to form an event frame sequence. The event frame sequence is essentially a three-dimensional data structure containing time, spatial height, and spatial width dimensions. This event frame representation transforms asynchronous event streams into a synchronous, video-like representation, allowing it to be directly input into deep learning models such as convolutional neural networks for processing.

[0030] Furthermore, to enhance the visualization and feature representation capabilities of event frames, the event accumulation map can be normalized. Since the number of events varies significantly across different time windows, the range of directly accumulated values ​​can differ by tens or even hundreds of times. Normalization maps the accumulated values ​​to a uniform range, eliminating the impact of varying event density across different time periods and improving the stability of feature extraction.

[0031] 102. Based on the coordinates of the eye feature points and the event frame sequence, the spatiotemporal features are extracted and filtered using a temporal prediction network to obtain the eye position and gaze direction at the prediction time. In this embodiment, the step of extracting spatiotemporal features and performing filtering processing based on the eye feature point coordinates and the event frame sequence to obtain the eye position and gaze direction at the predicted time includes: inputting the reference features corresponding to the eye feature point coordinates and the event frame sequence into the temporal prediction network to perform multi-scale spatial feature extraction and temporal state encoding to obtain a temporal feature representation; calculating the motion information of the eye feature points through the prediction module based on the temporal feature representation to obtain the eye feature point coordinates at the predicted time; inputting the eye feature point coordinates at the predicted time into the filter for state estimation, and obtaining the three-dimensional position of the eye through state transition and observation update calculations; and obtaining the gaze direction through geometric calculation based on the target key points in the eye feature point coordinates at the predicted time.

[0032] Specifically, the temporal prediction network adopts a multi-module collaborative architecture to process multi-source information from RGB images and event frame sequences. The network mainly consists of three core components: a feature pyramid network, convolutional long short-term memory units, and an attention fusion module.

[0033] First, image regions corresponding to each feature point are extracted from a reference image patch based on the coordinates of the eye feature points. For each feature point, a fixed-size image patch is cropped centered on that point; these patches contain texture and structural information surrounding the feature point. The extracted reference image patches and event frame sequences are then input into a temporal prediction network for feature extraction and temporal encoding to obtain a temporal feature representation. This temporal feature representation integrates spatial structural information and temporal motion information.

[0034] After obtaining the temporal feature representation, it is input into the prediction module to calculate the motion information of eye feature points. The prediction module typically consists of several convolutional and fully connected layers, used to map the temporal feature representation into the displacement of feature points. The output of this module is the displacement vector of each eye feature point relative to the current time step at the prediction time, including horizontal and vertical displacement components. The predicted displacement vector is added to the current eye feature point coordinates to obtain the eye feature point coordinates at the prediction time. These predicted coordinates reflect the network's estimate of the eye position at future time steps.

[0035] It should be noted that predictions relying solely on neural networks may contain noise and errors, especially under conditions of rapid eye movement or occlusion. To improve the stability and accuracy of predictions, this embodiment introduces a filter for post-processing the prediction results. The filter employs the Kalman filter algorithm, a classic state estimation method that can fuse predicted and observed values ​​to reduce the impact of noise.

[0036] The Kalman filter maintains a state vector containing the three-dimensional position and velocity information of the eye. At each time step, the filter first predicts the state based on the motion model, that is, predicting the current state based on the state and velocity of the previous time step. The state transition process uses a linear or nonlinear motion model; commonly used models include uniform motion models or uniformly accelerated motion models. Then, the filter uses the predicted eye feature point coordinates output by the neural network as observations, mapping the state space to the observation space through the observation model. The observation update step calculates the residual between the predicted and observed values ​​and corrects the state based on the Kalman gain. The Kalman gain reflects the degree of confidence in the predicted and observed values; a larger gain indicates more reliable observations, and the state update is more dependent on the observed values; a smaller gain indicates more reliable predictions, and the state update is more dependent on the predicted values.

[0037] Through an iterative process of state transitions and observation updates, the Kalman filter gradually converges to the optimal estimate. The state vector output by the filter contains the three-dimensional position information of the eyeball, that is, the coordinates of the eyeball center in three-dimensional space. This three-dimensional position is recovered from the two-dimensional image coordinates using binocular stereo vision or monocular depth estimation methods. Binocular stereo vision methods use the disparity information of the left and right cameras to calculate depth, while monocular depth estimation methods infer depth through learning or geometric constraints.

[0038] After obtaining the coordinates of the eye feature points at the predicted time, the gaze direction is further calculated. The gaze direction represents the direction of the optical axis of the eye and can be derived from the geometric relationships of the eye feature points. In this embodiment, target key points are selected from the predicted eye feature point coordinates, including the pupil center points and corner feature points of both eyes. Based on the three-dimensional coordinates of the pupil center points, the gaze vector of both eyes can be calculated. In actual calculation, an eye coordinate system is established through the geometric relationship formed by the pupil center points and the corner feature points, and then the direction of the optical axis of the eye is calculated. The final output gaze direction is represented in the form of a three-dimensional vector, the direction of which points from the eye center to the gaze target, which is the gaze direction of the eye at the predicted time.

[0039] Furthermore, the temporal prediction network includes a feature pyramid network, a convolutional long short-term memory unit, and an attention fusion module. The step of inputting the reference features corresponding to the eye feature point coordinates and the event frame sequence into the temporal prediction network for multi-scale spatial feature extraction and temporal state encoding to obtain a temporal feature representation includes: extracting reference feature blocks from the reference image block based on the eye feature point coordinates, and extracting event feature blocks at corresponding positions in the event frame sequence; inputting the reference feature blocks and event feature blocks into the feature pyramid network respectively, and performing feature extraction through convolutional layers of different scales to obtain multi-scale reference feature maps and multi-scale event feature maps; inputting the multi-scale event feature maps into the convolutional long short-term memory unit in chronological order, updating the hidden state through a gating mechanism to obtain a temporal hidden state; and inputting the multi-scale reference feature maps and the temporal hidden state into the attention fusion module, calculating attention weights, and performing weighted fusion on the multi-scale event feature maps to obtain a temporal feature representation.

[0040] Specifically, before feature extraction, it is necessary to locate and extract local regions corresponding to eye feature points from the reference image patch and the event frame sequence. Based on the coordinates of the eye feature points, rectangular regions are cropped from the reference image patch with each feature point as the center, according to a preset window size, to obtain the reference feature block. This window size is usually set to 31 by 31 pixels to ensure that sufficient local texture information is included. For each frame in the event frame sequence, a rectangular region of the same size is extracted at the corresponding position based on the eye feature point coordinates to obtain the event feature block sequence.

[0041] It should be noted that when extracting event feature blocks, the potential spatial differences between the RGB camera and the event camera need to be considered. Coordinate mapping is performed using pre-calibrated camera parameters to accurately transform the coordinates of feature points in the RGB image to the event camera coordinate system, ensuring that the extracted event feature blocks strictly correspond spatially to the reference feature blocks.

[0042] After extracting the reference feature blocks and event feature blocks, they are input into a feature pyramid network for multi-scale feature extraction. The feature pyramid network employs an encoder-decoder structure. The encoder part progressively reduces the spatial resolution of the feature maps through multiple convolutional and pooling operations while increasing the number of feature channels. The decoder part fuses high-level semantic features with low-level detail features through upsampling and lateral connection operations. After processing by the feature pyramid network, a sequence of multi-scale reference feature maps and multi-scale event feature maps is obtained, containing multiple levels such as original resolution, half-resolution, and quarter-resolution.

[0043] The multi-scale event feature map sequence is input into a convolutional long short-term memory (LSTM) unit in chronological order for temporal encoding. Each LTM unit maintains two state tensors: the hidden state and the cell state. These states are updated using three gating mechanisms: an input gate, a forget gate, and an output gate. The input gate controls the extent to which current input information is written into the cell state; the forget gate controls the extent to which historical information is forgotten; and the output gate controls the extent to which the cell state information is output as the hidden state. The activation values ​​of these three gates are calculated using convolution operations and the sigmoid activation function. After iterative processing of the entire sequence, the LTM unit outputs the hidden state corresponding to each time step, i.e., the temporal hidden state.

[0044] After obtaining the multi-scale reference feature maps and temporal hidden states, they are input into the attention fusion module for adaptive feature fusion. The attention fusion module first concatenates or adds the multi-scale reference feature maps and temporal hidden states, then inputs them into a small convolutional network to calculate an attention weight map. This weight map represents the importance of different positions and channels in the multi-scale event feature maps. The attention weight map is then multiplied element-wise with the multi-scale event feature maps to achieve feature weighting; features with larger weights are enhanced, while features with smaller weights are suppressed. The weighted feature maps are then fused through global pooling and convolution operations to obtain a temporal feature representation. This representation integrates the spatial structure information of the reference image, the temporal motion information of the event sequence, and the key features selected by the attention mechanism.

[0045] 103. Based on the eye position at the predicted time and the gaze direction, calculate the optical axis direction of iris imaging and convert it into control parameters of the imaging system; In this embodiment, calculating the optimal optical axis direction for iris imaging based on the eye position and gaze direction at the predicted time is a crucial step in achieving high-quality iris acquisition. Iris imaging quality is significantly affected by eye posture and the angle between the imaging optical axis and the iris image. When the optical axis is perpendicular to the iris plane, the clearest iris image with minimal distortion can be obtained.

[0046] First, the normal vector of the iris plane is calculated based on the predicted gaze direction. The gaze direction vector represents the direction of the eye's optical axis; ideally, the iris plane is approximately perpendicular to the optical axis. Using an anatomical model of the eye, the geometric relationship between the gaze direction and the iris plane normal vector can be established. The direction of the iris plane normal vector is basically consistent with the gaze direction, but it needs to be corrected for factors such as corneal curvature and pupil decentering. Based on the corrected normal vector, the equation representing the iris plane in three-dimensional space can be determined.

[0047] After obtaining the iris plane equation, and combining it with the three-dimensional position of the eye at the predicted time, the target direction that the imaging optical axis should align with is calculated. The optimal imaging optical axis should be aligned with the iris plane normal vector, that is, the optical axis direction vector is parallel or antiparallel to the normal vector. Through vector operations, the direction vector of the imaging optical axis can be determined. This direction vector points from the optical center of the imaging device to the center of the iris of the eye, and its direction is aligned with the iris plane normal vector.

[0048] It should be noted that in practical applications, the imaging device is usually mounted on a gimbal, which can rotate with two degrees of freedom in the horizontal and vertical directions. Therefore, it is necessary to convert the optical axis direction vector in three-dimensional space into the azimuth and pitch angles of the gimbal. The azimuth angle represents the rotation angle of the gimbal in the horizontal plane, and the pitch angle represents the rotation angle of the gimbal in the vertical plane. The conversion process uses a spherical coordinate system transformation method, and calculates the corresponding azimuth and pitch angle values ​​using inverse trigonometric functions based on the three components of the optical axis direction vector.

[0049] In addition to adjusting the gimbal angle, the lens focal length also needs to be adjusted based on the eye distance to ensure a clear iris image. The spatial distance from the eye to the imaging device can be calculated based on the eye's three-dimensional position at the predicted moment. This distance is calculated using the Euclidean distance between the eye's position coordinates and the imaging device's position coordinates. Depending on the lens's imaging principle and depth-of-field characteristics, different shooting distances require different focal length settings. By establishing a pre-defined distance-focal length mapping, the optimal focal length value for the current distance can be found or calculated. The focal length adjustment is the difference between the target focal length and the current focal length.

[0050] The calculated gimbal azimuth angle, gimbal pitch angle, and lens focal length adjustment are combined to form the control parameters of the imaging system. These control parameters are output in the form of digital or analog signals to drive the gimbal motor and lens focusing mechanism.

[0051] 104. Control the imaging system according to the control parameters of the imaging system, trigger acquisition at the predicted time, and obtain an iris image.

[0052] In this embodiment, after receiving the control parameters, it is necessary to coordinate and control the various actuators of the imaging device to ensure that the iris is exactly in the optimal imaging position when the predicted time arrives.

[0053] First, the gimbal rotation angle parameters are sent to the gimbal controller. The gimbal controller parses the received azimuth and pitch angle commands and drives the corresponding stepper motors or servo motors to perform the rotation. The gimbal rotates horizontally according to the azimuth angle and vertically according to the pitch angle. The motor rotation speed and acceleration need to be planned based on the angle change and remaining time to ensure that attitude adjustment is completed before the predicted time. To avoid the impact of mechanical vibration on image quality, the gimbal requires a short stabilization time after reaching the target position, typically set to tens of milliseconds.

[0054] Simultaneously, the focal length adjustment is sent to the lens controller. The lens controller drives the focusing mechanism to move the lens group, changing the focal length of the optical system. The focusing mechanism typically uses a voice coil motor or stepper motor to adjust the focal length by moving the lens elements. The speed of focal length adjustment also needs to be planned based on the adjustment amount and time window. Modern lenses typically support fast focusing, capable of completing a wide range of focal length adjustments within hundreds of milliseconds.

[0055] While the gimbal and lens are adjusting, a timer module monitors the time difference between the current moment and the predicted moment in real time. This time difference is obtained by subtracting the system's current timestamp from the predicted moment's timestamp. When the time difference is less than a preset threshold, such as less than fifty or one hundred milliseconds, it indicates that the predicted moment is about to arrive, triggering the image acquisition preparation process. This process includes checking whether the gimbal and lens have been adjusted, confirming whether mechanical vibration has stabilized, and setting the camera's exposure parameters.

[0056] When the time difference decreases to near zero or enters the preset acquisition time window, an acquisition command is sent to the camera sensor. Upon receiving the command, the camera sensor immediately begins exposure; light is projected onto the sensor surface through the lens, and the sensor converts the light signal into an electrical signal. The exposure time is set automatically or manually depending on the ambient light intensity, typically between a few milliseconds and tens of milliseconds. After exposure, the sensor reads the signal and performs analog-to-digital conversion to generate digital image data. This image data contains the texture information of the iris, which is the acquired iris image.

[0057] It should be noted that, to improve the success rate of image acquisition, multiple frames can be acquired around the predicted time, and then the best frame can be selected using a quality assessment algorithm. Quality assessment can be based on indicators such as image sharpness, iris visible area, and illumination uniformity.

[0058] In this embodiment, image frames and event streams are acquired through a hybrid acquisition method using both an event camera and an RGB camera. Keypoint detection is used to obtain the coordinates of eye feature points, and the event stream is converted into an event frame sequence. A temporal prediction network is then used to extract and filter spatiotemporal features from the eye feature point coordinates and the event frame sequence to predict the eye position and gaze direction at future moments. Based on the predicted eye position and gaze direction, the optimal optical axis direction for iris imaging is calculated and converted into control parameters such as the gimbal angle and lens focal length of the imaging system. The imaging system is then synchronously adjusted and acquisition is triggered when the predicted moment arrives. This method achieves active iris tracking acquisition by predicting the eye movement trajectory, solving the response lag problem of traditional passive tracking and improving the success rate of iris acquisition.

[0059] Please see Figure 2 Another embodiment of the iris acquisition method based on eye movement prediction in this application includes: 201. Process the acquired image frames and event stream, obtain the coordinates of eye feature points through key point detection, and convert the event stream into event frame representation to obtain the eye feature point coordinates and event frame sequence; 202. Based on the coordinates of the eye feature points and the event frame sequence, the spatiotemporal features are extracted and filtered using a temporal prediction network to obtain the eye position and gaze direction at the prediction time. In this embodiment, steps 201-202 are similar to steps 101-102, and will not be described again here.

[0060] 203. Calculate the normal vector of the iris plane based on the gaze direction to obtain the iris plane equation, and calculate the imaging optical axis direction vector aligned with the normal vector based on the iris plane equation and the eye position at the prediction time. In this embodiment, the gaze direction vector represents the direction of the eye's optical axis in three-dimensional space. This vector can be represented by three components, corresponding to the three axes of the three-dimensional coordinate system. According to the optical model of the eye, the iris plane is approximately perpendicular to the eye's optical axis; therefore, the direction of the normal vector of the iris plane is basically consistent with the direction of the gaze direction vector. In actual calculations, the gaze direction vector can be directly used as the initial estimate of the iris plane normal vector. However, due to the refractive effect of the cornea and the eccentricity of the pupil relative to the center of eye rotation, there is a certain deviation between the actual iris plane normal vector and the gaze direction vector. To improve calculation accuracy, the normal vector can be corrected according to the eye's anatomical parameters; the correction amount is usually within a few degrees.

[0061] After obtaining the iris plane normal vector, and combining it with the eye position at the prediction time, a spatial equation for the iris plane can be established. The iris plane can be represented using a point-normal form equation, that is, the plane is determined by the iris center point and the normal vector. The iris center point can be approximated as the projection point of the eye center in the gaze direction, or determined based on the three-dimensional coordinates of the pupil center. The general form of the plane equation includes the three components of the normal vector and the coordinates of a point on the plane. These parameters uniquely determine the position and orientation of the iris plane in three-dimensional space.

[0062] To obtain optimal iris imaging quality, the imaging optical axis should be aligned with the normal vector of the iris plane; that is, the optical axis direction vector should be parallel to the normal vector. The starting point of the imaging optical axis direction vector is the optical center of the imaging device, and the ending point points towards the center of the iris. This direction vector is calculated by vector subtraction, that is, subtracting the coordinates of the optical center of the imaging device from the coordinates of the iris center point to obtain a vector pointing from the optical center to the iris center. The direction of this vector is the initial optical axis direction. To achieve alignment with the normal vector, this vector needs to be adjusted so that its direction is consistent with the normal vector of the iris plane. The adjustment method can be vector projection or rotation transformation, rotating the initial optical axis direction vector to a direction parallel to the normal vector. The resulting imaging optical axis direction vector after adjustment is the final target optical axis direction.

[0063] 204. Convert the imaging optical axis direction vector into azimuth and pitch angles in the imaging system coordinate system to obtain the gimbal rotation angle; In this embodiment, the imaging optical axis direction vector is a three-dimensional vector represented in the world coordinate system or camera coordinate system. To control the gimbal rotation to the target attitude, this vector needs to be converted into azimuth and pitch angles in the gimbal coordinate system. The gimbal coordinate system typically has the center of rotation of the gimbal as its origin and the horizontal plane as its reference plane. The azimuth angle represents the angle between the projection of the optical axis onto the horizontal plane and the reference direction, while the pitch angle represents the angle between the optical axis and the horizontal plane.

[0064] The transformation process first involves converting the imaging optical axis direction vector from the world coordinate system to the gimbal coordinate system. This transformation is achieved through a coordinate transformation matrix, which contains rotation and translation components, reflecting the relative position and attitude relationship between the two coordinate systems. The transformation matrix is ​​obtained through the gimbal's installation parameters and calibration process. Multiplying the optical axis direction vector by the transformation matrix on the left yields the direction vector in the gimbal coordinate system.

[0065] In the gimbal coordinate system, the azimuth angle is calculated using the horizontal component of the direction vector. Specifically, the azimuth angle is the angle between the projection of the direction vector onto the horizontal plane and the gimbal reference direction. This angle is calculated using the arctangent function, with the input being the ratio of the two horizontal components of the projection vector. The pitch angle is calculated using the angle between the direction vector and the horizontal plane. This angle is calculated using the arcsine or arctangent function, with the input being the ratio of the vertical component of the direction vector to the horizontal projection length.

[0066] It should be noted that the azimuth angle typically ranges from -180 degrees to +180 degrees or 0 degrees to 360 degrees, while the pitch angle typically ranges from -90 degrees to +90 degrees. The calculated angle value is the target angle to which the gimbal needs to rotate. To obtain the actual rotation amount of the gimbal, the current angle of the gimbal also needs to be acquired, read through an encoder or angle sensor. The gimbal rotation angle is equal to the target angle minus the current angle; this rotation angle can be positive or negative, representing forward and reverse rotation, respectively.

[0067] 205. Calculate the distance from the imaging system to the eye based on the eye position at the predicted time, calculate the lens focal length adjustment based on the distance, and output the gimbal rotation angle and the lens focal length adjustment as control parameters of the imaging system. In this embodiment, the eye position is given in three-dimensional coordinates, and the position of the imaging device is also represented in three-dimensional coordinates. The distance between the two is calculated using the Euclidean distance formula, which involves squaring the coordinate differences in the three coordinate axes, summing them, and then taking the square root. The calculated distance value represents the straight-line distance from the center of the eye to the optical center of the imaging device.

[0068] Based on the imaging principle of lenses, different shooting distances require different focal length settings to achieve a clear image of the subject. The focal length of a lens satisfies the lens imaging formula in relation to the object distance and image distance. For adjustable-focus lenses, focal length adjustment is achieved by moving the lens group. In practical applications, the distance-focal length mapping relationship of the lens is usually pre-calibrated, establishing a lookup table or fitting function. Based on the calculated eye distance, the corresponding target focal length value is obtained through lookup table interpolation or function calculation.

[0069] Once the target focal length is determined, the focal length adjustment amount needs to be calculated. The focal length adjustment amount equals the target focal length minus the current focal length. The current focal length can be read through the lens's encoder or position sensor. The focal length adjustment amount can be positive or negative; a positive value indicates that the focal length needs to be increased, and a negative value indicates that the focal length needs to be decreased. This adjustment amount is converted into the number of motor steps or the movement distance of the lens focusing mechanism, and sent as a control command to the lens controller.

[0070] After calculating the gimbal rotation angle and lens focal length adjustment, these two sets of parameters are packaged into control parameters for the imaging system. The control parameters are output in a structured data format, including fields such as azimuth, pitch, and focal length adjustment. These parameters are transmitted to the hardware control module via a communication interface, driving the gimbal and lens to perform corresponding actions.

[0071] 206. Send the gimbal rotation angle to the gimbal controller to drive the gimbal to rotate according to the azimuth and pitch angles to obtain the adjusted gimbal attitude; In this embodiment, after receiving the azimuth and pitch angle commands, the gimbal controller first performs command parsing and validity checks. The validity check includes verifying whether the angle values ​​are within the gimbal's mechanical movement range and whether they exceed the gimbal's maximum rotational speed and acceleration limits. After passing the checks, the gimbal controller calculates the motion trajectory based on the current attitude and the target attitude. The motion trajectory planning uses a trapezoidal velocity curve or an S-shaped velocity curve, dividing the rotation process into three stages: acceleration, constant speed, and deceleration, to reduce mechanical shock and vibration.

[0072] The gimbal controller drives the horizontal and vertical axis motors to rotate. The horizontal axis motor rotates according to the azimuth angle command, and the vertical axis motor rotates according to the pitch angle command. The motors employ a closed-loop control method, with the encoder providing real-time feedback on the motor's rotation angle. The controller calculates the control input based on the difference between the feedback value and the target value, and adjusts the motor drive current using a proportional-integral-derivative (PID) control algorithm to achieve precise position control. During rotation, the controller continuously monitors the motor's operating status, including parameters such as speed, current, and temperature, to ensure safe operation.

[0073] Once both axis motors have reached the target position, the gimbal's attitude reaches the predetermined state. At this point, the controller enters hold mode, maintaining gimbal attitude stability through a position closed loop to resist external disturbances. To eliminate the impact of mechanical vibration on imaging, the gimbal needs to wait for a settling time after reaching the target position, typically set to 50 to 100 milliseconds. After the settling time is complete, the gimbal controller sends an attitude adjustment completion signal to the main control unit, indicating that the gimbal is ready for image acquisition. The adjusted gimbal attitude ensures that the imaging optical axis points to the predicted eye position, creating optimal geometric conditions for iris acquisition.

[0074] 207. Send the lens focal length adjustment amount to the lens controller to drive the lens focusing mechanism to adjust the focal length and obtain the adjusted lens focal length. In this embodiment, after receiving the focal length adjustment command, the lens controller converts the adjustment amount into a physical displacement of the focusing mechanism. The focusing mechanism typically uses a voice coil motor or a stepper motor to drive the lens groups to move along the optical axis, changing the relative positions between the lens groups to achieve focal length adjustment. Voice coil motors are characterized by fast response speed and high positioning accuracy, making them suitable for fast focusing scenarios. Stepper motors have the advantages of low cost and simple control, making them suitable for ordinary focusing needs.

[0075] The lens controller calculates the target position of the motor based on the focal length adjustment. For a voice coil motor, the target position is converted into a drive current value; for a stepper motor, the target position is converted into a number of step pulses. The controller sends a drive signal to the motor, and the motor begins to move the lens group. During the movement, the position of the lens group is monitored in real time by a position sensor or encoder. The controller adjusts the drive signal based on the position feedback to achieve precise position control. Modern lenses are typically equipped with autofocus sensors, which can use phase detection or contrast detection technology to assist focusing and improve focusing accuracy.

[0076] After the lens group moves to the target position, the lens reaches the predetermined focal length. The controller verifies whether the focal length has accurately reached the target value through sensors, and performs fine adjustments if there is a deviation. After the focal length adjustment is completed, the lens controller sends a focusing completion signal back to the main control unit. The adjusted lens focal length matches the distance to the eyeball, ensuring that the iris plane is within the depth of field of the lens, resulting in a clear iris image. It should be noted that, to further improve image quality, the aperture size can also be automatically adjusted according to lighting conditions to balance depth of field and light intake.

[0077] 208. Monitor the time difference between the current time and the predicted time. When the time difference is less than a preset threshold, trigger an image acquisition command and control the camera sensor to perform exposure and reading according to the image acquisition command to obtain an iris image.

[0078] In this embodiment, the main control unit continuously monitors the current time using a high-precision timer. The current time is obtained by reading the system clock, which typically has a time resolution of milliseconds or microseconds. The main control unit compares the current time with the predicted time and calculates the time difference between them. The time difference is the predicted time minus the current time. A positive time difference indicates that the predicted time has not yet arrived, while a time difference close to zero or negative indicates that the predicted time has arrived or has passed.

[0079] The main control unit sets a preset threshold as the acquisition trigger condition, which is typically set to 50 to 100 milliseconds. When the detected time difference is less than the preset threshold, it indicates that the predicted time is about to arrive, and the main control unit executes the acquisition preparation process. The preparation process includes checking whether the gimbal and lens have been adjusted, verifying whether the mechanical vibration has stabilized, and confirming that the camera sensor is in a ready state. After the preparation process is completed, the main control unit generates an image acquisition command and sends it to the camera control module.

[0080] Upon receiving the acquisition command, the camera control module immediately controls the sensor to begin exposure. During exposure, the sensor's photosensitive element converts the received light signal into electrical charge, the magnitude of which is proportional to the light intensity and exposure time. The exposure time is automatically set based on the ambient light intensity and target brightness, typically ranging from a few milliseconds to tens of milliseconds. After exposure, the sensor performs charge reading, sequentially reading the charge of each pixel and converting it into a digital signal. The reading process uses progressive scanning or a rolling shutter mode, requiring tens of milliseconds to complete the reading of all pixels. The read digital signal undergoes analog-to-digital conversion and preprocessing by the image processing chip, including noise reduction, gain adjustment, and color correction, ultimately generating complete iris image data. This image data is stored in a cache for subsequent use by the iris recognition algorithm.

[0081] In this embodiment, image frames and event streams are acquired through a hybrid acquisition method using both an event camera and an RGB camera. Keypoint detection is used to obtain the coordinates of eye feature points, and the event stream is converted into an event frame sequence. A temporal prediction network is then used to extract and filter spatiotemporal features from the eye feature point coordinates and the event frame sequence to predict the eye position and gaze direction at future moments. Based on the predicted eye position and gaze direction, the optimal optical axis direction for iris imaging is calculated and converted into control parameters such as the gimbal angle and lens focal length of the imaging system. The imaging system is then synchronously adjusted and acquisition is triggered when the predicted moment arrives. This method achieves active iris tracking acquisition by predicting the eye movement trajectory, solving the response lag problem of traditional passive tracking and improving the success rate of iris acquisition.

[0082] The above describes the iris acquisition method based on eye movement prediction in the embodiments of the present invention. The following describes the iris acquisition device based on eye movement prediction in the embodiments of the present invention. Please refer to [link to relevant documentation] for this iris acquisition device. Figure 3 One embodiment of the iris acquisition device based on eye movement prediction in this invention includes: The feature extraction module 301 is used to process the acquired image frames and event streams, obtain the coordinates of eye feature points through key point detection, and convert the event stream into event frame representation to obtain the eye feature point coordinates and event frame sequence; The trajectory prediction module 302 is used to extract spatiotemporal features and perform filtering processing through a temporal prediction network based on the coordinates of the eye feature points and the event frame sequence to obtain the eye position and gaze direction at the prediction time. The parameter conversion module 303 is used to calculate the optical axis direction of iris imaging based on the eye position and the gaze direction at the predicted time, and convert it into control parameters of the imaging system. The active acquisition module 304 is used to control the imaging system according to the control parameters of the imaging system, trigger acquisition at the predicted time, and obtain an iris image.

[0083] In this embodiment of the invention, the iris acquisition device based on eye-tracking prediction operates the aforementioned iris acquisition method based on eye-tracking prediction. The device acquires image frames and event streams using a hybrid approach of an event camera and an RGB camera. It obtains the coordinates of eye feature points through keypoint detection and converts the event stream into an event frame sequence. A temporal prediction network is used to extract and filter the spatiotemporal features of the eye feature point coordinates and the event frame sequence to predict the eye position and gaze direction at future moments. Based on the predicted eye position and gaze direction, the optimal optical axis direction for iris imaging is calculated and converted into control parameters such as the gimbal angle and lens focal length of the imaging system. When the predicted moment arrives, the imaging system is synchronously adjusted and acquisition is triggered. This method achieves active iris tracking acquisition by predicting the eye movement trajectory, solving the response lag problem of traditional passive tracking and improving the success rate of iris acquisition.

[0084] above Figure 3 The iris acquisition device based on eye movement prediction in the embodiments of the present invention will be described in detail from the perspective of unitized functional entities. The iris acquisition device based on eye movement prediction in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0085] Figure 4This is a schematic diagram of the structure of an eye-tracking prediction-based iris acquisition device 300 provided in an embodiment of the present invention. The eye-tracking prediction-based iris acquisition device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 410 (e.g., one or more processors) and a memory 420, and one or more storage media 430 (e.g., one or more mass storage devices) storing application programs 333 or data 432. The memory 420 and storage media 430 can be temporary or persistent storage. The program stored in the storage media 430 may include one or more units (not shown in the diagram), each unit may include a series of instruction operations on the eye-tracking prediction-based iris acquisition device 400. Furthermore, the processor 410 may be configured to communicate with the storage media 430 and execute the series of instruction operations in the storage media 430 on the eye-tracking prediction-based iris acquisition device 400 to implement the steps of the aforementioned eye-tracking prediction-based iris acquisition method.

[0086] The iris acquisition device 400 based on eye-tracking prediction may also include one or more power supplies 440, one or more wired or wireless network interfaces 450, one or more input / output interfaces 460, and / or one or more operating systems 431, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 4 The illustrated structure of the eye-tracking prediction-based iris acquisition device does not constitute a limitation on the eye-tracking prediction-based iris acquisition device provided by the present invention. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0087] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the iris acquisition method based on eye movement prediction.

[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0089] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An iris acquisition method based on eye-tracking prediction, characterized in that, The iris acquisition method based on eye movement prediction includes: The acquired image frames and event streams are processed, the coordinates of eye feature points are obtained through key point detection, and the event stream is converted into an event frame representation to obtain the eye feature point coordinates and event frame sequence; Based on the coordinates of the eye feature points and the event frame sequence, spatiotemporal features are extracted and filtered using a temporal prediction network to obtain the eye position and gaze direction at the prediction time. Based on the eye position and the gaze direction at the predicted time, the optical axis direction of iris imaging is calculated and converted into control parameters of the imaging system; The imaging system is controlled according to its control parameters, and acquisition is triggered at the predicted time to obtain an iris image.

2. The iris acquisition method based on eye movement prediction according to claim 1, characterized in that, The process of processing the acquired image frames and event streams, obtaining eye feature point coordinates through key point detection, and converting the event stream into event frame representation to obtain eye feature point coordinates and event frame sequences includes: The image frame is subjected to eye region detection to obtain the eye image region, and key point localization is performed on the eye image region to output a preset number of eye feature point coordinates; Based on the coordinates of the eye feature points, the corresponding image blocks are extracted from the eye image region to obtain reference image blocks; The event stream is accumulated according to a time window, and the accumulated events are spatially mapped into an image representation to obtain an event frame sequence; Based on the coordinates of the eye feature points, the event image block at the corresponding position is extracted from the event frame sequence to obtain the event frame sequence corresponding to the reference image block.

3. The iris acquisition method based on eye movement prediction according to claim 2, characterized in that, The step of accumulating the event stream according to a time window, mapping the accumulated events spatially into an image representation, and obtaining an event frame sequence includes: Extract the timestamp and spatial coordinates for each event in the event stream, and mark the event polarity according to the direction of brightness change; The event stream is segmented into time segments according to a preset time window length, and events within the same time period are grouped together to obtain multiple event time periods. Events within each time period are accumulated according to spatial coordinates, and positive and negative events are accumulated to their corresponding pixel positions to obtain the event accumulation map for each time period; Arrange the event accumulation graph in chronological order to obtain the event frame sequence.

4. The iris acquisition method based on eye movement prediction according to claim 2, characterized in that, The step of extracting spatiotemporal features and performing filtering processing based on the eye feature point coordinates and the event frame sequence through a temporal prediction network to obtain the eye position and gaze direction at the prediction time includes: The reference features corresponding to the coordinates of the eye feature points and the event frame sequence are input into the temporal prediction network to perform multi-scale spatial feature extraction and temporal state encoding to obtain a temporal feature representation. Based on the temporal feature representation, the motion information of the eye feature points is calculated by the prediction module to obtain the coordinates of the eye feature points at the prediction time; The coordinates of the eye feature points at the predicted time are input into the filter for state estimation. The three-dimensional position of the eyeball is obtained through state transition and observation update calculation. The gaze direction is obtained through geometric calculation based on the target key points in the eye feature point coordinates at the predicted time.

5. The iris acquisition method based on eye movement prediction according to claim 4, characterized in that, The temporal prediction network includes a feature pyramid network, convolutional long short-term memory units, and an attention fusion module; The step of inputting the reference features corresponding to the eye feature point coordinates and the event frame sequence into the temporal prediction network to perform multi-scale spatial feature extraction and temporal state encoding to obtain a temporal feature representation includes: Reference feature blocks are extracted from the reference image block based on the coordinates of the eye feature points, and event feature blocks are extracted from the corresponding positions in the event frame sequence; The reference feature block and the event feature block are respectively input into the feature pyramid network, and features are extracted through convolutional layers of different scales to obtain multi-scale reference feature maps and multi-scale event feature maps. The multi-scale event feature map is input into the convolutional long short-term memory unit in chronological order, and the hidden state is updated through a gating mechanism to obtain the temporal hidden state; The multi-scale reference feature map and the temporal hidden state are input into the attention fusion module, attention weights are calculated, and the multi-scale event feature map is weighted and fused to obtain a temporal feature representation.

6. The iris acquisition method based on eye movement prediction according to claim 1, characterized in that, The step of calculating the optical axis direction of iris imaging based on the eye position at the predicted time and the gaze direction, and converting it into control parameters for the imaging system, includes: The normal vector of the iris plane is calculated based on the gaze direction to obtain the iris plane equation. Based on the iris plane equation and the eye position at the prediction time, the imaging optical axis direction vector aligned with the normal vector is calculated. The imaging optical axis direction vector is converted into the azimuth and elevation angles in the imaging system coordinate system to obtain the gimbal rotation angle. The distance from the imaging system to the eye is calculated based on the eye position at the predicted time, and the lens focal length adjustment is calculated based on the distance. The gimbal rotation angle and the lens focal length adjustment are then output as control parameters of the imaging system.

7. The iris acquisition method based on eye movement prediction according to claim 6, characterized in that, The step of controlling the imaging system according to the control parameters of the imaging system and triggering acquisition at the predicted time to obtain an iris image includes: The gimbal rotation angle is sent to the gimbal controller, which drives the gimbal to rotate according to the azimuth and pitch angles to obtain the adjusted gimbal attitude. The lens focal length adjustment amount is sent to the lens controller to drive the lens focusing mechanism to adjust the focal length, thereby obtaining the adjusted lens focal length. The system monitors the time difference between the current time and the predicted time. When the time difference is less than a preset threshold, it triggers an image acquisition command and controls the camera sensor to perform exposure and reading according to the image acquisition command to obtain an iris image.

8. An iris acquisition device based on eye movement prediction, characterized in that, The iris acquisition device based on eye movement prediction includes: The feature extraction module is used to process the acquired image frames and event streams, obtain the coordinates of eye feature points through key point detection, and convert the event stream into event frame representation to obtain the eye feature point coordinates and event frame sequence; The trajectory prediction module is used to extract spatiotemporal features and perform filtering processing through a temporal prediction network based on the coordinates of the eye feature points and the event frame sequence to obtain the eye position and gaze direction at the prediction time. The parameter conversion module is used to calculate the optical axis direction of iris imaging based on the eye position and the gaze direction at the predicted time, and convert it into control parameters of the imaging system. An active acquisition module is used to control the imaging system according to the control parameters of the imaging system, trigger acquisition at the predicted time, and obtain an iris image.

9. An iris acquisition device based on eye movement prediction, characterized in that, The iris acquisition device based on eye movement prediction includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the eye-tracking prediction-based iris acquisition device to perform the steps of the eye-tracking prediction-based iris acquisition method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instruction is executed by the processor, it implements the steps of the iris acquisition method based on eye movement prediction as described in any one of claims 1-7.

Images