Anterior segment eye movement tracking method, apparatus, and electronic device

Abstract

The application provides a front segment eye movement tracking method and device and electronic equipment, relates to the technical field of eye movement tracking, synchronously collects a pupil diagram and an OCT structure diagram, and proposes a comprehensive eye movement tracking strategy considering pupil center offset and the OCT structure diagram, ensures that in the process of optical coherence tomography imaging, even if eye movement or blinking occurs, high-quality image acquisition can be maintained, and the uniformity of orthoview cross-sectional images is improved. Corresponding compensation measures are implemented on the fundus structure diagram based on a preset pupil center offset threshold, the requirement of the system on extremely high real-time performance is reduced, a wider hardware platform can support efficient eye movement compensation. In the OCT acquisition process, the image is dynamically monitored, the eye movement or closed eye condition can be synchronously evaluated, the data collected each time is ensured to meet the preset quality standard, the consistency and reliability of the whole imaging process are improved, and clear and accurate OCT images can be obtained regardless of the change of the patient state.

Description

Technical Field

[0001] This invention relates to the field of eye-tracking technology, and in particular to anterior segment eye-tracking method, apparatus, and electronic device. Background Technology

[0002] Optical coherence tomography (OCT) can achieve micron-level three-dimensional imaging and has been widely used in ophthalmic clinical examinations. In anterior segment OCT imaging, three-dimensional scanning is essential to obtain accurate tissue structure and morphological characterization as well as to image the anterior segment microvessels. However, three-dimensional scanning often cannot be completed in a short time. Patients may experience involuntary eye movements during the scanning process, such as microsaccades, drift, or tremors. In addition, some patients may experience frequent blinking. These factors can greatly affect the accuracy and repeatability of the scan, increasing the difficulty of diagnosis and treatment. Summary of the Invention

[0003] The purpose of this invention is to provide an anterior segment eye-tracking method, device, and electronic device that, by combining a pupil center localization algorithm with blink detection results, comprehensively evaluates the state of the current frame and processes the data in a timely manner during the acquisition process, thereby obtaining higher quality OCT structure maps.

[0004] In a first aspect, embodiments of the present invention provide an anterior segment eye-tracking method, the method comprising: controlling a preset pupil camera to acquire images of a target eye at a preset acquisition frequency to obtain each frame of pupil images of the target eye; and synchronously controlling a preset OCT camera to scan the target eye based on the acquisition frequency of the pupil camera to obtain a fundus structure map corresponding to each frame of pupil images; determining the pupil center of each frame of pupil images, and determining the pupil offset corresponding to each frame of pupil center according to a preset pupil image template; identifying whether the fundus structure map meets preset acquisition conditions; if it meets the conditions, adjusting the scanning parameters of the OCT camera when the pupil offset exceeds a preset offset threshold; otherwise, performing image compensation on the fundus structure map corresponding to the pupil image based on the pupil offset.

[0005] In conjunction with the first aspect, the present invention provides a first implementation of the first aspect, wherein the step of adjusting the scanning parameters of the OCT camera includes: converting the pupil offset into the galvanometer offset voltage of the OCT camera; and adjusting the galvanometer position of the OCT camera based on the galvanometer offset voltage.

[0006] In conjunction with the first aspect, this invention provides a second implementation of the first aspect, wherein the step of converting the pupil offset into the galvanometer offset voltage of the OCT camera includes: calculating the pixel ratio of the pupil image and the fundus structure image; calculating the pixel offset corresponding to the pixel ratio based on the offset; converting the pixel offset into an actual physical distance; calculating the adjustment amount of the voltage range of the OCT camera's galvanometer based on the actual physical distance; and determining the galvanometer offset voltage of the OCT camera.

[0007] In conjunction with the first aspect, the present invention provides a third implementation of the first aspect, wherein the step of performing image compensation on the fundus structure image corresponding to the pupil image based on the pupil offset includes: recording the pupil offset and performing translation compensation on the fundus structure image corresponding to the pupil image based on the pupil offset.

[0008] In conjunction with the first aspect, the present invention provides a fourth implementation of the first aspect, wherein the above method further includes: acquiring a pupil image from a pupil camera and a fundus structure map corresponding to the pupil image; reconstructing the fundus structure map; determining whether the fundus structure map meets preset scanning requirements; wherein the scanning requirements are determined based on the pupil integrity of the target eyeball or the signal intensity of the fundus structure map; if so, determining the pupil image as a pupil image template.

[0009] In conjunction with the first aspect, the present invention provides a fifth implementation of the first aspect, wherein the step of determining the pupil center of each frame of pupil image includes: performing image preprocessing on the pupil image; extracting the pupil edge from the pupil image; and performing image fitting on the pupil image based on the pupil edge to determine the pupil center of the pupil image.

[0010] In conjunction with the first aspect, this embodiment of the invention provides a sixth implementation of the first aspect, wherein the step of identifying whether the fundus structure image meets the preset acquisition conditions includes: determining the signal intensity deviation between fundus structure images corresponding to each adjacent pupil image; determining whether the signal intensity deviation is less than a preset intensity deviation threshold; and if so, determining that the fundus structure image meets the preset acquisition conditions.

[0011] In conjunction with the first aspect, the present invention provides a seventh implementation of the first aspect, wherein the steps of controlling a preset pupil camera to acquire images of a target eyeball at a preset acquisition frequency, and synchronously controlling a preset OCT camera to scan the target eyeball based on the acquisition frequency of the pupil camera, include: acquiring a preset scan start signal; triggering the pupil camera to start based on the scan start signal; and triggering the OCT camera to start.

[0012] Secondly, embodiments of the present invention provide an anterior segment eye-tracking device, wherein the device includes: a data acquisition module, used to control a preset pupil camera to acquire images of a target eyeball at a preset acquisition frequency to obtain each frame of pupil image of the target eyeball; and, based on the acquisition frequency of the pupil camera, synchronously control a preset OCT camera to scan the target eyeball to obtain a fundus structure map corresponding to each frame of pupil image; a data processing module, used to determine the pupil center of each frame of pupil image, and determine the pupil offset corresponding to each frame of pupil center according to a preset pupil image template; a recognition module, used to recognize whether the fundus structure map meets preset acquisition conditions; and an execution module, used to adjust the scanning parameters of the OCT camera when the fundus structure map meets the preset acquisition conditions and when the pupil offset exceeds a preset offset threshold, otherwise, perform image compensation on the fundus structure map corresponding to the pupil image according to the pupil offset.

[0013] Thirdly, embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the anterior segment eye-tracking method of any of the above embodiments.

[0014] The embodiments of this invention bring the following beneficial effects: The anterior segment eye-tracking method, device, and electronic device provided by the embodiments of this invention simultaneously acquire pupil images and OCT structural images, and propose a comprehensive eye-tracking strategy that considers pupil center offset and OCT structural images. This ensures that high-quality image acquisition can be maintained even if eye movement or blinking occurs during optical coherence tomography (OCT) imaging, and can improve the uniformity of the emphyseal cross-section image. Specifically, the embodiments of this invention combine pupil images and structural images for analysis to determine the degree of pupil center offset. Corresponding compensation measures are only implemented when a threshold is reached, reducing the system's requirements for extremely high real-time performance and enabling a wider range of hardware platforms to support efficient eye-tracking compensation. Furthermore, the embodiments of this invention dynamically monitor images during OCT acquisition, simultaneously assessing eye movement or eye closure, and immediately executing corresponding processing strategies. This not only ensures that the data acquired each time meets preset quality standards, but also improves the consistency and reliability of the entire imaging process. The compensation mechanism provided by this invention can flexibly respond to different types of involuntary eye movements (such as microsaccades, drift, or tremor) and blinking events, ensuring that clear and accurate OCT images can be obtained regardless of changes in the patient's condition.

[0015] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 A flowchart of an anterior segment eye-tracking method provided in an embodiment of the present invention;

[0019] Figure 2 A flowchart of another anterior segment eye-tracking method provided in an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a control signal provided in an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of the logical framework for anterior segment eye tracking provided in an embodiment of the present invention;

[0022] Figure 5 A logic diagram of anterior segment eye tracking provided in an embodiment of the present invention;

[0023] Figure 6 This is a schematic diagram of the structure of an anterior segment eye-tracking device provided in an embodiment of the present invention;

[0024] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0027] Because 3D scanning requires a large scanning area and cannot be completed quickly, patients may blink or move their eyes during the process. If data is collected during blinking, there will be no signal in the OCT structural image, directly leading to the loss of some OCT data. Involuntary eye movements can cause misalignment in the collected OCT structural images, affecting the accuracy of subsequent quantitative data and the repeatability of the scan. In existing technologies, the impact of eye movements and blinking is reduced by increasing the speed of OCT scanning, but eye tracking technology is still required, which increases the complexity and cost of the equipment. Alternatively, eye tracking devices can be used to monitor eye movements in real time and adjust the scanning parameters of the OCT system based on the monitored data to achieve image compensation. However, this method has very high real-time requirements for the equipment and cannot account for blinking and the quality of the OCT structural images during eye movements.

[0028] To address the aforementioned issues, embodiments of the present invention provide an anterior segment eye-tracking method, apparatus, and electronic device that can ensure high-quality image acquisition even during eye movements or blinking during optical coherence tomography (OCT) imaging.

[0029] To facilitate understanding, an anterior segment eye-tracking method provided by an embodiment of the present invention will be described first. Figure 1 A flowchart of an anterior segment eye-tracking method provided by an embodiment of the present invention is shown, as follows: Figure 1 As shown, the method includes the following steps:

[0030] Step S102: Control the preset pupil camera to acquire images of the target eyeball at a preset acquisition frequency to obtain each frame of pupil image of the target eyeball; and, based on the acquisition frequency of the pupil camera, synchronously control the preset OCT camera to scan the target eyeball to obtain the fundus structure map corresponding to each frame of pupil image.

[0031] Step S104: Determine the pupil center of each frame of pupil image, and determine the pupil offset corresponding to the pupil center of each frame according to the preset pupil image template.

[0032] Step S106: Identify whether the fundus structure image meets the preset acquisition conditions.

[0033] Step S108: If the condition is met, when the pupil offset exceeds the preset offset threshold, adjust the scanning parameters of the OCT camera; otherwise, perform image compensation on the fundus structure map corresponding to the pupil image based on the pupil offset.

[0034] Pupil images are primarily used to monitor eye movements in the horizontal and vertical directions, while OCT structural images provide a three-dimensional view from the surface to deep tissues, allowing the system to detect these subtle changes. For anterior segment OCTA imaging, multiple scans of the same location are required to extract blood flow signals. Pupil images are planar images, while OCT structural images are depth-oriented images; the two are not always entirely equivalent. This invention evaluates and ensures the quality of each scan by analyzing the depth-oriented images generated by OCT, using OCT structural images to assist in judging the acquisition effect. It relies not only on the two-dimensional planar information provided by pupil images but also provides a more comprehensive understanding of the actual eye movements and their impact on imaging.

[0035] In practical implementation, this invention embodiment strictly synchronizes the acquisition actions of the pupil camera and the OCT camera to provide accurate time-aligned data. Combined with the pupil center localization algorithm and the judgment of OCT structural map acquisition conditions, the state of the current frame is comprehensively evaluated, thereby obtaining a higher-quality OCT structural map. In optical coherence tomography (OCT) imaging, after the OCT camera acquires data from the eye, it needs to undergo a series of processing steps to reconstruct a detailed OCT structural map. In one embodiment, the OCT camera and the pupil camera are synchronously controlled to acquire data from the eye. The data acquired by the OCT camera is reconstructed to obtain a fundus structural map. The quality of the reconstructed fundus structural map is then judged. When the fundus structural map meets the acquisition conditions, a corresponding compensation strategy is applied based on the pupil offset; otherwise, the eye is re-acquired, and the fundus structural map is re-determined to ensure that the data acquired each time meets the preset quality standards. Specifically, this invention embodiment evaluates the degree of eye movement by calculating the change in the pupil center relative to the reference position in each frame, i.e., the pupil offset. Furthermore, in the eye-tracking process, the embodiments of the present invention allow for a certain threshold deviation in the pupil center offset. Only when the deviation exceeds the corresponding threshold will the scanning parameters of the OCT camera be adjusted according to the preset compensation logic. Otherwise, the fundus structure image is compensated by post-processing, such as compensating the fundus structure image during reconstruction, which reduces the real-time requirements of the acquisition and analysis of the preceding and following frames.

[0036] In summary, the anterior segment eye-tracking method provided by this invention simultaneously acquires pupil images and OCT structural images, and proposes a comprehensive eye-tracking strategy considering pupil center offset and OCT structural images. This ensures high-quality image acquisition even during optical coherence tomography (OCT) imaging, even if eye movements or blinking occur. Specifically, this invention combines pupil images and structural images for analysis to determine the degree of pupil center offset. Only when the offset exceeds a threshold is mirror movement implemented for re-acquisition, reducing the system's requirements for extremely high real-time performance and enabling a wider range of hardware platforms to support efficient eye-tracking compensation. Furthermore, this invention dynamically monitors images during OCT acquisition, simultaneously assesses eye movements or eye closure, and executes corresponding processing strategies in real time. This not only ensures that each acquired data meets preset quality standards but also improves the consistency and reliability of the entire imaging process. The compensation mechanism provided by this invention can flexibly address different types of involuntary eye movements (such as microsaccades, drift, or tremors) and blinking events, ensuring clear and accurate OCT images regardless of changes in the patient's condition.

[0037] Based on the above embodiments, this invention provides another anterior segment eye-tracking method. Figure 2 A flowchart of another anterior segment eye-tracking method provided by an embodiment of the present invention is shown, with reference to... Figure 2 The method includes the following steps:

[0038] Step S202: Control the preset pupil camera to acquire images of the target eyeball at a preset acquisition frequency to obtain each frame of pupil image of the target eyeball; and, based on the acquisition frequency of the pupil camera, synchronously control the preset OCT camera to scan the target eyeball to obtain the fundus structure map corresponding to each frame of pupil image.

[0039] In the specific implementation, a preset scan start signal is obtained, and based on the scan start signal, the pupil camera is triggered to start, and the OCT camera is triggered to start. Specifically, Figure 3 A schematic diagram of the control signals for synchronous acquisition by the pupil camera and OCT camera is shown. The trigger signals of the pupil camera, the OCT camera, and the vibration waveform of the galvanometer are all referenced using the same clock. Then, the trigger signals of the pupil camera and the OCT camera are synchronized. Figure 3 In the figure, from top to bottom, are the scan start signal, pupil camera trigger signal, OCT frame trigger signal, and galvanometer drive waveform. It can be seen from the figure that the pupil camera trigger signal and the OCT frame trigger signal trigger the acquisition synchronously. In addition, when the scan trigger is paused, the compensation voltage of the galvanometer can be applied to the galvanometer drive waveform to achieve galvanometer compensation.

[0040] Step S204: Determine the pupil center of each frame of pupil image, and determine the pupil offset corresponding to the pupil center of each frame according to the preset pupil image template.

[0041] A high-quality pupil image can be selected as a template. This image should have a clear pupil boundary and uniform illumination. The position of the pupil center is marked in the template image as a reference point for subsequent matching. Further, a preset matching method can be used to match the pupil center of the pupil image with the pupil image template, calculating the pupil offset. The offset can be calculated based on the coordinates of the pupil center in the template image and the coordinates of the pupil center in the current frame image. Specifically, the pupil image from the pupil camera and the corresponding fundus structure image can be acquired; the fundus structure image is reconstructed; it is determined whether the fundus structure image meets the preset scanning requirements; if so, the pupil image is selected as the pupil image template. The scanning requirements are determined based on the pupil integrity of the target eye or the signal intensity of the fundus structure image. That is, the presence of a closed eye or poor OCT quality is determined based on the OCT structure image. If the OCT fundus structure image does not show a closed eye or poor quality, the corresponding pupil image is used as the image template. In one implementation, the first pupil image that meets both the requirements of the pupil image and the fundus structure image can be used as the pupil image template. Alternatively, the pupil image template can be updated gradually to ensure that the pupil center does not shift relative to the image.

[0042] This invention preprocesses a pupil image to extract the pupil edge. Based on the pupil edge, image fitting is performed to determine the pupil center. Specifically, before pupil extraction, image preprocessing helps reduce noise and enhance pupil contrast. Binarization is performed using a suitable threshold algorithm, followed by extraction of the pupil edge using traditional edge detection, Hough circle transform, deep learning, and other methods. Finally, the pupil center is obtained through fitting. Specifically, median filtering can be used to reduce noise in the pupil image, and then binarization is performed based on an adaptive threshold to convert the grayscale image into a binary image, highlighting the pupil region. Furthermore, morphological opening and closing operations are performed to remove small noise points and fill holes, optimizing connected regions in the binary image. The pupil region is obtained using the largest connected component; connected components are labeled in the binary image to find all connected regions. The area of ​​each connected region is calculated, and the region with the largest area is selected as the pupil region. Finally, the pupil contour is accurately fitted using a least-squares ellipse to obtain the ellipse parameters.

[0043] Step S206: Determine the signal intensity deviation between the fundus structure images corresponding to each adjacent pupil image; determine whether the signal intensity deviation is less than a preset intensity deviation threshold; if so, determine that the fundus structure image meets the preset acquisition conditions.

[0044] Because 3D scanning requires a large scanning area and cannot be completed quickly, patients may experience involuntary eye movements during the process, leading to misalignment in the acquired OCT structural images or blinking. If data is acquired during blinking, there will be no signal in the OCT structural image, directly resulting in the loss of some OCT data, affecting the accuracy of subsequent quantification data and the repeatability of the scan, leading to poor uniformity in the reconstructed emphyseal cross-section. Blinking can be judged using pupil images, but it is difficult to determine a metric for blinking based on pupil images. How much pupil obstruction will cause a significant weakening of the OCT structural image intensity, which may lead to missed blinks. Sometimes, after eye movement or blinking resumes, the OCT structural image signal weakens, resulting in non-uniformity in the 3D scan en-face image (emphyseal cross-section). This embodiment of the invention directly judges blinking based on significant changes in OCT structural image intensity, which has higher accuracy. This invention incorporates OCT intensity change detection during eye tracking. While calculating the pupil center offset, it reconstructs the OCT structural map and uses the intensity of both the pupil map and the OCT signal to determine if blinking has occurred, filtering out frames with significantly weakened intensity. A threshold for intensity change between adjacent frames is used for selection. If the change exceeds the threshold, data loss in critical areas may occur; in this case, the data at that location is rescanned, and blink data is discarded until the blink ends. If the change is less than the threshold, the next scan is performed, improving the uniformity of the emphyseal image. Therefore, this invention incorporates blink detection in addition to pupil center offset calculation during analysis, enabling timely data processing during acquisition.

[0045] Step S208: When the pupil offset exceeds the preset offset threshold, the pupil offset is converted into the galvanometer offset voltage of the OCT camera; based on the galvanometer offset voltage, the position of the galvanometer of the OCT camera is adjusted.

[0046] Step S210: Otherwise, record the pupil offset and perform translation compensation on the fundus structure map corresponding to the pupil image based on the pupil offset.

[0047] OCT structural image enhancement is generally achieved by superimposing multiple images acquired at the same location. Similarly, OCTA reconstruction also involves acquiring multiple OCT images at the same location and then extracting blood flow signals. Both methods have requirements regarding the offset of multiple OCT structural images at the same location; the positional deviation of frames at the same location cannot be too large. This embodiment of the invention filters frames with large offsets in the X and Y directions by setting an offset threshold for multiple OCT structural images at the same location, filtering and re-acquiring frames with large offsets in these directions to improve the success rate of subsequent image registration. In this embodiment, an allowable offset threshold is set. Within this threshold, galvanometer compensation is not required; it can be compensated by translation after OCT image reconstruction. Outside the threshold, compensation is performed by galvanometer movement. If the offset of the re-acquired image is less than the offset threshold, compensation is performed through post-processing. In summary, this embodiment of the invention reduces the real-time requirements of galvanometer compensation and can obtain high-quality OCT structural images for quantization.

[0048] The reconstruction of the OCT structure map is completed by the GPU. After reconstruction, the intensity of the OCT structure map is obtained by calculating the average value of the entire image. Combining the above steps, by setting a threshold for the intensity difference between consecutive frames, it is possible to determine whether blinking has occurred, thereby determining whether to rescan.

[0049] The pupil camera and OCT camera synchronously acquire data and transmit it to the computer. The computer calculates the offset of the pupil center and reconstructs the OCT structure map. Based on the eye-tracking strategy, it determines whether there is a blink and the intensity change of the OCT structure map. It also decides whether the OCT galvanometer needs compensation. If compensation is needed, the offset pixels of the OCT are sent to the lower-level OCT galvanometer control system. The lower-level system converts the pixels into bias voltages to compensate for the galvanometer drive waveform. Correspondingly, Figure 4 The logical framework for anterior segment eye tracking is shown. Figure 5 A logical diagram of anterior segment eye tracking is shown.

[0050] Specifically, the pixel ratio of the pupil image and the fundus structure image is calculated; based on the offset, the pixel offset corresponding to the pixel ratio is calculated; the pixel offset is converted into actual physical distance, and the adjustment amount of the voltage range of the OCT camera's galvanometer is calculated based on the actual physical distance to determine the galvanometer offset voltage of the OCT camera. In practical implementation, the anterior segment OCT uses the pupil image to locate the position of the anterior segment OCT scan, and uses a scaled square target to register the pupil image and the OCT image, adjusting the scanning area of ​​the pupil. Assuming the scanning area is an L*Lmm rectangle, based on the resolution P of the pupil camera... X *P Y The horizontal and vertical pixel dimensions of the pupil camera can be obtained in L / P. X and L / P YThe scanning range of an OCT structural image is determined by the rotation angle of the galvanometer, which in turn is determined by the scanning voltage range of the galvanometer. Adjusting the X-ray galvanometer scanning voltage range V... X and Y-mirror scanning voltage range V Y To obtain an OCT structure image of length L*Lmm. The known region scan OCT structure image resolution is K. X *K Y Then the horizontal and vertical pixel dimensions of the OCT structure diagram are L / K. X and L / K Y The voltage corresponding to each pixel is V. X / K X and V Y / K Y The final pixel ratio of the pupil image and OCT image in the horizontal and vertical directions is K. X / P X and K Y / P Y .

[0051] During eye tracking, pupil images are continuously acquired and the offset Z of the pupil center between two frames is calculated. X and Z Y Then, it is converted into the pixel offset K of the OCT structure map. X *Z X / P X and K Y *Z Y / P Y Based on this offset, the offset voltage of the OCT mirror is converted to Z. X *V X / P X and Z Y *V Y / P Y It compensates for the OCT scan position to achieve the purpose of tracking eye movements.

[0052] In summary, the alternative anterior segment eye-tracking method provided by this invention avoids invalid data acquisition caused by eye movements or blinking by intelligently filtering and post-processing the acquired data, reducing the need for rescanning, thereby saving medical resources and improving diagnostic efficiency. It also reduces scan failures or image quality problems caused by eye movements or blinking, thus alleviating patient discomfort and anxiety, simplifying the doctor's workflow, and improving diagnostic accuracy. Furthermore, this invention uses a pupil camera and an OCT camera to simultaneously acquire data from the eye and proposes a novel eye-tracking strategy. By determining the pupil center of the current frame's pupil image and the template pupil image and calculating the offset of the pupil center, this offset allows for a preset deviation threshold with the OCT galvanometer position. When the offset is less than this threshold, the offset can be compensated after OCT image reconstruction; when it is greater than this threshold, the galvanometer is offset until both are within the deviation threshold, and then post-processing compensation is performed. By introducing the deviation threshold, the real-time requirements for galvanometer compensation are reduced, improving the robustness of eye-tracking.

[0053] Furthermore, embodiments of the present invention also provide an anterior segment eye-tracking device. Figure 6 This diagram illustrates the structure of an anterior segment eye-tracking device according to an embodiment of the present invention. Figure 6 The device includes: a data acquisition module 100, used to control a preset pupil camera to acquire images of the target eyeball at a preset acquisition frequency, obtaining each frame of pupil image of the target eyeball; and, based on the acquisition frequency of the pupil camera, synchronously controlling a preset OCT camera to scan the target eyeball, obtaining the fundus structure map corresponding to each frame of pupil image; a data processing module 200, used to determine the pupil center of each frame of pupil image, and determine the pupil offset corresponding to the pupil center of each frame according to a preset pupil image template; a recognition module 300, used to identify whether the fundus structure map meets the preset acquisition conditions; and an execution module 400, used to adjust the scanning parameters of the OCT camera when the fundus structure map meets the preset acquisition conditions and when the pupil offset exceeds a preset offset threshold, otherwise, to perform image compensation on the fundus structure map corresponding to the pupil image based on the pupil offset.

[0054] The anterior segment eye-tracking device provided in this embodiment of the invention has the same technical features as the anterior segment eye-tracking method provided in the above embodiments, so it can also solve the same technical problems and achieve the same technical effects.

[0055] Furthermore, based on the above embodiments, the aforementioned offset threshold includes an offset threshold; the execution module 400 is also used to convert the pupil offset amount into the galvanometer offset voltage of the OCT camera; and to adjust the position of the galvanometer of the OCT camera based on the galvanometer offset voltage. The execution module 400 is also used to calculate the pixel ratio of the pupil image and the fundus structure image; to calculate the pixel offset amount corresponding to the pixel ratio based on the pupil offset amount; to convert the pixel offset amount into an actual physical distance; and to calculate the adjustment amount of the voltage range of the galvanometer of the OCT camera based on the actual physical distance, thereby determining the galvanometer offset voltage of the OCT camera.

[0056] The aforementioned execution module 400 is also used to record the pupil offset and perform translation compensation on the fundus structure map corresponding to the pupil image based on the pupil offset.

[0057] The aforementioned data processing module 200 is also used to acquire the pupil image from the pupil camera and the corresponding fundus structure image; reconstruct the fundus structure image; determine whether the fundus structure image meets the preset scanning requirements; wherein the scanning requirements are determined based on the pupil integrity of the target eye or the signal intensity of the fundus structure image; if so, the pupil image is determined as the pupil image template.

[0058] The aforementioned data processing module 200 is also used to perform image preprocessing on the pupil image; extract the pupil edge from the pupil image; and perform image fitting on the pupil image based on the pupil edge to determine the pupil center of the pupil image.

[0059] The aforementioned recognition module 300 is also used to determine the signal intensity deviation between the fundus structure images corresponding to each adjacent pupil image; determine whether the signal intensity deviation is less than a preset intensity deviation threshold; and if so, determine that the fundus structure image meets the preset acquisition conditions.

[0060] The aforementioned data acquisition module 100 is also used to acquire a preset scan start signal; based on the scan start signal, to trigger the pupil camera to start, and to trigger the OCT camera to start.

[0061] This invention also provides an electronic device, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor. The processor executes the machine-executable instructions to implement the aforementioned multi-objective comprehensive optimization decision-making method. See also... Figure 7 As shown, the electronic device includes a processor 70 and a memory 71. The memory 71 stores machine-executable instructions that can be executed by the processor 70. The processor 70 executes the machine-executable instructions to implement the above-mentioned multi-objective integrated optimization decision-making method.

[0062] Furthermore, Figure 7The illustrated electronic device also includes a bus 72 and a communication interface 73. The processor 70, communication interface 73, and memory 71 are connected via the bus 72. The memory 71 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk drive. Communication between this system network element and at least one other network element is achieved through at least one communication interface 73 (which can be wired or wireless). The interface can use the Internet, wide area network, local area network, metropolitan area network, etc. The bus 72 can be an ISA (Industrial Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Enhanced Industry Standard Architecture) bus, etc. These buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, Figure 7 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0063] The processor 70 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 70 or by instructions in software form. The processor 70 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 71. Processor 70 reads information from memory 71 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiment. This embodiment also provides a machine-readable storage medium storing machine-executable instructions. When these machine-executable instructions are invoked and executed by the processor, they cause the processor to implement the aforementioned multi-objective comprehensive optimization decision-making method.

[0064] The computer program product of the anterior segment eye-tracking method, device, and electronic device provided in this invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. Specific implementations can be found in the method embodiments and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the preceding method embodiments, and will not be repeated here. Furthermore, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0065] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for anterior segment eye tracking, characterized in that, The method includes: A preset pupil camera is controlled to acquire images of the target eyeball at a preset acquisition frequency to obtain each frame of the pupil image of the target eyeball; and, based on the acquisition frequency of the pupil camera, a preset OCT camera is synchronously controlled to scan the target eyeball to obtain the fundus structure map corresponding to each frame of the pupil image; Determine the pupil center of each frame of the pupil image, and determine the pupil offset corresponding to the pupil center of each frame according to the preset pupil image template; Identify whether the fundus structure image meets the preset acquisition conditions; If the condition is met, when the pupil offset exceeds a preset offset threshold, the scanning parameters of the OCT camera are adjusted; otherwise, image compensation is performed on the fundus structure map corresponding to the pupil image based on the pupil offset. The steps for adjusting the scanning parameters of the OCT camera include: The pupil offset is converted into the galvanometer offset voltage of the OCT camera; The position of the galvanometer of the OCT camera is adjusted based on the galvanometer offset voltage. The step of converting the pupil offset into the galvanometer offset voltage of the OCT camera includes: Calculate the pixel ratio between the pupil image and the fundus structure image; Calculate the pixel offset corresponding to the pixel ratio based on the pupil offset; The pixel offset is converted into an actual physical distance, and the adjustment amount of the voltage range of the OCT camera's galvanometer is calculated based on the actual physical distance to determine the galvanometer offset voltage of the OCT camera. The step of performing image compensation on the fundus structure image corresponding to the pupil image based on the pupil offset includes: Record the pupil offset, and perform translation compensation on the fundus structure map corresponding to the pupil image based on the pupil offset.

2. The method according to claim 1, characterized in that, The method further includes: Acquire the pupil image from the pupil camera, and the corresponding fundus structure image of the pupil image; The aforementioned fundus structure image is reconstructed; Determine whether the fundus structure image meets the preset scanning requirements; wherein, the scanning requirements are determined based on the pupil integrity of the target eye or the signal intensity of the fundus structure image; If so, the pupil image is determined as the pupil image template.

3. The method according to claim 1, characterized in that, The step of determining the pupil center of each frame of the pupil image includes: Perform image preprocessing on the pupil image; Extract the pupil edge from the pupil image; Based on the pupil edge, the pupil image is fitted to determine the pupil center of the pupil image.

4. The method according to claim 1, characterized in that, The step of identifying whether the fundus structure image meets the preset acquisition conditions includes: Determine the signal intensity deviation between the fundus structure images corresponding to each adjacent frame of the pupil image; Determine whether the signal strength deviation is less than a preset strength deviation threshold; If so, determine that the fundus structure image meets the preset acquisition conditions.

5. The method according to claim 1, characterized in that, The steps of controlling a preset pupil camera to acquire images of the target eyeball at a preset acquisition frequency, and synchronously controlling a preset OCT camera to scan the target eyeball based on the acquisition frequency of the pupil camera, include: Obtain the preset scan start signal; Based on the scan start signal, the pupil camera is triggered to start, and the OCT camera is triggered to start.

6. An anterior segment eye-tracking device, characterized in that, The device includes: The data acquisition module is used to control a preset pupil camera to acquire images of the target eyeball at a preset acquisition frequency, thereby obtaining each frame of the pupil image of the target eyeball; and, based on the acquisition frequency of the pupil camera, to synchronously control a preset OCT camera to scan the target eyeball, thereby obtaining a fundus structure map corresponding to each frame of the pupil image. The data processing module is used to determine the pupil center of each frame of the pupil image and to determine the pupil offset corresponding to the pupil center of each frame according to the preset pupil image template. The identification module is used to identify whether the fundus structure image meets the preset acquisition conditions; The execution module is used to adjust the scanning parameters of the OCT camera when the pupil offset exceeds a preset offset threshold, provided that the fundus structure image meets the preset acquisition conditions; otherwise, it performs image compensation on the fundus structure image corresponding to the pupil image based on the pupil offset. The execution module is further configured to: convert the pupil offset into the galvanometer offset voltage of the OCT camera; and adjust the galvanometer position of the OCT camera based on the galvanometer offset voltage; The execution module is further configured to: calculate the pixel ratio of the pupil image and the fundus structure image; calculate the pixel offset corresponding to the pixel ratio based on the pupil offset; convert the pixel offset into an actual physical distance; calculate the adjustment amount of the voltage range of the galvanometer of the OCT camera based on the actual physical distance; and determine the galvanometer offset voltage of the OCT camera. The execution module is further configured to: record the pupil offset and perform translation compensation on the fundus structure map corresponding to the pupil image based on the pupil offset.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the anterior segment eye-tracking method according to any one of claims 1-5.

Images