Anterior segment eye movement tracking method and device and electronic equipment

Abstract

The invention provides an anterior segment eye movement tracking method, an anterior segment eye movement tracking device and electronic equipment, relates to the technical field of eye movement tracking, synchronously acquires a pupil image and an OCT (Optical Coherence Tomography) structural diagram, provides a comprehensive eye movement tracking strategy considering pupil center offset and the OCT structural diagram, and ensures that in an optical coherence tomography imaging process, even if eye movement or blinking occurs, the eye movement can be accurately tracked. And high-quality image acquisition can be maintained, and the uniformity of the front-view cross-section image is improved. Corresponding compensation measures are taken on the fundus structure diagram based on the preset pupil center offset threshold value, the requirement of the system for extremely high real-time performance is lowered, and a wider hardware platform can support efficient eye movement compensation. In the OCT acquisition process, the image is dynamically monitored, the eye movement or eye closing condition can be synchronously evaluated, it is guaranteed that data acquired each time meets the preset quality standard, the consistency and reliability of the whole imaging process are improved, and the clear and accurate OCT image can be obtained no matter how the state of a patient changes.

Description

Technical Field

[0001] The present invention relates to the field of eye tracking technology, and in particular to a method, device and electronic equipment for tracking eye movements of the front segment. Background Art

[0002] Optical coherence tomography (OCT) can achieve micron-level three-dimensional imaging and has been widely used in clinical ophthalmic examinations. In anterior segment OCT imaging, three-dimensional scanning is very necessary to obtain accurate tissue structure and morphological characterization and to image the anterior segment microvasculature. However, three-dimensional scanning cannot be completed in a short period of time. Patients will have involuntary eye movements during the entire scanning process, such as micro-saccades, drifts, or tremors. In addition, some patients will blink frequently. These factors will greatly affect the accuracy and repeatability of the scan and increase the difficulty of diagnosis and treatment. Summary of the invention

[0003] The purpose of the present invention is to provide a method, device and electronic device for front-segment eye tracking, which combines the pupil center positioning algorithm with the blink detection result, comprehensively evaluates the state of the current frame, and processes the data in time during the acquisition process, so as to obtain a higher quality OCT structure image.

[0004] In a first aspect, an embodiment of the present invention provides a front-segment eye tracking method, the method comprising: controlling a preset pupil camera to capture images of a target eyeball at a preset acquisition frequency to obtain each frame of pupil image of the target eyeball; and, based on the acquisition frequency of the pupil camera, synchronously controlling a preset OCT camera to scan the target eyeball to obtain a fundus structure map corresponding to each frame of pupil image; determining the pupil center of each frame of pupil image, and determining the pupil offset corresponding to each frame of pupil center according to a preset pupil image template; identifying whether the fundus structure map meets the preset acquisition conditions; if so, when the pupil offset exceeds the preset offset threshold, adjusting the scanning parameters of the OCT camera, otherwise, performing image compensation on the fundus structure map corresponding to the pupil image according to the pupil offset.

[0005] In combination with the first aspect, an embodiment of the present invention provides a first implementation of the first aspect, wherein the step of adjusting the scanning parameters of the OCT camera includes: converting the pupil offset into a galvanometer offset voltage of the OCT camera; and adjusting the galvanometer position of the OCT camera based on the galvanometer offset voltage.

[0006] In combination with the first aspect, an embodiment of the present invention provides a second implementation of the first aspect, wherein the step of converting the pupil offset into the galvanometer offset voltage of the OCT camera includes: calculating the pixel ratio of the pupil image and the fundus structure image; calculating the pixel offset corresponding to the pixel ratio based on the offset; converting the pixel offset into an actual physical distance, calculating the adjustment amount of the voltage range of the galvanometer of the OCT camera based on the actual physical distance, and determining the galvanometer offset voltage of the OCT camera.

[0007] In combination with the first aspect, an embodiment of the present invention provides a third implementation of the first aspect, wherein the step of performing image compensation on the fundus structure map corresponding to the pupil image according to the pupil offset includes: recording the pupil offset, and performing translation compensation on the fundus structure map corresponding to the pupil image according to the pupil offset.

[0008] In combination with the first aspect, an embodiment of the present invention provides a fourth implementation of the first aspect, wherein the above method also includes: obtaining a pupil image of a pupil camera, and a fundus structure map corresponding to the pupil image; reconstructing the fundus structure map; determining whether the fundus structure map meets a preset scanning requirement; wherein the scanning requirement is determined based on the pupil integrity of the target eyeball or the signal strength of the fundus structure map; if so, determining the pupil image as a pupil image template.

[0009] In combination with the first aspect, an embodiment of the present invention provides a fifth implementation of the first aspect, wherein the step of determining the pupil center of each frame of pupil image includes: performing image preprocessing on the pupil image; extracting the pupil edge from the pupil image; performing image fitting on the pupil image based on the pupil edge to determine the pupil center of the pupil image.

[0010] In combination with the first aspect, an embodiment of the present invention provides a sixth implementation of the first aspect, wherein the step of identifying whether the fundus structure image meets the preset acquisition conditions includes: determining the signal strength deviation between the fundus structure images corresponding to each adjacent frame of pupil images; judging whether the signal strength deviation is less than a preset strength deviation threshold; if so, determining that the fundus structure image meets the preset acquisition conditions.

[0011] In combination with the first aspect, an embodiment of the present invention provides a seventh implementation of the first aspect, wherein the steps of controlling a preset pupil camera to capture images of a target eyeball according to a preset acquisition frequency, and synchronously controlling a preset OCT camera to scan the target eyeball based on the acquisition frequency of the pupil camera include: obtaining a preset scan start signal; triggering the pupil camera to start based on the scan start signal, and triggering the OCT camera to start.

[0012] In a second aspect, an embodiment of the present invention provides an anterior segment eye tracking device, wherein the device includes: a data acquisition module, used to control a preset pupil camera to acquire images of a target eyeball according to a preset acquisition frequency, so as to obtain each frame of pupil image of the target eyeball; and, based on the acquisition frequency of the pupil camera, synchronously controlling a preset OCT camera to scan the target eyeball, so as to obtain a fundus structure map corresponding to each frame of pupil image; a data processing module, used to determine the pupil center of each frame of pupil image, and determine the pupil offset corresponding to each frame of pupil center according to a preset pupil image template; an identification module, used to identify whether the fundus structure map meets the preset acquisition conditions; an execution module, used to adjust the scanning parameters of the OCT camera when the fundus structure map meets the preset acquisition conditions and the pupil offset exceeds the preset offset threshold; otherwise, image compensation is performed on the fundus structure map corresponding to the pupil image according to the pupil offset.

[0013] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the eye tracking method in the previous section of any of the above embodiments are implemented.

[0014] The embodiments of the present invention bring the following beneficial effects: the embodiments of the present invention provide a method, device and electronic device for tracking anterior eye movement, synchronously collect pupil images and OCT structure images, and propose a comprehensive eye tracking strategy that takes into account pupil center offset and OCT structure images, which can ensure that during the optical coherence tomography (OCT) imaging process, even if eye movement or blinking occurs, high-quality image acquisition can be maintained, and the uniformity of the front view cross-sectional image can be improved. Among them, the embodiments of the present invention combine the pupil image with the structure image for analysis, determine the degree of pupil center offset, and implement corresponding compensation measures only when the threshold is reached, which reduces the system's requirements for extremely high real-time performance, so that a wider range of hardware platforms can support efficient eye movement compensation. Moreover, the embodiments of the present invention also dynamically monitor the image during the OCT acquisition process, can synchronously evaluate eye movement or eye closure, and immediately execute the corresponding processing strategy. This not only ensures that the data collected each time meets the preset quality standards, but also improves the consistency and reliability of the entire imaging process. The compensation mechanism provided by the present invention can flexibly respond to different types of involuntary eye movements (such as micro-saccades, drifts, or tremors) and blinking events, ensuring that clear and accurate OCT images can be obtained regardless of how the patient's condition changes.

[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of a method for tracking eye movements in the first half of the first half provided by an embodiment of the present invention;

[0019] Figure 2 A flowchart of another method for tracking eye movements in the front section provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a control signal provided by an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of a logical framework of eye tracking for the first half of the session provided by an embodiment of the present invention;

[0022] Figure 5 A logical schematic diagram of eye tracking for the front section provided by an embodiment of the present invention;

[0023] Figure 6 A schematic diagram of the structure of a front segment eye tracking device provided by an embodiment of the present invention;

[0024] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Since three-dimensional scanning requires a large scanning range and cannot be completed quickly, the patient may blink or move his eyes during this process. If data is collected during blinking, the OCT structure image will have no signal, which directly leads to the loss of part of the OCT data. Involuntary eye movement causes the collected OCT structure image to be misaligned, affecting the accuracy of subsequent quantitative data and the repeatability of the scan. In the prior art, the speed of OCT scanning is increased to reduce the impact of eye movement and blinking, but eye tracking technology is still required, and the complexity and cost of the equipment are increased; or, an eye tracking device is used to monitor eye movement in real time, and the scanning parameters of the OCT system are adjusted according to the monitored data to achieve image compensation, but this method has very high requirements on the real-time performance of the equipment and cannot take into account blinking during eye movement and the quality of the OCT structure image.

[0028] To solve the above problems, embodiments of the present invention provide a front segment eye tracking method, device and electronic device, which can ensure high-quality image acquisition even if eye movement or blinking occurs during optical coherence tomography (OCT) imaging.

[0029] For ease of understanding, a front-segment eye movement tracking method provided by an embodiment of the present invention is first described. Figure 1 FIG. 4 shows a flow chart of a method for tracking eye movements in the first segment provided by an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0030] Step S102, controlling a preset pupil camera to capture images of a target eyeball at a preset acquisition frequency to obtain pupil images of each frame of the target eyeball; and synchronously controlling a preset OCT camera to scan the target eyeball based on the acquisition frequency of the pupil camera to obtain a fundus structure image corresponding to each frame of the pupil image.

[0031] Step S104, determining the pupil center of each frame of pupil image, and determining the pupil offset corresponding to each frame of pupil center according to a preset pupil image template.

[0032] Step S106, identifying whether the fundus structure image meets the preset acquisition conditions.

[0033] Step S108, if satisfied, when the pupil offset exceeds a preset offset threshold, the scanning parameters of the OCT camera are adjusted; otherwise, image compensation is performed on the fundus structure image corresponding to the pupil image according to the pupil offset.

[0034] Pupil images are mainly used to monitor the movement of the eyeball in the horizontal and vertical directions. The OCT structure map provides a three-dimensional view from the surface to the deep tissue, allowing the system to detect these subtle changes. For anterior segment OCTA imaging, multiple scans of the same position are required to extract blood flow signals. The pupil image is a plane image, while the OCT structure map is a depth-direction image. The two are sometimes not completely equivalent. The embodiment of the present invention evaluates and ensures the quality of each scan by analyzing the depth-direction image generated by OCT, and uses the OCT structure map to assist in judging the acquisition effect. It does not rely solely on the two-dimensional plane information provided by the pupil image, and can more comprehensively understand the actual movement of the eyeball and its impact on imaging.

[0035] In specific implementation, the embodiment of the present invention controls the acquisition actions of the pupil camera and the OCT camera to be strictly synchronized to provide accurate time-aligned data, and then combines the pupil center positioning algorithm with the judgment of the acquisition conditions of the OCT structure diagram to comprehensively evaluate the state of the current frame, so that a higher quality OCT structure diagram can be obtained. In optical coherence tomography (OCT) imaging, after the OCT camera collects data from the eyeball, it needs to go through a series of processing steps to reconstruct a detailed OCT structure diagram. In one embodiment, the OCT camera and the pupil camera are synchronously controlled to collect data from the eyeball, and the data collected by the OCT camera is reconstructed to obtain a fundus structure diagram. The quality of the reconstructed fundus structure diagram is judged. When the fundus structure diagram meets the acquisition conditions, a corresponding compensation strategy is implemented based on the pupil offset. Otherwise, the eyeball is re-collected and the fundus structure diagram is re-determined to ensure that the data collected each time meets the preset quality standards. Specifically, the embodiment of the present invention evaluates the degree of eye movement by calculating the change in the pupil center relative to the reference position of each frame, that is, the pupil offset. Moreover, in the eye tracking process, the embodiment of the present invention allows a certain threshold deviation for the pupil center offset. Only when the corresponding threshold is exceeded will the scanning parameters of the OCT camera be adjusted according to the preset compensation logic. Otherwise, the fundus structure image is compensated by post-processing, such as compensating the fundus structure image during the reconstruction process, thereby reducing the real-time requirements for the acquisition and analysis of the previous and next frames.

[0036] In summary, an embodiment of the present invention provides a method for tracking eye movements in the front segment, synchronously collects pupil images and OCT structure images, and proposes a comprehensive eye tracking strategy that takes into account the pupil center offset and the OCT structure image, which can ensure that during the optical coherence tomography (OCT) imaging process, even if eye movement or blinking occurs, high-quality image acquisition can be maintained. Among them, the embodiment of the present invention combines the pupil image with the structure image for analysis, determines the degree of pupil center offset, and only implements the galvanometer movement and re-collection when it is greater than the threshold, which reduces the system's requirements for extremely high real-time performance, allowing a wider range of hardware platforms to support efficient eye movement compensation. Moreover, the embodiment of the present invention also dynamically monitors the image during the OCT acquisition process, synchronously evaluates the eye movement or eye closure, and immediately executes the corresponding processing strategy. This not only ensures that the data collected each time meets the preset quality standards, but also improves the consistency and reliability of the entire imaging process. For different types of involuntary eye movements (such as micro-saccades, drifts or tremors) and blinking events, the compensation mechanism provided by the present invention can be flexibly responded to ensure that clear and accurate OCT images can be obtained regardless of how the patient's state changes.

[0037] Based on the above embodiment, the present invention provides another method for tracking eye movements in the front segment. Figure 2 FIG. 1 is a flowchart of another method for tracking eye movements in the front section provided by an embodiment of the present invention, referring to FIG. Figure 2 , the method comprises the following steps:

[0038] Step S202, controlling a preset pupil camera to capture images of the target eyeball at a preset acquisition frequency to obtain each frame of the pupil image of the target eyeball; and synchronously controlling a preset OCT camera to scan the target eyeball based on the acquisition frequency of the pupil camera to obtain a fundus structure image corresponding to each frame of the pupil image.

[0039] In a specific implementation, a preset scanning start signal is obtained, and based on the scanning start signal, the pupil camera is triggered to start, and the OCT camera is triggered to start. Specifically, Figure 3 The control signal schematic diagram of the pupil camera and OCT camera synchronous acquisition is shown. The trigger signal of the pupil camera, the trigger signal of the OCT camera and the vibration waveform of the galvanometer are all referenced by the same clock, and then the trigger signal of the pupil camera is synchronized with the signal trigger signal of the OCT camera. Figure 3 In the figure, from top to bottom are the scan start signal, pupil camera trigger signal, OCT frame trigger signal and galvanometer drive waveform. It can be seen from the figure that the pupil camera trigger signal and the OCT frame trigger signal synchronously trigger the acquisition. In addition, when the scan trigger is paused, the galvanometer compensation voltage can be added to the galvanometer drive waveform to achieve galvanometer compensation.

[0040] Step S204, determining the pupil center of each frame of pupil image, and determining the pupil offset corresponding to each frame of pupil center according to a preset pupil image template.

[0041] A high-quality pupil image can be selected as a template, and the image should have clear pupil boundaries and uniform lighting conditions. The position of the pupil center is marked in the template image as a reference point for subsequent matching. Furthermore, the pupil center of the pupil image and the pupil image template can be matched using a preset matching method to calculate the pupil offset. Among them, the offset can be calculated based on the coordinates of the pupil center in the template image and the coordinates of the pupil center in the current frame image. Specifically, the pupil image of the pupil camera and the fundus structure map corresponding to the pupil image can be obtained; the fundus structure map is reconstructed; it is determined whether the fundus structure map meets the preset scanning requirements; if so, the pupil image is determined as the pupil image template. Among them, the scanning requirements are determined based on the pupil integrity of the target eyeball or the signal strength of the fundus structure map, that is, whether the eyes are closed or the OCT quality is poor based on the OCT structure map. If the fundus structure map of the OCT camera does not have closed eyes or poor quality, the corresponding pupil image is used as the image template. In one implementation, the first pupil image whose pupil image and fundus structure image both meet the requirements may be used as the pupil image template, or the pupil image template may be gradually updated to ensure that the pupil center is not relatively offset.

[0042] The embodiment of the present invention extracts the pupil edge from the pupil image by performing image preprocessing on the pupil image. The pupil image is image fitted based on the pupil edge to determine the pupil center of the pupil image. In specific implementation, before extracting the pupil, image preprocessing helps to reduce noise and enhance the contrast of the pupil. Binarization is performed by selecting a suitable threshold algorithm, and then the pupil edge is extracted by traditional edge detection, Hough circle transform, deep learning and other methods, and finally the center of the pupil is obtained by fitting. Among them, the pupil image can be median filtered to reduce noise, and then binarized based on an adaptive threshold to convert the grayscale image into a binary image to highlight the pupil area. Further, morphological opening and closing operations are performed to remove small noise points and fill holes, and to optimize the connected areas in the binary image. The pupil area is obtained using the maximum connected domain, and the connected domain is marked on the binary image to find all connected areas. The area of ​​each connected area is calculated, and the area with the largest area is selected as the pupil area. Finally, the pupil contour is accurately fitted using the least squares ellipse to obtain the ellipse parameters.

[0043] Step S206, determining the signal strength deviation between fundus structure images corresponding to each adjacent frame of pupil image; judging whether the signal strength deviation is less than a preset strength deviation threshold; if so, determining that the fundus structure image meets the preset acquisition condition.

[0044] Since the three-dimensional scanning needs to scan a large range and cannot be completed quickly, the patient will have involuntary eye movements during this process, resulting in the dislocation of the collected OCT structure map or blinking. If data is collected during blinking, there will be no signal in the OCT structure map, which will directly lead to the loss of part of the OCT data, affecting the accuracy of the subsequent quantitative data and the repeatability of the scan, resulting in poor uniformity of the reconstructed front view cross-sectional map. Blinking can be judged by the pupil map, but it is difficult to determine a metric for judging blinking on the pupil map. How much the pupil is blocked will cause the intensity of the OCT structure map to be significantly weakened, which may lead to the omission of blinking judgment. Sometimes the OCT structure map signal will be weakened after the eye movement or blinking is restored, which will cause the three-dimensional scanning en-face map (front view cross-sectional map) to be uneven. The embodiment of the present invention directly judges blinking by the obvious change in the intensity of the OCT structure map, and this method has a higher accuracy rate. The embodiment of the present invention adds the judgment of OCT intensity change during the eye tracking process, reconstructs the OCT structure diagram while calculating the pupil center offset, judges whether there is blinking based on the pupil diagram and the intensity of the OCT signal, and filters the acquisition frames with obvious intensity reduction. The selection is made through the threshold of the intensity change of adjacent frames. If the change is greater than the threshold, data loss in the key area may occur. Correspondingly, the data at that position is rescanned and the blinking data is discarded until the blinking ends. If it is less than the threshold, the following scan is performed, which can improve the uniformity of the front view cross-sectional image. Based on this, in addition to calculating the pupil center offset in the analysis, the embodiment of the present invention also adds the judgment of blinking action, so that the data can be processed in time during the acquisition process.

[0045] Step S208, when the pupil offset exceeds a preset offset threshold, the pupil offset is converted into a galvanometer offset voltage of the OCT camera; and based on the galvanometer offset voltage, the galvanometer position of the OCT camera is adjusted.

[0046] Step S210: otherwise, record the pupil offset, and perform translation compensation on the fundus structure image corresponding to the pupil image according to the pupil offset.

[0047] The enhancement of OCT structure images is generally achieved by superimposing images collected at the same position multiple times. In addition, the reconstruction of OCTA is also achieved by collecting OCT images at the same position multiple times, and then decorrelating and extracting blood flow signals. Both methods have requirements for the offset of several OCT structure images at the same position, and the position deviation of the frames at the same position cannot be too large. The embodiment of the present invention sets an offset threshold for several OCT structure images at the same position to filter, filter and re-collect frames with large offsets in the X and Y directions to improve the success rate of later image registration. In the embodiment of the present invention, a threshold for allowable deviation is set. Within this threshold, the galvanometer does not need to be compensated, and corresponding translation compensation can be performed after the later OCT image reconstruction. Outside the threshold, the embodiment of the present invention compensates by moving the galvanometer. If the re-collected offset is less than the offset threshold, compensation is performed by post-processing methods. In summary, the embodiment of the present invention can reduce the real-time requirements of the galvanometer compensation, and can obtain high-quality OCT structure images and quantify them.

[0048] The reconstruction of the OCT structure image is completed by GPU. After the reconstruction, the intensity of the OCT structure image is obtained by calculating the average value of the entire image. Combining the above steps, by setting the threshold of the intensity difference between the previous and next frames, it can be determined whether the eye is blinking, thereby determining whether to rescan.

[0049] The pupil camera and OCT camera synchronously collect and transmit data to the computer. The computer calculates the offset of the pupil center and reconstructs the OCT structure diagram. It also determines whether there is blinking and the intensity change of the OCT structure diagram based on the eye tracking strategy. It also determines whether the OCT galvanometer should be compensated. If compensation is required, the OCT offset pixel is sent to the OCT galvanometer control lower computer, which converts the pixel into a bias voltage to compensate the galvanometer drive waveform. Correspondingly, Figure 4 The logical framework of eye tracking in the previous section is shown. Figure 5 A schematic diagram of the eye tracking logic in the previous section is shown.

[0050] Specifically, the pixel ratio of the pupil image and the fundus structure image is calculated; based on the offset, the pixel offset corresponding to the pixel ratio is calculated; the pixel offset is converted into the actual physical distance, and the adjustment amount of the voltage range of the galvanometer of the OCT camera is calculated based on the actual physical distance to determine the galvanometer offset voltage of the OCT camera. In the specific implementation, the anterior segment OCT uses the pupil image to locate the position of the anterior segment OCT scan, and uses a square target with a scale to align the pupil image and the OCT image to adjust the pupil scanning area. Assuming that the scanning area is a rectangle of L*Lmm, according to the resolution P of the pupil camera X *P Y The horizontal and vertical pixel sizes of the pupil camera can be obtained as L / P X and L / P YThe scanning range of the OCT structure diagram is determined by the rotation angle of the galvanometer, and the rotation angle of the galvanometer is determined by the scanning voltage range of the galvanometer. X and Y galvanometer scanning voltage range V Y To obtain an L*Lmm long OCT structure image. The resolution of the known area scan OCT structure image is K X *K Y , then the horizontal and vertical pixel sizes of the OCT structure image are L / K X and L / K Y The voltage corresponding to each pixel is V X / K X and V Y / K Y Finally, the horizontal and vertical pixel ratios of pupil and OCT images are K. X / P X and K Y / P Y .

[0051] During eye tracking, pupil images are continuously acquired and the offset Z of the pupil center between two frames is calculated. X and Z Y , and then converted into the pixel offset K of the OCT structure image X *Z X / P X and K Y *Z Y / P Y , according to this offset, the offset voltage of the OCT galvanometer is converted into Z X *V X / P X and Z Y *V Y / P Y To compensate for the OCT scanning position and achieve the purpose of tracking eye movements.

[0052] In summary, another anterior segment eye tracking method provided by an embodiment of the present invention avoids invalid data collection caused by eye movement or blinking by intelligently screening and post-processing correction of the collected data, reduces the need for rescanning, thereby saving medical resources and improving diagnosis and treatment efficiency. It can also reduce scanning failures or image quality problems caused by eye movement or blinking, thereby reducing the discomfort and anxiety of patients, while also simplifying the doctor's operating procedures and improving diagnostic accuracy. In addition, an embodiment of the present invention uses a pupil camera and an OCT camera to synchronously collect eyeballs, and proposes a new eye tracking strategy. By determining the pupil center of the pupil map of the current frame and the template pupil map and calculating the offset of the pupil center, the offset allows a preset deviation threshold with the OCT galvanometer position. When the deviation is less than the threshold, the deviation can be offset compensated after the OCT image reconstruction, and when it is greater than the threshold, the galvanometer is offset until both are within the deviation threshold, and then compensated by post-processing. Due to the introduction of the deviation threshold, the requirements for the real-time compensation of the galvanometer can be reduced, and the robustness of eye tracking can be improved.

[0053] Furthermore, an embodiment of the present invention also provides a front segment eye tracking device, Figure 6 FIG. 1 shows a schematic diagram of the structure of a front segment eye tracking device provided by an embodiment of the present invention, referring to FIG. Figure 6 The device includes: a data acquisition module 100, which is used to control a preset pupil camera to acquire images of a target eyeball according to a preset acquisition frequency to obtain each frame of pupil images of the target eyeball; and, based on the acquisition frequency of the pupil camera, synchronously control a preset OCT camera to scan the target eyeball to obtain a fundus structure image corresponding to each frame of pupil images; a data processing module 200, which is used to determine the pupil center of each frame of pupil images, and determine the pupil offset corresponding to each frame of pupil center according to a preset pupil image template; an identification module 300, which is used to identify whether the fundus structure image meets the preset acquisition conditions; an execution module 400, which is used to adjust the scanning parameters of the OCT camera when the fundus structure image meets the preset acquisition conditions and the pupil offset exceeds the preset offset threshold, otherwise, perform image compensation on the fundus structure image corresponding to the pupil image according to the pupil offset.

[0054] The anterior segment eye tracking device provided in the embodiment of the present invention has the same technical features as the anterior segment eye tracking method provided in the above embodiment, and therefore can also solve the same technical problems and achieve the same technical effects.

[0055] Further, based on the above embodiment, the above offset threshold includes an offset threshold; the above execution module 400 is also used to convert the pupil offset into a galvanometer offset voltage of the OCT camera; and adjust the galvanometer position of the OCT camera based on the galvanometer offset voltage. The above execution module 400 is also used to calculate the pixel ratio of the pupil image and the fundus structure image; calculate the pixel offset corresponding to the pixel ratio according to the pupil offset; convert the pixel offset into an actual physical distance, calculate the adjustment amount of the voltage range of the galvanometer of the OCT camera based on the actual physical distance, and determine the galvanometer offset voltage of the OCT camera.

[0056] The execution module 400 is further used to record the pupil offset, and perform translation compensation on the fundus structure image corresponding to the pupil image according to the pupil offset.

[0057] The above-mentioned data processing module 200 is also used to obtain the pupil image of the pupil camera and the fundus structure map corresponding to the pupil image; reconstruct the fundus structure map; determine whether the fundus structure map meets the preset scanning requirements; wherein the scanning requirements are determined based on the pupil integrity of the target eyeball or the signal strength of the fundus structure map; if so, determine the pupil image as the pupil image template.

[0058] The above-mentioned data processing module 200 is also used to perform image preprocessing on the pupil image; extract the pupil edge from the pupil image; perform image fitting on the pupil image based on the pupil edge to determine the pupil center of the pupil image.

[0059] The above-mentioned identification module 300 is also used to determine the signal strength deviation between the fundus structure images corresponding to each adjacent frame of pupil image; judge whether the signal strength deviation is less than a preset strength deviation threshold; if so, determine that the fundus structure image meets the preset acquisition conditions.

[0060] The data acquisition module 100 is also used to obtain a preset scan start signal; based on the scan start signal, trigger the pupil camera to start, and trigger the OCT camera to start.

[0061] The embodiment of the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned multi-objective comprehensive optimization decision method. Figure 7 As shown, the electronic device includes a processor 70 and a memory 71, wherein the memory 71 stores machine executable instructions that can be executed by the processor 70, and the processor 70 executes the machine executable instructions to implement the above-mentioned multi-objective comprehensive optimization decision-making method.

[0062] Further, Figure 7The electronic device shown also includes a bus 72 and a communication interface 73, and the processor 70, the communication interface 73 and the memory 71 are connected through the bus 72. Among them, the memory 71 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 73 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 72 can be an ISA (Industrial Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Enhanced Industry Standard Architecture) bus, etc. The above-mentioned bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0063] The processor 70 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 70. The above processor 70 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 71, and the processor 70 reads the information in the memory 71 and completes the steps of the method of the above embodiment in combination with its hardware. This embodiment also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions prompt the processor to implement the above multi-objective comprehensive optimization decision-making method.

[0064] A computer program product of a method, device and electronic device for front-segment eye tracking provided by an embodiment of the present invention includes a computer-readable storage medium storing a program code, and the instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can refer to the method embodiment, which will not be repeated here. A person skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the above method embodiment, which will not be repeated here. In addition, in the description of the embodiment of the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For a person of ordinary skill in the art, the specific meaning of the above terms in the present invention can be understood according to the specific situation. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes. In the description of the present invention, it should be noted that the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0065] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for tracking eye movements in the front segment, characterized in that: The method comprises: Control a preset pupil camera to collect images of a target eyeball according to a preset collection frequency to obtain each frame of pupil image of the target eyeball; and synchronously control a preset OCT camera to scan the target eyeball based on the collection frequency of the pupil camera to obtain a fundus structure image corresponding to each frame of the pupil image; Determine the pupil center of each frame of the pupil image, and determine the pupil offset corresponding to the pupil center of each frame according to a preset pupil image template; Identify whether the fundus structure image meets the preset acquisition conditions; If satisfied, when the pupil offset exceeds a preset offset threshold, the scanning parameters of the OCT camera are adjusted; otherwise, image compensation is performed on the fundus structure image corresponding to the pupil image according to the pupil offset.

2. The method according to claim 1, characterized in that The step of adjusting the scanning parameters of the OCT camera comprises: Converting the pupil offset into a galvanometer offset voltage of the OCT camera; The galvanometer position of the OCT camera is adjusted based on the galvanometer offset voltage.

3. The method according to claim 2, characterized in that The step of converting the pupil offset into a galvanometer offset voltage of the OCT camera comprises: Calculating the pixel ratio of the pupil image and the fundus structure image; Calculating a pixel offset corresponding to the pixel ratio according to the pupil offset; The pixel offset is converted into an actual physical distance, and an adjustment amount of a voltage range of a galvanometer of the OCT camera is calculated based on the actual physical distance to determine an offset voltage of the galvanometer of the OCT camera.

4. The method according to claim 1, characterized in that: The step of performing image compensation on the fundus structure image corresponding to the pupil image according to the pupil offset comprises: The pupil offset is recorded, and translation compensation is performed on the fundus structure image corresponding to the pupil image according to the pupil offset.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a pupil image of the pupil camera and a fundus structure image corresponding to the pupil image; Reconstructing the fundus structure image; Determining whether the fundus structure image meets a preset scanning requirement; wherein the scanning requirement is determined based on the pupil integrity of the target eyeball or the signal strength of the fundus structure image; If yes, the pupil image is determined as a pupil image template.

6. The method according to claim 1, characterized in that The step of determining the pupil center of each frame of the pupil image comprises: Performing image preprocessing on the pupil image; extracting a pupil edge from the pupil image; Image fitting is performed on the pupil image based on the pupil edge to determine the pupil center of the pupil image.

7. The method according to claim 1, characterized in that The step of identifying whether the fundus structure image meets the preset acquisition conditions includes: Determine the signal intensity deviation between fundus structure images corresponding to each adjacent frame of the pupil image; Determining whether the signal strength deviation is less than a preset strength deviation threshold; If yes, it is determined that the fundus structure image meets the preset acquisition conditions.

8. The method according to claim 1, characterized in that The steps of controlling a preset pupil camera to collect images of a target eyeball according to a preset collection frequency, and synchronously controlling a preset OCT camera to scan the target eyeball based on the collection frequency of the pupil camera include: Obtain a preset scan start signal; Based on the scan start signal, the pupil camera is triggered to start, and the OCT camera is triggered to start.

9. A front segment eye tracking device, characterized in that: The device comprises: A data acquisition module, used to control a preset pupil camera to acquire images of a target eyeball according to a preset acquisition frequency, so as to obtain each frame of pupil images of the target eyeball; and, based on the acquisition frequency of the pupil camera, synchronously control a preset OCT camera to scan the target eyeball, so as to obtain a fundus structure image corresponding to each frame of the pupil image; A data processing module, used to determine the pupil center of each frame of the pupil image, and determine the pupil offset corresponding to the pupil center of each frame according to a preset pupil image template; An identification module, used to identify whether the fundus structure image meets a preset acquisition condition; The execution module is used to adjust the scanning parameters of the OCT camera when the fundus structure image meets the preset acquisition conditions and the pupil offset exceeds the preset offset threshold; otherwise, perform image compensation on the fundus structure image corresponding to the pupil image according to the pupil offset.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the front segment eye tracking method described in any one of claims 1-8 are implemented.

Images