Method and device for correcting image collected by pupil camera, equipment and storage medium

Abstract

The invention relates to a correction method and device for an image collected by a pupil camera, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: collecting eye data of an examined eye by using a pupil camera and OCT equipment, wherein the eye data comprises a first image collected by the pupil camera and a second image collected by the OCT equipment; determining at least one first target point based on the first image, and determining at least one second target point corresponding to the at least one first target point from the second image; and determining a mapping function based on the at least one first target point and the at least one second target point, wherein the mapping function is used for correcting the eye image acquired by the pupil camera. By adopting the method, the accuracy of the eye image acquired by the pupil camera can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pupil camera, and in particular to a pupil camera image correction method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] Pupil cameras are often used for alignment, eye fixation detection and preoperative parameter measurement, but their measurement accuracy is limited due to their own geometric distortion, non-linear characteristics of pixel-physical coordinates, and corneal and lens refraction effects, which in turn affects the reliability of subsequent data.

[0003] In the prior art, the offline checkerboard method is mostly used to calibrate the images collected by the pupil camera.

[0004] However, this method cannot deal with dynamic disturbances such as device drift and eye micro-movement, and has the problem of insufficient correction accuracy. SUMMARY

[0005] Therefore, it is necessary to provide a pupil camera image correction method, device, computer equipment, computer readable storage medium and computer program product with high accuracy to solve the above technical problems.

[0006] In a first aspect, the present application provides a pupil camera image correction method, comprising:

[0007] Collecting eye data of the eye to be examined by using a pupil camera and an OCT device, the eye data comprising a first image collected by the pupil camera and a second image collected by the OCT device;

[0008] Determining at least one first target point based on the first image, and determining at least one second target point corresponding to the at least one first target point from the second image;

[0009] Determining a mapping function based on the at least one first target point and the at least one second target point, the mapping function being used to correct the eye image collected by the pupil camera.

[0010] In one embodiment, collecting eye data of the eye to be examined by using a pupil camera and an OCT device comprises: collecting initial eye data of the eye to be examined by using a pupil camera and an OCT device, the initial eye data comprising an initial first image collected by the pupil camera and an initial second image collected by the OCT device; determining quality information of the initial eye data, and determining the initial eye data as the eye data if the quality information of the initial eye data meets a preset condition.

[0011] In one of the embodiments, determining the quality information of the initial eye data includes: obtaining a signal amplitude and a noise amplitude of the initial eye data, and determining a signal-to-noise ratio of the initial eye data based on the signal amplitude and the noise amplitude; obtaining an edge integrity and a contrast of the pupil in the initial eye data, and determining a pupil detection confidence of the initial eye data based on the edge integrity and the contrast; obtaining displacement information of the pupil in the initial eye data, and determining an instantaneous motion of the initial eye data based on the displacement information; and determining the quality information of the initial eye data based on the signal-to-noise ratio, the pupil detection confidence, and the instantaneous motion.

[0012] In one of the embodiments, determining the mapping function based on the at least one first target point and the at least one second target point includes: obtaining pixel coordinates of the at least one first target point, and obtaining physical coordinates of the at least one second target point; for any first target point in the at least one first target point, constructing a target point pair based on the pixel coordinates of the first target point and the physical coordinates of the second target point corresponding to the first target point to obtain at least one target point pair; and determining the mapping function based on the at least one target point pair.

[0013] In one of the embodiments, determining the mapping function based on the at least one target point pair includes: fitting the at least one target point pair based on a preset fitting algorithm to obtain an initial mapping function; and optimizing the initial mapping function based on a preset interpolation algorithm to obtain the mapping function.

[0014] In one of the embodiments, determining the mapping function based on the at least one target point pair includes: fitting the at least one target point pair based on a physical model to obtain an initial mapping function; the physical model is constructed based on a target geometric mapping relationship and / or a target optical path transmission characteristic, the target geometric mapping relationship is a geometric mapping relationship between the pupil camera and the OCT device, and the target optical path transmission characteristic includes an optical path transmission characteristic of the pupil camera and an optical path transmission characteristic of the OCT device; and optimizing the initial mapping function based on a pre-trained network model to obtain the mapping function.

[0015] In one of the embodiments, the method further includes: obtaining an initial eye image of the eye under examination collected by the pupil camera; the initial eye image includes a corneal reflection point; performing correction processing on the initial eye image based on the mapping function to obtain a target eye image, and obtaining real-time position information of the corneal reflection point in the target eye image; comparing the real-time position information with standard position information of the corneal reflection point, and determining an eye movement condition of the eye under examination according to a comparison result; and the standard position information is position information of the corneal reflection point when the eye under examination does not move.

[0016] In a second aspect, the application further provides a correction device for an image collected by a pupil camera, including:

[0017] The collection module is configured to collect eye data of the eye to be detected by using the pupil camera and the OCT device, and the eye data comprises a first image collected by the pupil camera and a second image collected by the OCT device.

[0018] The determination module is configured to determine at least one first target point based on the first image, and determine at least one second target point corresponding to the at least one first target point from the second image.

[0019] The execution module is configured to determine a mapping function based on the at least one first target point and the at least one second target point, and the mapping function is used for correcting the eye image collected by the pupil camera.

[0020] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method in any of the embodiments of the first aspect when executing the computer program.

[0021] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the method in any of the embodiments of the first aspect when executed by a processor.

[0022] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, and the computer program implements the steps of the method in any of the embodiments of the first aspect when executed by a processor.

[0023] The image correction method, device, computer device, computer readable storage medium and computer program product provided in the present application, first, the eye data of the eye to be detected is collected by using the pupil camera and the OCT device, and the eye data comprises a first image collected by the pupil camera and a second image collected by the OCT device; then, at least one first target point is determined based on the first image, and at least one second target point corresponding to the at least one first target point is determined from the second image; and finally, a mapping function is determined based on the at least one first target point and the at least one second target point, and the mapping function is used for correcting the eye image collected by the pupil camera. The image correction method provided in the present application constructs the mapping function by using the eye data collected by the OCT device and the pupil camera. Since the eye data collected by the OCT device is used as a reference benchmark, the geometric distortion of the pupil camera, the non-linear characteristics of the pixel to the physical coordinates, and the refractive error of the cornea and the lens can be effectively avoided, and then the eye image collected by the pupil camera is corrected based on the constructed mapping function, so that the accuracy of the eye image collected by the pupil camera can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0025] Figure 1 A flowchart of a method for correcting images collected by a pupil camera in an embodiment;

[0026] Figure 2 A flowchart of a method for collecting eye data of an eye under examination by using a pupil camera and an OCT device in an embodiment;

[0027] Figure 3 A flowchart of a method for determining quality information of initial eye data in an embodiment;

[0028] Figure 4 A flowchart of a method for determining a mapping function in an embodiment;

[0029] Figure 5 A flowchart of a method for determining a mapping function in another embodiment;

[0030] Figure 6 A flowchart of a method for determining a mapping function in another embodiment;

[0031] Figure 7 A flowchart of a method after determining a mapping function in an embodiment;

[0032] Figure 8 A flowchart of a method for correcting images collected by a pupil camera in another embodiment;

[0033] Figure 9 A block diagram of a correcting device for images collected by a pupil camera in an embodiment;

[0034] Figure 10 An internal structure diagram of a computer device in an embodiment;

[0035] Figure 11 An internal structure diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0036] In order to make the purposes, technical solutions and advantages of the present application clearer, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0037] It should be noted that the terms "first", "second", etc. used in the present application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "a plurality of" used in the present application refers to two or more. The term "and / or" used in the present application refers to one of the options or any combination of the options.

[0038] Pupil cameras are often used for alignment, eye fixation detection, and preoperative parameter measurement, but their measurement accuracy is limited due to their own geometric distortion, pixel-physical coordinate nonlinear characteristics, and corneal and lens refraction effects, which in turn affects the reliability of subsequent data.

[0039] In the prior art, offline checkerboard methods are mostly used to calibrate images collected by pupil cameras. However, this approach cannot deal with dynamic disturbances such as device drift and eye micro-movement, and there is a problem of insufficient correction accuracy.

[0040] Swept-source optical coherence tomography (SS-OCT, Swept-Source Optical Coherence Tomography) as a frequency domain OCT technology, with a swept laser as the core light source, has the advantages of fast scanning speed and deep imaging depth, and has been widely used in medical imaging fields such as ophthalmology.

[0041] Currently, although there are solutions to integrate SS-OCT devices with pupil cameras, such solutions are only used to implement focusing functions and do not involve pupil camera calibration and distortion correction.

[0042] Therefore, the present application provides a pupil camera image correction method. First, eye data of a subject eye is collected by a pupil camera and an OCT device. The eye data includes a first image collected by the pupil camera and a second image collected by the OCT device. Then, at least one first target point is determined based on the first image, and at least one second target point corresponding to the at least one first target point is determined from the second image. Next, a mapping function is determined based on the at least one first target point and the at least one second target point. The mapping function is used to correct the eye image collected by the pupil camera. The pupil camera image correction method provided by the present application uses eye data collected by the OCT device and the pupil camera to construct a mapping function. Since the eye data collected by the OCT device is used as a reference, the geometric distortion of the pupil camera, the pixel-to-physical coordinate nonlinear characteristics, and the corneal and lens refraction errors can be effectively avoided. Furthermore, the eye image collected by the pupil camera can be corrected based on the constructed mapping function, which can effectively improve the accuracy of the eye image collected by the pupil camera.

[0043] The correction method of the image collected by the pupil camera provided in the application can be executed by a computer device, which can be a terminal or a server.

[0044] In an exemplary embodiment, as shown in Figure 1 A correction method of an image collected by a pupil camera is provided, which comprises the following steps:

[0045] Step 101: collecting eye data of an eye to be examined by using a pupil camera and an OCT device.

[0046] The pupil camera refers to a special imaging device for ophthalmic examination and surgery, which can capture images of the pupil area of a human eye. Optionally, the pupil camera can be a near-infrared camera, a visible light camera, etc. The lens of the pupil camera adopts a low-distortion design or a telecentric design, and is matched with an adjustable focusing structure.

[0047] The OCT device can be an SS-OCT device. The SS-OCT device uses a frequency-sweeping laser as a light source, and performs high-resolution and non-invasive tomographic imaging on eye tissues by using the principle of optical interference, so as to obtain real three-dimensional anatomical physical coordinates of corneal and retinal tissues.

[0048] Optionally, the eye data comprises a first image collected by the pupil camera and a second image collected by the OCT device.

[0049] In some exemplary embodiments, the computer device can collect the eye data of the eye to be examined by using the pupil camera and the OCT device.

[0050] Specifically, the computer device can send a trigger signal to the pupil camera and the OCT device, so as to obtain the eye data of the eye to be examined collected by the pupil camera and the OCT device synchronously, i.e., the time sequence of the first image collected by the pupil camera and the second image collected by the OCT device is synchronized.

[0051] Further, if the sampling rates of the pupil camera and the OCT device are inconsistent, the computer device can perform time sequence matching on the first image collected by the pupil camera and the second image collected by the OCT device by using a nearest neighbor method or an interpolation method on a time axis, so as to ensure the time sequence synchronization of the first image collected by the pupil camera and the second image collected by the OCT device.

[0052] Step 102: determining at least one first target point based on the first image, and determining at least one second target point corresponding to the at least one first target point from the second image.

[0053] Optionally, the first target point can be a corneal reflection point. Illustratively, the corneal reflection point refers to a high-contrast reflection mark point formed on the surface of the cornea by projecting structured light onto the cornea of the human eye. In other optional embodiments, the first target point can also include, but is not limited to, a corneal vertex, a pupil center, other stable feature points extracted based on eye anatomy, etc. Among them, different types of first target points can be selected or combined for use according to the imaging mode, system accuracy requirements and real-time requirements.

[0054] In some illustrative embodiments, after the computer device collects the eye data of the eye under test using the pupil camera and the OCT device, the computer device can determine at least one first target point based on the first image.

[0055] Specifically, the computer device can determine a high-contrast reflection region from the first image based on a threshold segmentation algorithm, and screen the first target point meeting the preset morphological features from the reflection region, wherein the preset morphological features can be set by the technician in advance, such as a circular or circular-like contour, a preset pixel area range, a uniform brightness distribution and a clear edge feature.

[0056] The computer device can also determine a high-contrast reflection region from the first image based on a contour detection algorithm, and extract edge contour information of the reflection region, and determine the first target point by matching the preset contour features, wherein the preset contour features can be set by the technician in advance, such as a circularity threshold of the contour, an edge smoothness range, and a pixel interval of the contour diameter.

[0057] Further, after the computer device determines at least one first target point based on the first image, the computer device can determine at least one second target point corresponding to the at least one first target point from the second image.

[0058] Specifically, after the computer device determines at least one first target point based on the first image, the computer device can obtain the pixel coordinates of the at least one first target point, and determine at least one second target point corresponding to the at least one first target point from the second image based on the pixel coordinates of the at least one first target point.

[0059] In optional embodiments of the present application, after the computer device determines at least one first target point and at least one second target point corresponding to the at least one first target point, the computer device can remove the false target points therefrom based on a random sample consensus (RANSAC) algorithm.

[0060] Step 103, determining a mapping function based on the at least one first target point and the at least one second target point.

[0061] The mapping function can be used to correct the eye image collected by the pupil camera. As an example, since the mapping function is determined based on the at least one first target point and the at least one second target point determined based on the three-dimensional image collected by the OCT device, a corresponding relationship between the two-dimensional pixel space of the pupil camera and the real three-dimensional physical space of the eye tissue can be established within the effective range of the mapping function. By inputting the coordinates of a pixel point in the eye image collected by the pupil camera into the mapping function, the corresponding position of the pixel point in the three-dimensional physical space can be calculated, thereby compensating for the geometric distortion of the pupil camera, the nonlinearity of the pixel-physical coordinates, and the errors caused by the cornea and lens refraction, and realizing the correction of the image collected by the pupil camera.

[0062] In some example embodiments, after determining the at least one first target point based on the first image and determining the at least one second target point corresponding to the at least one first target point from the second image, the computer device can determine the mapping function based on the at least one first target point and the at least one second target point.

[0063] Specifically, the computer device can input the at least one first target point and the at least one second target point into the pre-trained fitting model, so that the fitting model performs fitting processing on the at least one first target point and the at least one second target point to obtain the mapping function output by the fitting model.

[0064] The above-mentioned correction method of the image collected by the pupil camera first collects eye data of the examined eye by using the pupil camera and the OCT device, the eye data including the first image collected by the pupil camera and the second image collected by the OCT device, then determines the at least one first target point based on the first image and determines the at least one second target point corresponding to the at least one first target point from the second image, and then determines the mapping function based on the at least one first target point and the at least one second target point, the mapping function being used to correct the eye image collected by the pupil camera. The correction method of the image collected by the pupil camera provided in the present application constructs the mapping function by using the eye data collected by the OCT device and the pupil camera. Since the eye data collected by the OCT device is used as a reference benchmark, the geometric distortion of the pupil camera, the nonlinearity of the pixel to physical coordinates, and the refraction errors of the cornea and lens can be effectively avoided, and then the eye image collected by the pupil camera is corrected based on the constructed mapping function, which can effectively improve the accuracy of the eye image collected by the pupil camera.

[0065] In one example embodiment, as shown in FIG. 1, the eye data of the examined eye is collected by using the pupil camera and the OCT device, including the following steps: Figure 2

[0066] ​Step 201, acquiring initial eye data of the eye under test by using the pupil camera and the OCT device.

[0067] Optionally, the initial eye data can include an initial first image acquired by the pupil camera and an initial second image acquired by the OCT device.

[0068] In some example embodiments, the computer device can acquire the initial eye data of the eye under test by using the pupil camera and the OCT device.

[0069] Specifically, the computer device can send a trigger signal to the pupil camera and the OCT device to acquire the initial eye data of the eye under test acquired synchronously by the pupil camera and the OCT device, i.e., the time sequence synchronization of the initial first image acquired by the pupil camera and the initial second image acquired by the OCT device.

[0070] Further, if the sampling rates of the pupil camera and the OCT device are inconsistent, the computer device can perform time sequence matching on the initial first image acquired by the pupil camera and the initial second image acquired by the OCT device by using a nearest neighbor method or an interpolation method on a time axis to ensure the time sequence synchronization of the initial first image acquired by the pupil camera and the initial second image acquired by the OCT device.

[0071] Step 202, determining quality information of the initial eye data, and determining the initial eye data as the eye data in a case where the quality information of the initial eye data meets a preset condition.

[0072] Optionally, the quality information of the initial eye data can be used to evaluate the reliability of the initial eye data.

[0073] The preset condition can be preset by a technician according to actual needs. For example, the preset condition can be that a quality score determined based on the quality information of the initial eye data is greater than a preset score threshold. The preset score thresholds corresponding to the initial first image acquired by the pupil camera and the initial second image acquired by the OCT device can be the same or different.

[0074] In some example embodiments, after acquiring the initial eye data of the eye under test by using the pupil camera and the OCT device, the computer device can determine the quality information of the initial eye data.

[0075] Specifically, the computer device can perform quality evaluation on the initial eye data based on a quality evaluation algorithm to determine the quality information of the initial eye data. The quality evaluation algorithm can be a natural scene statistics based no-reference image spatial domain quality evaluation algorithm, an enhanced no-reference image quality evaluation algorithm, a no-reference quality evaluation algorithm based on context modeling, a blind image quality index algorithm, etc.

[0076] Further, the computer device can determine the initial eye data as the eye data when the quality information of the initial eye data meets a preset condition after determining the quality information of the initial eye data.

[0077] Specifically, the computer device can determine whether the quality information of the initial eye data meets a preset condition after determining the quality information of the initial eye data, and if so, the initial eye data can be determined as the eye data.

[0078] In one example embodiment, as shown in Figure 3 the quality information of the initial eye data includes the following steps:

[0079] Step 301, obtaining the signal amplitude and noise amplitude of the initial eye data, and determining the signal-to-noise ratio of the initial eye data based on the signal amplitude and the noise amplitude.

[0080] Optionally, the signal amplitude refers to the intensity value of the effective target signal in the initial eye data, that is, the pixel gray scale amplitude of the light reflection point region in the initial first image and the reflection light signal amplitude corresponding to the target tissue such as cornea in the initial second image.

[0081] The noise amplitude refers to the intensity value of the invalid interference signal in the initial eye data, which includes device electronic noise, environmental light interference, and artifact signals generated by micro-movement of the eye to be detected.

[0082] In some example embodiments, the computer device can first obtain the signal amplitude and noise amplitude of the initial eye data, and then determine the signal-to-noise ratio of the initial eye data based on the signal amplitude and the noise amplitude.

[0083] Specifically, the computer device can first determine the original signal-to-noise ratio based on the signal amplitude and the noise amplitude, and the original signal-to-noise ratio can be represented as , wherein, is the signal amplitude of the initial eye data, is the noise amplitude of the initial eye data.

[0084] Further, the computer device obtains a preset lower limit threshold of signal-to-noise ratio and an upper limit threshold of signal-to-noise ratio, the lower limit threshold of signal-to-noise ratio can be represented as , and the upper limit threshold of signal-to-noise ratio can be represented as , may be 6dB, may be 30dB, and the signal-to-noise ratio of the initial eye data can be represented as , , is a truncation function for limiting the calculation result in the interval of 0~1. For example, The closer to 1, the better the quality of the initial eye data represents.

[0085] At step 302, the edge integrity and contrast of the pupil in the initial eye data are obtained, and a pupil detection confidence of the initial eye data is determined based on the edge integrity and the contrast.

[0086] Optionally, the edge integrity can be used to represent the continuity and integrity of the pupil profile in the initial eye data. The contrast refers to the gray scale contrast between the pupil region and the surrounding iris region.

[0087] In some example embodiments, the computer device can obtain the edge integrity and the contrast of the pupil in the initial eye data, and determine a pupil detection confidence of the initial eye data based on the edge integrity and the contrast.

[0088] Specifically, the pupil detection confidence can be represented as PupilConf, PupilConf = w1*EdgeFra + w2*FitScore + w3*Contrast, where w1, w2 and w3 are preset weights, optionally, w1 can be 0.4, w2 can be 0.4, and w3 can be 0.2, EdgeFra is the edge integrity, FitScore is the ellipse fitting residual normalization, and Contrast is the contrast.

[0089] At step 303, displacement information of the pupil in the initial eye data is obtained, and instantaneous motion of the initial eye data is determined based on the displacement information.

[0090] In some example embodiments, the computer device can first obtain the displacement information of the pupil in the initial eye data.

[0091] Specifically, the computer device can determine the displacement information of the pupil using two adjacent images in the initial eye data, where the displacement information of the pupil is the displacement of the pupil center between the two adjacent images.

[0092] Further, after obtaining the displacement information of the pupil in the initial eye data, the computer device can determine the instantaneous motion of the initial eye data based on the displacement information.

[0093] Specifically, the instantaneous motion can be represented as MotionNorm, wherein, is the displacement information, is a preset displacement reference, may be 1.0 mm, is a truncation function for limiting the calculation result in the range of 0-1.

[0094] For example, the closer MotionNorm is to 0, the smaller the instantaneous motion; the closer it is to 1, the larger the instantaneous motion.

[0095] Step 304: Determine the quality information of the initial eye data based on the signal-to-noise ratio, pupil detection confidence, and instantaneous motion.

[0096] In some exemplary embodiments, after obtaining the signal-to-noise ratio, pupil detection confidence level, and instantaneous motion, the computer device can determine the quality information of the initial eye data based on the signal-to-noise ratio, pupil detection confidence level, and instantaneous motion.

[0097] Specifically, computer equipment can perform a weighted summation of signal-to-noise ratio, pupil detection confidence, and instantaneous motion to determine the quality information of the initial eye data.

[0098] For example, quality information can be represented as Q. ,in, , and To preset weights, It can be 0.4. It can be 0.4. It can be 0.2.

[0099] In one exemplary embodiment, such as Figure 4 As shown, determining the mapping function based on at least one first target point and at least one second target point includes the following steps:

[0100] Step 401: Obtain the pixel coordinates of at least one first target point and the physical coordinates of at least one second target point.

[0101] Optionally, pixel coordinates refer to the position parameters of the first target point in the image pixel coordinate system, while physical coordinates refer to the position parameters of the second target point in the physical space coordinate system.

[0102] In some exemplary embodiments, after acquiring at least one first target point and at least one second target point, the computer device may acquire the pixel coordinates of at least one first target point and the physical coordinates of at least one second target point.

[0103] Specifically, the pixel coordinates of the first target point can be represented as: The physical coordinates of the second target point can be expressed as: .

[0104] Step 402: For any one of the first target points, construct a target point pair based on the pixel coordinates of the first target point and the physical coordinates of the second target point corresponding to the first target point, so as to obtain at least one target point pair.

[0105] In some example embodiments, after obtaining the pixel coordinates of the at least one first target point and the physical coordinates of the at least one second target point, the computer device can construct a target point pair for any of the at least one first target point based on the pixel coordinates of the first target point and the physical coordinates of the second target point corresponding to the first target point, to obtain at least one target point pair.

[0106] Specifically, assuming that there are three first target points, , and , there are also three second target points corresponding to the three first target points, , and , and there are three target point pairs corresponding to the three first target points, , , and .

[0107] Step 403: determining the mapping function based on the at least one target point pair.

[0108] In some example embodiments, after obtaining the at least one target point pair, the computer device can determine the mapping function based on the at least one target point pair.

[0109] Specifically, the computer device can fit the at least one target point pair based on a preset fitting algorithm to obtain an initial mapping function, and then optimize the initial mapping function based on a preset interpolation algorithm to obtain the mapping function.

[0110] The computer device can also fit the at least one target point pair using a physical model to obtain an initial mapping function, and then optimize the initial mapping function using a pre-trained network model to obtain the mapping function.

[0111] In one example embodiment, as shown in Figure 5 , determining the mapping function based on the at least one target point pair includes the following steps:

[0112] Step 501: fitting the at least one target point pair based on a preset fitting algorithm to obtain an initial mapping function.

[0113] Optionally, the preset fitting algorithm can be, for example, a Taylor polynomial fitting algorithm, a nonlinear least squares algorithm, a spline interpolation fitting algorithm, a quadratic radial-tangential polynomial algorithm, etc.

[0114] In some example embodiments, the computer device can construct a fitting equation and solve optimal parameters based on a preset fitting algorithm, with the pixel coordinate of the first target point in the at least one target point pair as an input variable and the physical coordinate of the second target point in the at least one target point pair as an output variable, to obtain an initial mapping function.

[0115] At step 502, the initial mapping function is optimized based on a preset interpolation algorithm to obtain a mapping function.

[0116] Optionally, the preset interpolation algorithm can be a thin plate spline interpolation algorithm, a sparse basis function algorithm, etc.

[0117] In some example embodiments, after the computer device fits the at least one target point pair based on the preset fitting algorithm to obtain an initial mapping function, the initial mapping function can be optimized based on a preset interpolation algorithm to obtain a mapping function.

[0118] Specifically, the computer device can first determine the fitting residual of the initial mapping function, that is, for each target point pair, the pixel coordinate of the first target point is input into the initial mapping function to obtain a predicted physical coordinate, and then the difference (residual) between the predicted physical coordinate and the actual physical coordinate of the second target point is calculated. Then, the preset interpolation algorithm is called to construct an interpolation model with the pixel coordinate of the first target point as input and the corresponding residual as output, and a residual compensation function is obtained by minimizing the global bending energy of the interpolation model (thin plate spline interpolation algorithm) or selecting key basis functions to fit the residual distribution (sparse basis function algorithm). Finally, the output result of the initial mapping function is superimposed with the output result of the residual compensation function to realize the optimization of the initial mapping function and obtain a mapping function.

[0119] In one example embodiment, as shown in FIG. 6, Figure 6 determining the mapping function based on the at least one target point pair includes the following steps:

[0120] At step 601, the at least one target point pair is fitted using a physical model to obtain an initial mapping function.

[0121] The physical model is constructed based on a target geometric mapping relationship and / or a target optical path transmission characteristic. The target geometric mapping relationship is a geometric mapping relationship between the pupil camera and the OCT device, and the target optical path transmission characteristic includes an optical path transmission characteristic of the pupil camera and an optical path transmission characteristic of the OCT device.

[0122] In some example embodiments, the computer device can fit the at least one target point pair using a physical model to obtain an initial mapping function.

[0123] Specifically, the computer device can construct a geometric mapping equation between pixel coordinates and physical coordinates by substituting the relative pose parameters of the pupil camera and the OCT device, the optical system parameters, and the optical path transmission characteristic parameters into a preset physical model. Then, with the target point pair as the constraint condition, the optimal parameters of the geometric mapping equation are solved by optimization algorithms such as the least squares method to obtain the initial mapping function.

[0124] If an optical distortion model is selected, the focus is on fitting and compensating for the radial and tangential distortions of the pupil camera; if a refractive model is selected, the refractive index parameters of media such as the cornea and lens are further incorporated to correct the coordinate deviation caused by light refraction.

[0125] Step 602: Optimize the initial mapping function using the pre-trained network model to obtain the mapping function.

[0126] Optionally, the network model can be a regression network model.

[0127] In some exemplary embodiments, after fitting at least one pair of target points to a physical model to obtain an initial mapping function, the computer device may optimize the initial mapping function using a pre-trained network model to obtain a further mapping function.

[0128] Specifically, the computer device can first calculate the fitting residual of the initial mapping function. That is, for each pair of target points, the pixel coordinates of the first target point are input into the initial mapping function to obtain the predicted physical coordinates. Then, the difference between the predicted physical coordinates and the actual physical coordinates of the second target point is calculated to form a residual dataset. Then, the pixel coordinates corresponding to the residual dataset are used as input and the residuals are used as output, and input into a pre-trained regression network model. The network model learns the nonlinear mapping relationship between the pixel coordinates and the residuals to obtain the residual compensation function. Finally, the output of the initial mapping function is superimposed with the output of the residual compensation function to optimize the initial mapping function and obtain the mapping function.

[0129] In one exemplary embodiment, such as Figure 7 As shown, after determining the mapping function, the method also includes the following steps:

[0130] Step 701: Obtain the initial eye image of the examined eye using a pupil camera.

[0131] The initial eye image includes corneal light-reflecting points. Corneal light-reflecting points are high-contrast reflective feature points formed on the anterior surface of the cornea by projecting a matrix or stripes onto the corneal surface of the examined eye using a structured light projector. In an optional embodiment, the corneal light-reflecting points may be light-reflecting points located near the corneal apex, corresponding to the optical axis of the system.

[0132] In some example embodiments, the computer device can acquire an initial eye image of the eye under examination by using the pupil camera.

[0133] Specifically, the computer device can first send a projection control instruction to the structured light projector to control the structured light projector to project structured light to the cornea of the eye under examination. After the projection is stable, the computer device can trigger the pupil camera to acquire an eye image of the eye under examination to obtain the initial eye image.

[0134] Step 702, correcting the initial eye image by using a mapping function to obtain a target eye image, and acquiring real-time position information of the corneal reflection point in the target eye image.

[0135] Optionally, the real-time position information of the corneal reflection point can be used to indicate the position of the corneal reflection point on the eye under examination.

[0136] In some example embodiments, after acquiring the initial eye image of the eye under examination by using the pupil camera, the computer device can correct the initial eye image by using a mapping function to obtain a target eye image.

[0137] Specifically, the computer device can first acquire the pixel coordinates of the corneal reflection point in the initial eye image, and then use the pixel coordinates of the corneal reflection point as input to complete the conversion from pixel coordinates to three-dimensional physical coordinates by using the mapping function, eliminate the deviation caused by the geometric distortion, coordinate nonlinearity and light refraction of the pupil camera, obtain the corrected three-dimensional physical coordinates of the corneal reflection point, and then perform pixel remapping processing on the initial eye image based on the corrected corneal reflection point coordinates to generate the target eye image.

[0138] Further, after correcting the initial eye image by using the mapping function to obtain the target eye image, the computer device can acquire real-time position information of the corneal reflection point in the target eye image.

[0139] Specifically, the computer device can use the three-dimensional physical coordinates of the corneal reflection point in the target eye image as the real-time position information of the corneal reflection point.

[0140] Step 703, comparing the real-time position information with standard position information of the corneal reflection point, and determining the eye movement condition of the eye under examination according to the comparison result.

[0141] The standard position information is the position information of the corneal reflection point when the eye under examination does not move.

[0142] Optionally, the eye movement condition of the eye under examination can be used to indicate whether the eye under examination moves, and the degree, direction, etc. of the eye movement.

[0143] In some example embodiments, after the computer device acquires the real-time position information of the corneal reflection point in the target eye image, the computer device can compare the real-time position information with the standard position information of the corneal reflection point, and determine the eye movement of the test eye according to the comparison result.

[0144] Specifically, the computer device can calculate the deviation value of the standard position information and the real-time position information in the three-dimensional direction, compare the deviation value with the preset deviation threshold value, if the deviation value is less than the preset deviation threshold value, it can be determined that the test eye does not have effective eye movement or does not have eye movement, if the deviation value is greater than or equal to the preset deviation threshold value, it can be determined that the test eye has effective eye movement, then the eye movement direction can be determined according to the deviation direction, and the eye movement degree can be determined according to the deviation size.

[0145] Further, in the case where the eye movement of the test eye indicates that the eye movement degree is greater than the preset eye movement degree threshold value, an eye movement reminder is outputted, and the eye movement reminder can be used to prompt to stop scanning and other safety operations.

[0146] Since the target eye image obtained after the correction processing by the mapping function avoids the geometric distortion of the pupil camera, the nonlinear deviation of the pixel to the physical coordinate, and the position deviation caused by the refraction of the light through the cornea and the lens, the eye movement detection result determined based on the target eye image has higher accuracy.

[0147] In one example embodiment, as shown in Figure 8 Another correction method of the image collected by the pupil camera is provided, including the following steps:

[0148] Step 810, acquiring initial eye data of the test eye by using the pupil camera and the OCT device, the initial eye data including an initial first image collected by the pupil camera and an initial second image collected by the OCT device;

[0149] Step 802, acquiring the signal amplitude and the noise amplitude of the initial eye data, and determining the signal-to-noise ratio of the initial eye data based on the signal amplitude and the noise amplitude; acquiring the edge integrity and the contrast of the pupil in the initial eye data, and determining the pupil detection confidence of the initial eye data based on the edge integrity and the contrast; acquiring the displacement information of the pupil in the initial eye data, and determining the instantaneous motion of the initial eye data based on the displacement information; determining the quality information of the initial eye data based on the signal-to-noise ratio, the pupil detection confidence and the instantaneous motion;

[0150] Step 803, in the case where the quality information of the initial eye data meets the preset condition, determining the initial eye data as the eye data, the eye data including the first image collected by the pupil camera and the second image collected by the OCT device;

[0151] Step 804, determining at least one first target point based on the first image, and determining at least one second target point corresponding to the at least one first target point from the second image; obtaining pixel coordinates of the at least one first target point, and obtaining physical coordinates of the at least one second target point; for any one of the at least one first target point, constructing a target point pair based on the pixel coordinates of the first target point and the physical coordinates of the second target point corresponding to the first target point, to obtain at least one target point pair;

[0152] Step 805, fitting the at least one target point pair based on a preset fitting algorithm to obtain an initial mapping function; optimizing the initial mapping function based on a preset interpolation algorithm to obtain a mapping function; or, fitting the at least one target point pair by using a physical model to obtain an initial mapping function, the physical model being constructed based on a target geometric mapping relationship and / or a target optical path transmission characteristic, the target geometric mapping relationship being a geometric mapping relationship between the pupil camera and the OCT device, and the target optical path transmission characteristic including an optical path transmission characteristic of the pupil camera and an optical path transmission characteristic of the OCT device; optimizing the initial mapping function by using a pre-trained network model to obtain the mapping function, the mapping function being used for correcting the eye image collected by the pupil camera;

[0153] Step 806, obtaining an initial eye image of the examined eye collected by using the pupil camera; the initial eye image including a corneal reflex point; performing correction processing on the initial eye image by using the mapping function to obtain a target eye image, and obtaining real-time position information of the corneal reflex point in the target eye image; comparing the real-time position information with standard position information of the corneal reflex point, and determining an eye movement condition of the examined eye according to a comparison result; the standard position information being position information of the corneal reflex point when the examined eye does not have eye movement.

[0154] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0155] Based on the same inventive concept, the present application further provides a pupil camera collected image correction device for implementing the pupil camera collected image correction method described above. The implementation scheme of the device for solving the problem is similar to the implementation scheme described in the above method, so the specific limitations in one or more pupil camera collected image correction device embodiments provided below can refer to the limitations of the pupil camera collected image correction method described above, which will not be repeated here.

[0156] In one exemplary embodiment, as shown in Figure 9 A pupil camera collected image correction device 900 is provided, comprising: an acquisition module 901, a determination module 902, and an execution module 903, wherein:

[0157] The acquisition module 901 is configured to acquire eye data of an eye under examination by using a pupil camera and an OCT device, the eye data comprising a first image collected by the pupil camera and a second image collected by the OCT device.

[0158] The determination module 902 is configured to determine at least one first target point based on the first image, and determine at least one second target point corresponding to the at least one first target point from the second image.

[0159] The execution module 903 is configured to determine a mapping function based on the at least one first target point and the at least one second target point, the mapping function being used for correcting an eye image collected by the pupil camera.

[0160] In one embodiment, the acquisition module 901 is specifically configured to acquire initial eye data of the eye under examination by using the pupil camera and the OCT device, the initial eye data comprising an initial first image collected by the pupil camera and an initial second image collected by the OCT device; determine quality information of the initial eye data, and determine the initial eye data as the eye data in a case where the quality information of the initial eye data meets a preset condition.

[0161] In one embodiment, the acquisition module 901 is specifically configured to acquire a signal amplitude and a noise amplitude of the initial eye data, and determine a signal-to-noise ratio of the initial eye data based on the signal amplitude and the noise amplitude; acquire an edge integrity and a contrast of a pupil in the initial eye data, and determine a pupil detection confidence of the initial eye data based on the edge integrity and the contrast; acquire displacement information of the pupil in the initial eye data, and determine an instantaneous motion of the initial eye data based on the displacement information; and determine the quality information of the initial eye data based on the signal-to-noise ratio, the pupil detection confidence, and the instantaneous motion.

[0162] In an embodiment, the execution module 903 is specifically configured to acquire pixel coordinates of at least one first target point and acquire physical coordinates of at least one second target point; for any first target point in the at least one first target point, construct a target point pair based on the pixel coordinates of the first target point and the physical coordinates of the second target point corresponding to the first target point, to obtain at least one target point pair; and determine the mapping function based on the at least one target point pair.

[0163] In an embodiment, the execution module 903 is specifically configured to perform fitting on the at least one target point pair based on a preset fitting algorithm, to obtain an initial mapping function; and perform optimization on the initial mapping function based on a preset interpolation algorithm, to obtain the mapping function.

[0164] In an embodiment, the execution module 903 is specifically configured to perform fitting on the at least one target point pair based on a physical model, to obtain an initial mapping function; the physical model is constructed based on a target geometric mapping relationship and / or a target optical path transmission characteristic, the target geometric mapping relationship being a geometric mapping relationship between a pupil camera and an OCT device, and the target optical path transmission characteristic including an optical path transmission characteristic of the pupil camera and an optical path transmission characteristic of the OCT device; and perform optimization on the initial mapping function based on a pre-trained network model, to obtain the mapping function.

[0165] In an embodiment, the execution module 903 is further configured to acquire an initial eye image of a test eye collected by using the pupil camera; the initial eye image includes a corneal reflex point; perform correction processing on the initial eye image based on the mapping function, to obtain a target eye image, and acquire real-time position information of the corneal reflex point in the target eye image; compare the real-time position information with standard position information of the corneal reflex point, and determine an eye movement condition of the test eye according to a comparison result; the standard position information is position information of the corneal reflex point when the test eye does not have eye movement.

[0166] The above various modules in the correction device for the image collected by the pupil camera can be all or partially implemented by software, hardware, and combinations thereof. The above various modules can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in the computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to the above various modules.

[0167] In an exemplary embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in FIG. 8. Figure 10As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control ability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the terminal outside through the network connection. The computer program is executed by the processor to implement a correction method of an image collected by a pupil camera.

[0168] In an exemplary embodiment, a computer device, which can be a terminal, is provided, and an internal structure diagram thereof can be as shown in the figure. Figure 11 As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control ability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the terminal outside through the network connection. The computer program is executed by the processor to implement a correction method of an image collected by a pupil camera.

[0169] Those skilled in the art can understand that, Figure 10 and Figure 11The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0170] In one exemplary embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0171] acquiring eye data of the eye under examination using the pupil camera and the OCT device, the eye data comprising a first image acquired by the pupil camera and a second image acquired by the OCT device;

[0172] determining at least one first target point based on the first image, and determining at least one second target point corresponding to the at least one first target point from the second image;

[0173] determining a mapping function based on the at least one first target point and the at least one second target point, the mapping function being used to correct an eye image acquired by the pupil camera.

[0174] In one embodiment, the processor further implements the following steps when executing the computer program: acquiring initial eye data of the eye under examination using the pupil camera and the OCT device, the initial eye data comprising an initial first image acquired by the pupil camera and an initial second image acquired by the OCT device; determining quality information of the initial eye data, and determining the initial eye data as the eye data in a case where the quality information of the initial eye data meets a preset condition.

[0175] In one embodiment, the processor further implements the following steps when executing the computer program: acquiring a signal amplitude and a noise amplitude of the initial eye data, and determining a signal-to-noise ratio of the initial eye data based on the signal amplitude and the noise amplitude; acquiring an edge integrity and a contrast of the pupil in the initial eye data, and determining a pupil detection confidence of the initial eye data based on the edge integrity and the contrast; acquiring displacement information of the pupil in the initial eye data, and determining an instantaneous motion of the initial eye data based on the displacement information; determining the quality information of the initial eye data based on the signal-to-noise ratio, the pupil detection confidence, and the instantaneous motion.

[0176] In one embodiment, the processor further implements the following steps when executing the computer program: acquiring pixel coordinates of the at least one first target point, and acquiring physical coordinates of the at least one second target point; for any one of the at least one first target point, constructing a target point pair based on the pixel coordinates of the first target point and the physical coordinates of the second target point corresponding to the first target point to obtain at least one target point pair; determining the mapping function based on the at least one target point pair.

[0177] In one embodiment, the processor, when executing the computer program, also implements the following steps: fitting the at least one target point pair based on a preset fitting algorithm to obtain an initial mapping function; and optimizing the initial mapping function based on a preset interpolation algorithm to obtain the mapping function.

[0178] In one embodiment, the processor, when executing the computer program, also implements the following steps: fitting the at least one target point pair by using a physical model to obtain an initial mapping function; the physical model is constructed based on a target geometric mapping relationship and / or a target optical path transmission characteristic, the target geometric mapping relationship being a geometric mapping relationship between the pupil camera and the OCT device, and the target optical path transmission characteristic including an optical path transmission characteristic of the pupil camera and an optical path transmission characteristic of the OCT device; and optimizing the initial mapping function by using a pre-trained network model to obtain the mapping function.

[0179] In one embodiment, the processor, when executing the computer program, also implements the following steps: obtaining an initial eye image of the examined eye collected by using the pupil camera; the initial eye image including a corneal reflex point; performing correction processing on the initial eye image by using the mapping function to obtain a target eye image, and obtaining real-time position information of the corneal reflex point in the target eye image; comparing the real-time position information with standard position information of the corneal reflex point, and determining an eye movement condition of the examined eye according to a comparison result; the standard position information being position information of the corneal reflex point when the examined eye does not have eye movement.

[0180] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium has stored thereon a computer program. The computer program, when executed by a processor, implements the following steps:

[0181] collecting eye data of the examined eye by using the pupil camera and the OCT device, the eye data including a first image collected by the pupil camera and a second image collected by the OCT device;

[0182] determining at least one first target point based on the first image, and determining at least one second target point corresponding to the at least one first target point from the second image;

[0183] determining a mapping function based on the at least one first target point and the at least one second target point, the mapping function being used for correcting an eye image collected by the pupil camera.

[0184] In one embodiment, the computer program, when executed by the processor, also implements the following steps: collecting initial eye data of the examined eye by using the pupil camera and the OCT device, the initial eye data including an initial first image collected by the pupil camera and an initial second image collected by the OCT device; determining quality information of the initial eye data, and determining the initial eye data as the eye data in a case where the quality information of the initial eye data satisfies a preset condition.

[0185] In one embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining a signal amplitude and a noise amplitude of the initial eye data, and determining a signal-to-noise ratio of the initial eye data based on the signal amplitude and the noise amplitude; obtaining an edge integrity and a contrast of the pupil in the initial eye data, and determining a pupil detection confidence of the initial eye data based on the edge integrity and the contrast; obtaining displacement information of the pupil in the initial eye data, and determining an instantaneous motion of the initial eye data based on the displacement information; determining quality information of the initial eye data based on the signal-to-noise ratio, the pupil detection confidence, and the instantaneous motion.

[0186] In one embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining pixel coordinates of at least one first target point, and obtaining physical coordinates of at least one second target point; for any first target point in the at least one first target point, constructing a target point pair based on the pixel coordinates of the first target point and the physical coordinates of the second target point corresponding to the first target point to obtain at least one target point pair; determining the mapping function based on the at least one target point pair.

[0187] In one embodiment, the computer program, when executed by the processor, further implements the following steps: fitting the at least one target point pair based on a preset fitting algorithm to obtain an initial mapping function; and optimizing the initial mapping function based on a preset interpolation algorithm to obtain the mapping function.

[0188] In one embodiment, the computer program, when executed by the processor, further implements the following steps: fitting the at least one target point pair based on a physical model to obtain an initial mapping function; the physical model is constructed based on a target geometric mapping relationship and / or a target optical path transmission characteristic, the target geometric mapping relationship being a geometric mapping relationship between the pupil camera and the OCT device, and the target optical path transmission characteristic including an optical path transmission characteristic of the pupil camera and an optical path transmission characteristic of the OCT device; and optimizing the initial mapping function based on a pre-trained network model to obtain the mapping function.

[0189] In one embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining an initial eye image of the eye under examination acquired by the pupil camera; the initial eye image including a corneal reflection point; performing correction processing on the initial eye image based on the mapping function to obtain a target eye image, and obtaining real-time position information of the corneal reflection point in the target eye image; comparing the real-time position information with standard position information of the corneal reflection point, and determining an eye movement condition of the eye under examination according to a comparison result; the standard position information being position information of the corneal reflection point when the eye under examination does not have eye movement.

[0190] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method according to any of the above embodiments.

[0191] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0192] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0193] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered to be within the scope of the present application.

[0194] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for correcting images acquired by a pupil camera, characterized in that, The method comprises: acquiring eye data of an eye under test by using a pupil camera and an OCT device, the eye data comprising a first image acquired by the pupil camera and a second image acquired by the OCT device; determining at least one first target point based on the first image, and determining at least one second target point corresponding to the at least one first target point from the second image; determining a mapping function based on the at least one first target point and the at least one second target point, the mapping function being used for correcting an eye image acquired by the pupil camera.

2. The method of claim 1, wherein, The acquiring of the eye data of the eye under test by using the pupil camera and the OCT device comprises: acquiring initial eye data of the eye under test by using the pupil camera and the OCT device, the initial eye data comprising an initial first image acquired by the pupil camera and an initial second image acquired by the OCT device; determining quality information of the initial eye data, and determining the initial eye data as the eye data in a case where the quality information of the initial eye data meets a preset condition.

3. The method of claim 2, wherein, The determining of the quality information of the initial eye data comprises: acquiring a signal amplitude and a noise amplitude of the initial eye data, and determining a signal-to-noise ratio of the initial eye data based on the signal amplitude and the noise amplitude; acquiring an edge integrity and a contrast of a pupil in the initial eye data, and determining a pupil detection confidence of the initial eye data based on the edge integrity and the contrast; acquiring displacement information of the pupil in the initial eye data, and determining an instantaneous motion of the initial eye data based on the displacement information; determining the quality information of the initial eye data based on the signal-to-noise ratio, the pupil detection confidence and the instantaneous motion.

4. The method according to any one of claims 1 to 3, characterized in that, The determining of the mapping function based on the at least one first target point and the at least one second target point comprises: acquiring pixel coordinates of the at least one first target point, and acquiring physical coordinates of the at least one second target point; for any first target point in the at least one first target point, constructing a target point pair based on the pixel coordinates of the first target point and the physical coordinates of the second target point corresponding to the first target point to obtain at least one target point pair; determining the mapping function based on the at least one target point pair.

5. The method of claim 4, wherein, The determining of the mapping function based on the at least one target point pair comprises: fitting the at least one target point pair based on a preset fitting algorithm to obtain an initial mapping function; optimizing the initial mapping function based on a preset interpolation algorithm to obtain the mapping function.

6. The method of claim 4, wherein, The determining of the mapping function based on the at least one target point pair comprises: fitting the at least one target point pair by using a physical model to obtain an initial mapping function, the physical model being constructed based on a target geometric mapping relationship and / or a target optical path transmission characteristic, the target geometric mapping relationship being a geometric mapping relationship between the pupil camera and the OCT device, and the target optical path transmission characteristic comprising an optical path transmission characteristic of the pupil camera and an optical path transmission characteristic of the OCT device; The initial mapping function is optimized by using a pre-trained network model to obtain the mapping function.

7. The method of claim 1, wherein, The method further comprises: An initial eye image of the eye under examination is acquired by using the pupil camera; the initial eye image includes a corneal reflection point; The initial eye image is corrected by using the mapping function to obtain a target eye image, and real-time position information of the corneal reflection point in the target eye image is acquired; The real-time position information and standard position information of the corneal reflection point are compared, and the eye movement of the eye under examination is determined according to the comparison result; the standard position information is the position information of the corneal reflection point when the eye under examination does not move.

8. An apparatus for correcting images captured by a pupillary camera, characterized by, The device comprises: An acquisition module is configured to acquire eye data of an eye under examination by using a pupil camera and an OCT device, wherein the eye data includes a first image acquired by the pupil camera and a second image acquired by the OCT device; A determination module is configured to determine at least one first target point based on the first image, and determine at least one second target point corresponding to the at least one first target point from the second image; An execution module is configured to determine a mapping function based on the at least one first target point and the at least one second target point, wherein the mapping function is used to correct an eye image acquired by the pupil camera. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7.

Images