A machine learning model training and lens zonule assessment system
Through machine learning models and image feature detection algorithms, combined with camera calibration, the problem of inaccurate evaluation of biomechanical state of lens suspension ligament in the prior art is solved, achieving higher diagnostic accuracy and treatment safety.
Patent Information
- Application Number
- CN202111521487.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-13
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-13
AI Technical Summary
The prior art cannot accurately evaluate the biomechanical status of the lens suspension ligament, resulting in inaccurate diagnosis of diseases related to suspension ligament abnormalities, affecting the patient's treatment timing and treatment effect.
The machine learning model training system is adopted to collect eyeball parameters and force information, establish a force analysis model, combine image feature detection algorithms and camera calibration, optimize eyeball images and obtain the biomechanical state of lens suspension ligament.
It improves the accuracy of the evaluation of the biomechanical status of the lens suspension ligament, reduces the error in disease diagnosis, and ensures the accuracy and safety of treatment.
Smart Images

Figure CN114418943B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical technology, and more specifically, to a machine learning model training and lens zonules evaluation system. Background Art
[0002] The lens suspensory ligament is a structure that the ciliary muscle pulls to fix the lens. The suspensory ligament adjusts the curvature of the lens as the ciliary muscle contracts and relaxes, achieving normal refractive function of the eyeball. During cataract extraction, abnormal relaxation of the suspensory ligament can make the lens unstable during the operation, increase the possibility of vitreous overflow and perfusion fluid stray syndrome, and even cause the lens to fall into the vitreous cavity after the suspensory ligament breaks. Abnormal relaxation of the suspensory ligament can lead to anterior displacement of the lens-iris septum, shallowing of the anterior chamber, and closure of the angle, which is also one of the main causes of secondary acute angle-closure glaucoma.
[0003] Suspensory ligament relaxation can cause many adverse consequences and damage the patient's vision health. However, the current clinical diagnosis of abnormal suspensory ligament relaxation can only be done through surgical exploration or ultrasound biomicroscopy (UBM). The former can directly and dynamically determine the elasticity of the suspensory ligament, but this method is invasive and cannot be used as a routine diagnostic method; the latter can only detect abnormal length of the suspensory ligament, which is a static examination and cannot directly determine the elasticity of the suspensory ligament. The sensitivity and specificity of the diagnosis of suspensory ligament relaxation are very low. Obviously, neither of them is an effective diagnostic method for suspensory ligament relaxation. The core of suspensory ligament relaxation is the abnormality of its biomechanical state, that is, the suspensory ligament has lost its due viscoelasticity, and the current clinical evaluation method for the biomechanical state of the suspensory ligament is still blank. The existing technology can measure the biomechanical state of the cornea. Its principle is simply to apply a suitable force to the cornea and record its deformation process. Then, the algorithm is used to identify the anterior and posterior boundaries of the cornea and calculate the deformation amplitude of the central corneal area. The ratio of the force applied to the cornea to the corneal deformation amplitude can reflect the biomechanical state of the cornea. Mechanically, the lens and the zonular ligament can be regarded as a whole, so the same principle can be applied to the zonular ligament of the lens. If the zonular ligament is abnormally loose, the lens will inevitably show a larger displacement after being subjected to the same force. The ratio of the force on the lens to the displacement of the lens can reflect the biomechanical state of the zonular ligament. In addition to the ratio of the force on the lens to the displacement of the lens, the ratio of the displacement of the lens to the deformation of the cornea can also reflect the biomechanical state of the zonular ligament. The existing technology cannot accurately obtain the force on the lens, the displacement of the lens and the deformation of the cornea, making it difficult to evaluate the biomechanical state of the zonular ligament of the lens. The inaccurate diagnosis of diseases related to abnormal zonular ligaments has become one of the urgent problems to be solved in the entire field of medical technology. Therefore, there is an urgent need for a machine learning model training and lens zonular ligament evaluation system that can accurately evaluate the biomechanical state of the lens zonular ligament. Summary of the invention
[0004] The present invention aims to overcome at least one defect of the above-mentioned prior art and provide a machine learning model training and a lens zonules evaluation system for evaluating the biomechanical state of the lens zonules.
[0005] The technical solution adopted by the present invention is:
[0006] A machine learning model training system, comprising:
[0007] A collection unit, used for collecting training samples; the training samples include: eyeball parameters and eyeball force information corresponding to the eyeball parameters; the eyeball parameters include: corneal biomechanics, corneal thickness, intraocular pressure and anterior chamber depth;
[0008] A training unit is used to establish a force analysis model based on a machine learning algorithm, and to train the force analysis model using the training samples.
[0009] Specifically, the lens is located inside the eyeball and cannot be directly subjected to force. This solution can read the force applied to the lens after the eyeball is stimulated by air pulses by implanting a pressure sensor into the lens of the ex vivo eyeball. The corresponding instruments can be used to obtain eyeball parameters such as corneal biomechanical state, corneal thickness, intraocular pressure, anterior chamber depth and eye axis. A force analysis model is established based on a machine learning algorithm, and the eyeball parameters are used as input data and the lens force magnitude as output indicators to train the force model. In the end, only the eyeball parameters need to be input into the force model to obtain the lens force magnitude F.
[0010] Furthermore, the machine learning algorithms include: LR regression algorithm, SVM algorithm, XGBoost algorithm, RandomForest algorithm and Extra Tree algorithm.
[0011] Specifically, the XGBoost algorithm is one of the most outstanding algorithms in the current machine learning algorithms. It takes the decision tree algorithm as the core idea, adds the L2 regularization term design and supports the second-order Taylor expansion of the loss function, and further uses the second-order derivative information of the data to optimize the model parameters, so that it has very good generalization and fit in practical applications. At the same time, traditional machine learning algorithms such as LR regression and SVM are also used for effect comparison. As one of the classic machine learning algorithms, LR regression uses a calculation method similar to the input and output of a single-layer neural network. It can use the gradient update method and the cross entropy loss calculation method to update the calculation parameters. It also has good nonlinear expression ability, and can also introduce regularization terms to ensure the generalization of the model algorithm; the SVM algorithm also has certain advantages in the classification hyperplane calculation through linear kernel / nonlinear functions, and can obtain good results. Finally, the integration of multiple algorithms such as LR, SVM, XGBoost, RandomForest and Extra Tree can effectively reduce the model error and make its evaluation of the lens stress more accurate.
[0012] A lens zonule assessment system, comprising:
[0013] An imaging unit, used for collecting an eyeball image and eyeball parameters; the eyeball image is an image of the eyeball tissue deformed and displaced after the eyeball is subjected to force;
[0014] An optimization unit, used for optimizing the eyeball image;
[0015] A processing unit, configured to extract edges of the optimized eyeball image based on an image feature detection algorithm, and to obtain the deformation amplitude and / or displacement of the eyeball tissue after the eyeball is subjected to force from the eyeball image after edge extraction;
[0016] An analysis unit, used for inputting the eyeball parameters into a trained force analysis model to obtain eyeball force information; the force analysis model is obtained after training by the above system;
[0017] The evaluation unit is used to evaluate the lens suspensory ligament according to the deformation amplitude and / or displacement of the eyeball tissue after the eyeball is subjected to force, and the corresponding eyeball force information, so as to obtain the biomechanical state of the lens suspensory ligament.
[0018] Specifically, the existing technology cannot obtain the biomechanical state of the lens suspensory ligament, resulting in inaccurate diagnosis of diseases related to abnormal suspensory ligament. Inaccurate disease diagnosis may delay the patient's condition and miss the best treatment time, or even take the wrong treatment method, resulting in medical accidents. In order to avoid the occurrence of medical accidents and improve the accuracy of diagnosing diseases related to abnormal suspensory ligament, the system of this solution evaluates the lens suspensory ligament according to the deformation amplitude and / or displacement of the eyeball tissue and the corresponding eyeball force information. First, the eyeball image and eyeball parameters are collected; secondly, the eyeball image is optimized to improve the resolution and clarity of the eyeball image; then, the eyeball image is edge extracted and the deformation amplitude and / or displacement of the eyeball tissue is extracted based on the image feature detection algorithm; then, the training samples are collected to train the force analysis model established based on the machine learning algorithm; finally, the eyeball parameters are input into the trained force analysis model to obtain the eyeball force information, and the lens suspensory ligament is evaluated according to the deformation amplitude and / or displacement of the eyeball tissue and the corresponding eyeball force information. This solution not only improves the quality of eyeball images by optimizing eyeball images and extracting edges based on image feature detection algorithms, but also makes the subsequent acquisition of eyeball tissue deformation amplitude and displacement more accurate. In addition, a machine learning model is used to reduce the error of eyeball force information and further improve the accuracy of the biomechanical state of the lens suspensory ligament.
[0019] Furthermore, it also includes:
[0020] The calibration unit is used to calibrate the image collector for collecting eye images before the image unit collects eye images and eye parameters.
[0021] Specifically, the image collector includes: a tilt-shift camera and a data acquisition computer connected thereto; the calibration is camera calibration (camera calibration). The tilt-shift camera is used to collect eye image data; the data acquisition computer is used to calculate the camera calibration result. Whether in image measurement or machine vision applications, the calibration of camera parameters is a very critical link, and the accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera. Therefore, in order to correct the distortion at the image point caused by factors such as lens aberration, the camera calibration needs to be performed before collecting the eye image and eye parameters.
[0022] Furthermore, the calibration unit includes:
[0023] The initial module is used to establish the image distortion error model;
[0024] A coordinate module is used to collect the original grid image and establish a coordinate system using the longitudinal and transverse grid lines in the grid image as coordinate axes to obtain the original grid node coordinates;
[0025] A node module is used to fit the middle area of the original grid image to obtain a target grid image and target grid node coordinates;
[0026] An error module, used for comparing the original grid node coordinates with the target grid node coordinates to obtain the image distortion error of each grid node;
[0027] A parameter module, used for inputting the target grid node coordinates and the corresponding image distortion error into the image distortion error model to calculate the distortion parameter;
[0028] An output module, used to input the target pixel point coordinates and the distortion parameters into the image distortion error model to obtain the corresponding image distortion error and the original pixel point coordinates; the target pixel point coordinates are the coordinates of the pixel points of the target grid image; the original pixel point coordinates are the coordinates of the points corresponding to the pixel points of the target grid image on the original grid image;
[0029] The correction module is used to collect the grayscale value of the original pixel point coordinates / the grayscale value adjacent to the original pixel point coordinates, and assign the grayscale value to the corresponding target pixel point to obtain the target grid image after distortion correction.
[0030] The specific steps of camera calibration are as follows:
[0031] An image distortion error model is established, and the model is:
[0032]
[0033] Among them, a0~a9 and b0~b9 are distortion parameters.
[0034] The original grid image is collected, and a coordinate system is established with the vertical and horizontal grid lines in the grid image as coordinate axes to obtain the original grid node coordinates; the acquisition of the original grid image is as follows: a tilt-shift camera is used to shoot a high-precision standard orthogonal grid plate, and the grid fills the entire imaging surface during shooting. The establishment of a coordinate system with the vertical and horizontal grid lines in the grid image as coordinate axes to obtain the original grid node coordinates is as follows: the grid row 0, column 0 node is set as the reference point, and the reference point coordinates are (x 00 ,y 00 ), the spacing between the horizontal grid lines is Δx, the spacing between the vertical grid lines is Δy, the angle between the horizontal grid lines and the horizontal direction is α, and the grid node coordinates are:
[0035]
[0036] Where i is the number of node columns, j is the number of node rows, k = tanα, d0 = y 00 -kx 00 , c0=x00 +ky 00 .
[0037] The middle area of the original grid image is fitted to obtain the target grid image and the target grid node coordinates. Since the aberration of the middle area of the eyeball is very small in actual shooting, this scheme uses the grid nodes within 1 / 3 of the middle area of the original grid image to fit the target grid image, that is, the node coordinates (x ij ,y ij ) into equation (1.2) to solve for the target grid image parameters k, d0, c0, Δx, Δy.
[0038] The original grid node coordinates are compared with the target grid node coordinates to obtain the image distortion error of each grid node; the image distortion error is
[0039] The target grid node coordinates and the corresponding image distortion error are input into the image distortion error model to calculate the distortion parameters; that is, (x ij ,y ij )as well as Input formula (1.1) and calculate the distortion parameters a0~a9, b0~b9.
[0040] Input the target pixel coordinates and the distortion parameters into the image distortion error model to obtain the corresponding image distortion error and original pixel coordinates; that is, input (x, y) and a0~a9, b0~b9 into formula (1.1) to obtain the image distortion of (x, y) and (x', y').
[0041] Collect the grayscale values adjacent to the original pixel coordinates / original pixel coordinates, and assign the grayscale values to the corresponding target pixel points to obtain the target grid image after distortion correction; since (x', y') is usually not an integer, it is necessary to interpolate the grayscale of the pixels near the position (x', y') to obtain the grayscale value corresponding to (x', y'), and assign the grayscale value to the target pixel point (x, y), so as to perform distortion correction on the target grid image.
[0042] Furthermore, the optimization unit is used to optimize the eyeball image, including:
[0043] The optimization unit is used to optimize the eyeball image based on a maximum a posteriori probability estimation algorithm to obtain an optimized eyeball image.
[0044] Specifically, since excessive force can cause damage to the eyeball, the air pulse applied to the eyeball must have an appropriate force when collecting the eyeball image. Because the displacement of the lens caused by an appropriate force that does not cause damage to the eyeball is in the micron level, the collected eyeball image must have a sufficiently high resolution to accurately identify the corresponding information (lens displacement, etc.) from the image. The steps of optimizing the eyeball image based on the maximum a posteriori probability estimation algorithm and obtaining a high-resolution eyeball image are specifically as follows:
[0045] Based on the maximum a posteriori estimation method, the reconstruction model is constructed:
[0046]
[0047] Among them, l is a low-resolution and high-noise image of the eye, N is the number of image pixels, h is a high-resolution image of the eye, D and H are blur and downsampling matrices, is a certain regular transformation, and γ is the penalty coefficient.
[0048] In order to simultaneously perform noise reduction in high-resolution reconstruction of eye images, the alternating direction method of multiplier (ADMM) is substituted into formula (2.1):
[0049]
[0050] Among them, z is the auxiliary variable and μ is the regularization coefficient.
[0051] Decompose equation (2.2) and solve:
[0052]
[0053]
[0054] Formula (2.2.1) and formula (2.2.2) are solved by iterative formula (2.3.1) (2.3.2) (2.3.3); the iterative formula (2.5) (2.6) (2.7) is as follows:
[0055]
[0056]
[0057] z k+1 =z k +(h k+1 -v k+1 ) (2.3.3)
[0058] in, It can be any available algorithm for solving Gaussian noise reduction.
[0059] Construct the fuzzy kernel function and solve the fuzzy matrix H;
[0060] The key to solving the fuzzy matrix H is to solve the fuzzy kernel function. Using the Gaussian point spread function (PSF) as the fuzzy kernel function can further reduce the content loss of the eye image. The Gaussian point spread function (PSF) is as follows:
[0061]
[0062] Among them, i, j represent the coordinates in the 2D coordinate system, s 2 is the variance, M is the normalization coefficient, and s is the parameter to be solved.
[0063] The statistical non-Bayesian minimization based on Stein unbiased risk estimation (SURE) is used to accurately estimate the PSF parameters while taking into account the downsampling factor. The blur-SURE method improved by the downsampling process can obtain the blur-MSE–SR standard, which is as follows:
[0064]
[0065] Among them, H0 is the unknown fuzzy matrix to be solved, W H,ρ =(H T H+ρC) -1 H T , C is a eigenvalue with ||ω|| 2 The circular matrix of ω = (ω1, ω2) determines the spatial frequency. Through the Gaussian noise statistical method, the following formula is obtained:
[0066]
[0067] Solve the minimum value of formula (2.6) to obtain the high-resolution image h of the eyeball.
[0068] The minimization of two scalar variables (PSF parameter s and regularization parameter ρ) can be determined by exhaustive enumeration within a certain range. The amount of relevant calculations can be further reduced by performing exhaustive enumeration after Fourier transformation.
[0069] Furthermore, the image feature detection algorithm is a canny edge detection algorithm.
[0070] Furthermore, the processing unit includes:
[0071] A smoothing processing module, used for smoothing the eyeball image using a Gaussian filter;
[0072] An extraction module, used for performing edge extraction on the smoothed high-resolution eyeball image using a canny edge detection algorithm to obtain the edge of the eyeball tissue; the eyeball tissue is the lens and cornea;
[0073] A displacement module is used to calculate the displacement of the lens after the eyeball is subjected to force according to the edge of the lens;
[0074] The deformation module is used to calculate the deformation amplitude of the cornea after the eyeball is subjected to force according to the edge of the cornea.
[0075] Specifically, the eye image is first smoothed using a Gaussian filter with a standard deviation of σ to reduce noise and obtain a clear eye image. Then, the local gradient of each point in the image is calculated. and edge direction tan -1 (g x / g y ). Then, the amplitude is non-maximum suppressed along the gradient direction to refine the edge of the eye tissue, retain the local maximum gradient and suppress other gradient values to obtain a fine edge of the eye tissue. Finally, the displacement of the lens △d1 is obtained according to the difference between the highest and lowest points of the center vertex of the lens edge during the movement process. The deformation amplitude of the cornea △d2 is obtained according to the difference between the highest and lowest points of the center vertex of the cornea during the movement process and the displacement of the entire cornea is subtracted.
[0076] Furthermore, the processing unit further includes:
[0077] The threshold module is used to set a grayscale threshold for the eyeball image to distinguish the cornea from the lens.
[0078] Furthermore, the evaluation unit comprises:
[0079] An eyeball analysis module is used to obtain the force magnitude of the lens from the eyeball force information;
[0080] A lens analysis module, used for calculating the ratio of the lens force to the lens displacement as a first ratio;
[0081] A comparison module, used for calculating the ratio of the lens displacement to the corneal deformation amplitude as a second ratio;
[0082] A calculation module is used to evaluate the biomechanical state of the lens zonules according to the lens force magnitude and the first ratio and / or the second ratio.
[0083] Specifically, the magnitude of force F on the lens is obtained from the force information of the eyeball; the ratio of the magnitude of force on the lens to the displacement of the lens F / △d1 is calculated to reflect the biomechanical state of the lens suspensory ligament; in addition, the ratio of the displacement of the lens to the deformation amplitude of the cornea △d1 / △d2 is calculated to also reflect the biomechanical state of the lens suspensory ligament.
[0084] Compared with the prior art, the beneficial effects of the present invention are as follows: the system of this scheme not only improves the quality of the eyeball image by optimizing the eyeball image and extracting the edge based on the image feature detection algorithm, but also makes the subsequent acquisition of the deformation amplitude and displacement of the eyeball tissue more accurate. In addition, a machine learning model is also used to reduce the error of the eyeball force information, further improving the accuracy of the biomechanical state of the lens suspensory ligament. On this basis, camera calibration is also performed to correct the distortion at the image point caused by factors such as lens aberration, so as to increase the accuracy of the evaluation of the biomechanical state of the lens suspensory ligament. BRIEF DESCRIPTION OF THE DRAWINGS
[0085] Figure 1 It is a schematic diagram of the system structure of the present invention;
[0086] Figure 2 It is a schematic diagram of an orthogonal grid node of the present invention;
[0087] Figure 3 Schematic diagram of the eye section before and after force. DETAILED DESCRIPTION
[0088] The drawings of the present invention are only for illustrative purposes and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.
[0089] Example
[0090] This embodiment provides a machine learning model training system, including:
[0091] A collection unit, used for collecting training samples; the training samples include: eyeball parameters and eyeball force information corresponding to the eyeball parameters; the eyeball parameters include: corneal biomechanics, corneal thickness, intraocular pressure and anterior chamber depth;
[0092] A training unit is used to establish a force analysis model based on a machine learning algorithm, and to train the force analysis model using the training samples.
[0093] Specifically, the lens is located inside the eyeball and cannot be directly subjected to force. This solution can read the force applied to the lens after the eyeball is stimulated by air pulses by implanting a pressure sensor into the lens of the ex vivo eyeball. The corresponding instruments can be used to obtain eyeball parameters such as corneal biomechanical state, corneal thickness, intraocular pressure, anterior chamber depth and eye axis. A force analysis model is established based on a machine learning algorithm, and the eyeball parameters are used as input data and the lens force magnitude as output indicators to train the force model. In the end, only the eyeball parameters need to be input into the force model to obtain the lens force magnitude F.
[0094] Furthermore, the machine learning algorithms include: LR regression algorithm, SVM algorithm, XGBoost algorithm, RandomForest algorithm and Extra Tree algorithm.
[0095] Specifically, the XGBoost algorithm is one of the most outstanding algorithms in the current machine learning algorithms. It takes the decision tree algorithm as the core idea, adds the L2 regularization term design and supports the second-order Taylor expansion of the loss function, and further uses the second-order derivative information of the data to optimize the model parameters, so that it has very good generalization and fit in practical applications. At the same time, traditional machine learning algorithms such as LR regression and SVM are also used for effect comparison. As one of the classic machine learning algorithms, LR regression uses a calculation method similar to the input and output of a single-layer neural network. It can use the gradient update method and the cross entropy loss calculation method to update the calculation parameters. It also has good nonlinear expression ability, and can also introduce regularization terms to ensure the generalization of the model algorithm; the SVM algorithm also has certain advantages in the classification hyperplane calculation through linear kernel / nonlinear functions, and can obtain good results. Finally, the integration of multiple algorithms such as LR, SVM, XGBoost, RandomForest and Extra Tree can effectively reduce the model error and make its evaluation of the lens stress more accurate.
[0096] A lens zonule assessment system, Figure 1 The system structure diagram of the present invention is shown in the figure, including:
[0097] An imaging unit, used for collecting an eyeball image and eyeball parameters; the eyeball image is an image of the eyeball tissue deformed and displaced after the eyeball is subjected to force;
[0098] An optimization unit, used for optimizing the eyeball image;
[0099] A processing unit, configured to extract edges of the optimized eyeball image based on an image feature detection algorithm, and to obtain the deformation amplitude and / or displacement of the eyeball tissue after the eyeball is subjected to force from the eyeball image after edge extraction;
[0100] An analysis unit, used for inputting the eyeball parameters into a trained force analysis model to obtain eyeball force information; the force analysis model is obtained after training by the above system;
[0101] The evaluation unit is used to evaluate the lens suspensory ligament according to the deformation amplitude and / or displacement of the eyeball tissue after the eyeball is subjected to force, and the corresponding eyeball force information, so as to obtain the biomechanical state of the lens suspensory ligament.
[0102] Specifically, the existing technology cannot obtain the biomechanical state of the lens suspensory ligament, resulting in inaccurate diagnosis of diseases related to abnormal suspensory ligament. Inaccurate disease diagnosis may delay the patient's condition and miss the best treatment time, or even take the wrong treatment method, resulting in medical accidents. In order to avoid the occurrence of medical accidents and improve the accuracy of diagnosing diseases related to abnormal suspensory ligament, the system of this solution evaluates the lens suspensory ligament according to the deformation amplitude and / or displacement of the eyeball tissue and the corresponding eyeball force information. First, the eyeball image and eyeball parameters are collected; secondly, the eyeball image is optimized to improve the resolution and clarity of the eyeball image; then, the eyeball image is edge extracted and the deformation amplitude and / or displacement of the eyeball tissue is extracted based on the image feature detection algorithm; then, the training samples are collected to train the force analysis model established based on the machine learning algorithm; finally, the eyeball parameters are input into the trained force analysis model to obtain the eyeball force information, and the lens suspensory ligament is evaluated according to the deformation amplitude and / or displacement of the eyeball tissue and the corresponding eyeball force information. This solution not only improves the quality of eyeball images by optimizing eyeball images and extracting edges based on image feature detection algorithms, but also makes the subsequent acquisition of eyeball tissue deformation amplitude and displacement more accurate. In addition, a machine learning model is used to reduce the error of eyeball force information and further improve the accuracy of the biomechanical state of the lens suspensory ligament.
[0103] Furthermore, it also includes:
[0104] The calibration unit is used to calibrate the image collector for collecting eye images before the image unit collects eye images and eye parameters.
[0105] Specifically, the image collector includes: a tilt-shift camera and a data acquisition computer connected thereto; the calibration is camera calibration (camera calibration). The tilt-shift camera is used to collect eye image data; the data acquisition computer is used to calculate the camera calibration result. Whether in image measurement or machine vision applications, the calibration of camera parameters is a very critical link, and the accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera. Therefore, in order to correct the distortion at the image point caused by factors such as lens aberration, the camera calibration needs to be performed before collecting the eye image and eye parameters.
[0106] Specifically, the image collector includes: a tilt-shift camera and a data acquisition computer connected thereto; the calibration is camera calibration (camera calibration). The tilt-shift camera is used to collect eye image data; the data acquisition computer is used to calculate the camera calibration result. Whether in image measurement or machine vision applications, the calibration of camera parameters is a very critical link, and the accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera. Therefore, in order to correct the distortion at the image point caused by factors such as lens aberration, the camera calibration needs to be performed before collecting the eye image and eye parameters.
[0107] Furthermore, the calibration unit includes:
[0108] The initial module is used to establish the image distortion error model;
[0109] A coordinate module is used to collect the original grid image and establish a coordinate system using the longitudinal and transverse grid lines in the grid image as coordinate axes to obtain the original grid node coordinates;
[0110] A node module is used to fit the middle area of the original grid image to obtain a target grid image and target grid node coordinates;
[0111] An error module, used for comparing the original grid node coordinates with the target grid node coordinates to obtain the image distortion error of each grid node;
[0112] A parameter module, used for inputting the target grid node coordinates and the corresponding image distortion error into the image distortion error model to calculate the distortion parameter;
[0113] An output module, used to input the target pixel point coordinates and the distortion parameters into the image distortion error model to obtain the corresponding image distortion error and the original pixel point coordinates; the target pixel point coordinates are the coordinates of the pixel points of the target grid image; the original pixel point coordinates are the coordinates of the points corresponding to the pixel points of the target grid image on the original grid image;
[0114] The correction module is used to collect the grayscale value of the original pixel point coordinates / the grayscale value adjacent to the original pixel point coordinates, and assign the grayscale value to the corresponding target pixel point to obtain the target grid image after distortion correction.
[0115] The specific steps of camera calibration are as follows:
[0116] An image distortion error model is established, and the model is:
[0117]
[0118] Among them, a0~a9 and b0~b9 are distortion parameters.
[0119] Collecting the original grid image, and establishing a coordinate system with the longitudinal and transverse grid lines in the grid image as coordinate axes to obtain the original grid node coordinates; Figure 2 The schematic diagram of the orthogonal grid node of the present invention is shown in the figure. The acquisition of the original grid image is as follows: the tilt-shift camera shoots the high-precision standard orthogonal grid plate, and the grid fills the entire imaging surface during shooting. The vertical and horizontal grid lines in the grid image are used as coordinate axes to establish a coordinate system, and the coordinates of the original grid node are obtained as follows: the grid row 0, column 0 node is set as the reference point, and the reference point coordinates are (x 00 ,y 00 ), the spacing between the horizontal grid lines is Δx, the spacing between the vertical grid lines is Δy, the angle between the horizontal grid lines and the horizontal direction is α, and the grid node coordinates are:
[0120]
[0121] Where i is the number of node columns, j is the number of node rows, k = tanα, d0 = y 00 -kx 00 , c0=x 00 +ky 00 .
[0122] The middle area of the original grid image is fitted to obtain the target grid image and the target grid node coordinates. Since the aberration of the middle area of the eyeball is very small in actual shooting, this scheme uses the grid nodes within 1 / 3 of the middle area of the original grid image to fit the target grid image, that is, the node coordinates (x ij ,y ij ) into equation (1.2) to solve for the target grid image parameters k, d0, c0, Δx, Δy.
[0123] The original grid node coordinates are compared with the target grid node coordinates to obtain the image distortion error of each grid node; the image distortion error is
[0124] The target grid node coordinates and the corresponding image distortion error are input into the image distortion error model to calculate the distortion parameters; that is, (x ij ,y ij )as well as Input formula (1.1) and calculate the distortion parameters a0~a9, b0~b9.
[0125] Input the target pixel coordinates and the distortion parameters into the image distortion error model to obtain the corresponding image distortion error and original pixel coordinates; that is, input (x, y) and a0~a9, b0~b9 into formula (1.1) to obtain the image distortion of (x, y) and (x', y').
[0126] The grayscale values near the original pixel coordinates / the original pixel coordinates are collected, and the grayscale values are assigned to the corresponding target pixel points to obtain the target grid image after distortion correction; since (x', y') is usually not an integer, it is necessary to interpolate the grayscale of the pixels near the position (x', y') to obtain the grayscale value corresponding to (x', y'), and assign the grayscale value to the target pixel point (x, y), so as to perform distortion correction on the target grid image.
[0127] Furthermore, the optimization unit is used to optimize the eyeball image, including:
[0128] The optimization unit is used to optimize the eyeball image based on a maximum a posteriori probability estimation algorithm to obtain an optimized eyeball image.
[0129] Specifically, since excessive force can cause damage to the eyeball, the air pulse applied to the eyeball must have an appropriate force when collecting the eyeball image. Because the displacement of the lens caused by an appropriate force that does not cause damage to the eyeball is in the micron level, the collected eyeball image must have a sufficiently high resolution to accurately identify the corresponding information (lens displacement, etc.) from the image. The steps of optimizing the eyeball image based on the maximum a posteriori probability estimation algorithm and obtaining a high-resolution eyeball image are specifically as follows:
[0130] Based on the maximum a posteriori estimation method, the reconstruction model is constructed:
[0131]
[0132] Among them, l is a low-resolution and high-noise image of the eye, N is the number of image pixels, h is a high-resolution image of the eye, D and H are blur and downsampling matrices, is a certain regular transformation, and γ is the penalty coefficient.
[0133] In order to simultaneously perform noise reduction in high-resolution reconstruction of eye images, the alternating direction method of multiplier (ADMM) is substituted into formula (2.1):
[0134]
[0135] Among them, z is the auxiliary variable and μ is the regularization coefficient.
[0136] Decompose equation (2.2) and solve:
[0137]
[0138]
[0139] Formula (2.2.1) and formula (2.2.2) are solved by iterative formula (2.3.1) (2.3.2) (2.3.3); the iterative formula (2.5) (2.6) (2.7) is as follows:
[0140] h k+1 =(H T D T DH+μI) -1 (H T D T l k +μ(v k -z k )) (2.3.1)
[0141]
[0142] z k+1 =z k +(h k+1 -v k+1 ) (2.3.3)
[0143] in, It can be any available algorithm for solving Gaussian noise reduction.
[0144] Construct the fuzzy kernel function and solve the fuzzy matrix H;
[0145] The key to solving the fuzzy matrix H is to solve the fuzzy kernel function. Using the Gaussian point spread function (PSF) as the fuzzy kernel function can further reduce the content loss of the eye image. The Gaussian point spread function (PSF) is as follows:
[0146]
[0147] Among them, i, j represent the coordinates in the 2D coordinate system, s 2 is the variance, M is the normalization coefficient, and s is the parameter to be solved.
[0148] The statistical non-Bayesian minimization based on Stein unbiased risk estimation (SURE) is used to accurately estimate the PSF parameters while taking into account the downsampling factor. The blur-SURE method improved by the downsampling process can obtain the blur-MSE–SR standard, which is as follows:
[0149]
[0150] Among them, H0 is the unknown fuzzy matrix to be solved, W H,ρ =(H H H+ρC) -1 H T , C is a eigenvalue with ||ω|| 2 And the circular matrix ω=(ω1,ω2), C determines the spatial frequency. Through the Gaussian noise statistical method, the following formula is obtained:
[0151]
[0152] Solve the minimum value of formula (2.6) to obtain the high-resolution image h of the eyeball.
[0153] The minimization of two scalar variables (PSF parameter s and regularization parameter ρ) can be determined by exhaustive method within a certain range. The amount of relevant calculations can be further reduced by exhaustive method after Fourier transformation.
[0154] Furthermore, the image feature detection algorithm is a canny edge detection algorithm.
[0155] Furthermore, the processing unit includes:
[0156] A smoothing processing module, used for smoothing the eyeball image using a Gaussian filter;
[0157] An extraction module, used for performing edge extraction on the smoothed high-resolution eyeball image using a canny edge detection algorithm to obtain the edge of the eyeball tissue; the eyeball tissue is the lens and cornea;
[0158] A displacement module is used to calculate the displacement of the lens after the eyeball is subjected to force according to the edge of the lens;
[0159] The deformation module is used to calculate the deformation amplitude of the cornea after the eyeball is subjected to force according to the edge of the cornea.
[0160] Specifically, the eye image is first smoothed using a Gaussian filter with a standard deviation of σ to reduce noise and obtain a clear eye image. Then, the local gradient of each point in the image is calculated. and edge direction tan -1 (g x / g y). Then, the amplitude is non-maximum suppressed along the gradient direction to refine the edge of the eye tissue, retain the local maximum gradient and suppress other gradient values to obtain a fine edge of the eye tissue. Finally, the displacement of the lens △d1 is obtained according to the difference between the highest and lowest points of the center vertex of the lens edge during the movement process. The deformation amplitude of the cornea △d2 is obtained according to the difference between the highest and lowest points of the center vertex of the cornea during the movement process and the displacement of the entire cornea is subtracted.
[0161] Furthermore, the processing unit further includes:
[0162] The threshold module is used to set a grayscale threshold for the eyeball image to distinguish the cornea from the lens.
[0163] Furthermore, the evaluation unit comprises:
[0164] An eyeball analysis module is used to obtain the force magnitude of the lens from the eyeball force information;
[0165] A lens analysis module, used for calculating the ratio of the lens force to the lens displacement as a first ratio;
[0166] A comparison module, used for calculating the ratio of the lens displacement to the corneal deformation amplitude as a second ratio;
[0167] A calculation module is used to evaluate the biomechanical state of the lens zonules according to the lens force magnitude and the first ratio and / or the second ratio.
[0168] Figure 3 It is a schematic diagram of the cross-section of the eye before and after the force is applied. As shown in the figure, the force F applied to the lens is obtained from the force information of the eyeball; the ratio of the force applied to the lens to the displacement of the lens F / △d1 is calculated, which can reflect the biomechanical state of the lens suspensory ligament; in addition, the ratio of the displacement of the lens to the deformation amplitude of the cornea △d1 / △d2 is calculated, which can also reflect the biomechanical state of the lens suspensory ligament.
[0169] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.
Claims
1. A lens zonule assessment system, characterized in that: include: An imaging unit, used for collecting an eyeball image and eyeball parameters; the eyeball image is an image of the eyeball tissue deformed and displaced after the eyeball is subjected to force; An optimization unit, used for optimizing the eyeball image; A processing unit, configured to extract edges of the optimized eyeball image based on an image feature detection algorithm, and to obtain the deformation amplitude and / or displacement of the eyeball tissue after the eyeball is subjected to force from the eyeball image after edge extraction; An analysis unit, used for inputting the eyeball parameters into a trained force analysis model to obtain eyeball force information; An evaluation unit, used to evaluate the suspensory ligament of the lens according to the deformation amplitude and / or displacement of the eyeball tissue after the eyeball is subjected to force, and the corresponding eyeball force information, so as to obtain the biomechanical state of the suspensory ligament of the lens; Also includes: A calibration unit, used for calibrating an image collector for collecting eyeball images before the image unit collects eyeball images and eyeball parameters; The calibration unit comprises: The initial module is used to establish the image distortion error model; A coordinate module is used to collect the original grid image and establish a coordinate system using the longitudinal and transverse grid lines in the grid image as coordinate axes to obtain the original grid node coordinates; A node module is used to fit the middle area of the original grid image to obtain a target grid image and target grid node coordinates; An error module, used for comparing the original grid node coordinates with the target grid node coordinates to obtain the image distortion error of each grid node; A parameter module, used for inputting the target grid node coordinates and the corresponding image distortion error into the image distortion error model to calculate the distortion parameter; An output module, used to input the target pixel point coordinates and the distortion parameters into the image distortion error model to obtain the corresponding image distortion error and the original pixel point coordinates; the target pixel point coordinates are the coordinates of the pixel points of the target grid image; the original pixel point coordinates are the coordinates of the points corresponding to the pixel points of the target grid image on the original grid image; The correction module is used to collect the grayscale value of the original pixel point coordinates / the grayscale value adjacent to the original pixel point coordinates, and assign the grayscale value to the corresponding target pixel point to obtain the target grid image after distortion correction.
2. A lens zonule assessment system according to claim 1, characterized in that: The stress analysis model is obtained by training a machine learning model training system, and the machine learning model training system includes: A collection unit, used for collecting training samples; the training samples include: eyeball parameters and eyeball force information corresponding to the eyeball parameters; the eyeball parameters include: corneal biomechanics, corneal thickness, intraocular pressure and anterior chamber depth; A training unit is used to establish a force analysis model based on a machine learning algorithm, and to train the force analysis model using the training samples.
3. A lens zonule assessment system according to claim 2, characterized in that: The machine learning algorithms include: LR regression algorithm, SVM algorithm, XGBoost algorithm, Random Forest algorithm and Extra Tree algorithm.
4. A lens zonule assessment system according to claim 1, characterized in that: The optimization unit is used to optimize the eyeball image, including: The optimization unit is used to optimize the eyeball image based on a maximum a posteriori probability estimation algorithm to obtain an optimized eyeball image.
5. The lens zonule assessment system according to claim 1, characterized in that: The image feature detection algorithm is a canny edge detection algorithm.
6. A lens zonule assessment system according to claim 5, characterized in that: The processing unit comprises: A smoothing processing module, used for smoothing the eyeball image using a Gaussian filter; An extraction module, used for performing edge extraction on the smoothed high-resolution eyeball image using a canny edge detection algorithm to obtain the edge of the eyeball tissue; the eyeball tissue is the lens and cornea; A displacement module is used to calculate the displacement of the lens after the eyeball is subjected to force according to the edge of the lens; The deformation module is used to calculate the deformation amplitude of the cornea after the eyeball is subjected to force according to the edge of the cornea.
7. A lens zonule assessment system according to claim 6, characterized in that: The processing unit also includes: The threshold module is used to set a grayscale threshold for the eyeball image to distinguish the cornea from the lens.
8. A lens zonule assessment system according to claim 7, characterized in that: The evaluation unit comprises: An eyeball analysis module is used to obtain the force magnitude of the lens from the eyeball force information; A lens analysis module, used for calculating the ratio of the lens force to the lens displacement as a first ratio; A comparison module, used for calculating the ratio of the lens displacement to the corneal deformation amplitude as a second ratio; A calculation module is used to evaluate the biomechanical state of the lens zonules according to the lens force magnitude and the first ratio and / or the second ratio.
