A method for quantifying the vault height after intraocular lens implantation

Abstract

The invention discloses a method for quantifying the postoperative arch height after intraocular lens implantation. The method comprises the following steps: firstly obtaining an anterior segment image to be measured; then determining the edge position information of each tissue structure in the anterior segment image to obtain an anterior segment structure edge image; then performing polynomial fitting processing on the edges of each tissue structure in the anterior segment structure edge image to obtain corneal anterior and posterior surface curves, intraocular lens anterior and posterior surface curves, and natural lens anterior surface curve; based on the intraocular lens posterior surface curve and the natural lens anterior surface curve, calculating an arch height distribution curve, integrating the arch height distribution curve over a predetermined area to calculate an arch height area; based on the arch height area and the integrated area, calculating a standardized arch height value; and based on the corneal posterior surface curve, the intraocular lens anterior surface curve, the intraocular lens posterior surface curve, and the natural lens anterior surface curve, calculating the ratio of the natural lens anterior chamber depth to the intraocular lens anterior chamber depth.

Description

Technical Field

[0001] The present invention relates to the technical field of ophthalmic clinical medical diagnosis, and in particular to a method for quantifying the arch height after artificial lens implantation, a computing device and a storage medium. Background Art

[0002] ICL is an implantable contact lens. ICL implantation surgery is different from corneal refractive surgery. It does not require corneal cutting. Instead, an artificial lens is implanted in the posterior chamber of the patient's eyeball through minimally invasive surgery to maintain the natural adjustment function of the lens and correct refractive errors at the same time. Its advantages are good predictability and reversibility. The surgical process is to implant an artificial lens into the eye through a tiny incision to achieve the purpose of correcting vision. ICL surgery is another innovative scientific and technological achievement in the field of modern ophthalmic refractive correction, especially in the correction of extreme myopia, it has an effect that traditional excimer laser surgery cannot match.

[0003] The measurement of the arch height (from the posterior surface of the intraocular lens to the anterior surface of the crystalline lens) after ICL surgery is a key objective indicator for judging the safety and effectiveness of the surgery. Most current technologies use anterior segment OCT images or Scheimpflug images to manually measure the pixel distance value in the middle position, and then convert the pixel size into an arch height value in μm. This measurement method has many errors: 1. It ignores the distortion caused by the optical path, that is, the pixel size at different positions of the image is inconsistent 2. There is uncertainty in the selection of the manual measurement boundary 3. Since both the intraocular lens and the natural lens are inside the eye (behind the cornea), the refractive effect of the cornea is ignored. In addition, the measurement results are single, only measuring the arch height of the central area and ignoring the surrounding distribution, resulting in instability in the analysis of the measurement results.

[0004] Therefore, it is necessary to provide a method for measuring the arch height after intraocular lens implantation, which can reduce the measurement error and the instability of the single central arch height measurement result, so as to solve the problems existing in the above prior art. Summary of the invention

[0005] In view of the above problems, the present invention is proposed to provide a method, a computing device and a storage medium for quantifying the arch height after intraocular lens implantation, which overcome the above problems or at least partially solve the above problems.

[0006] According to one aspect of the present invention, a method for quantifying the arch height after intraocular lens implantation is provided, in which the following steps are firstly to obtain an anterior segment image to be measured; then, the edge position information of the cornea, iris, intraocular lens and natural lens in the anterior segment image is determined to obtain an anterior segment structure edge image; then, polynomial fitting processing is performed on the edges of the cornea, iris, intraocular lens and natural lens in the anterior segment structure edge image to obtain a corneal anterior surface curve, a corneal posterior surface curve, an intraocular lens anterior surface curve, an intraocular lens posterior surface curve and a natural lens anterior surface curve; based on the intraocular lens posterior surface curve and the natural lens anterior surface curve, an arch height distribution curve is calculated, and the arch height distribution curve is integrated over a predetermined area to calculate an arch height area; based on the arch height area and the integrated area, a standardized arch height value is calculated; based on the corneal posterior surface curve, the intraocular lens anterior surface curve, the intraocular lens posterior surface curve and the natural lens anterior surface curve, the ratio of the natural lens anterior chamber depth to the intraocular lens anterior chamber depth is calculated.

[0007] This method quantifies the vault distribution curve, vault area, standardized vault value and the ratio of the anterior chamber depth of the natural lens to the anterior chamber depth of the artificial lens under the action of the sigmoid function to compensate for the instability caused by the single central vault measurement, which can provide more accurate and complete statistical characteristics for clinical practice.

[0008] Optionally, in the above method, the acquired anterior segment image may be a Scheimpflug anterior segment image acquired by a Scheimpflug camera.

[0009] Optionally, in the above method, based on the posterior surface curve of the cornea, the anterior surface curve of the artificial lens, and the posterior surface curve of the artificial lens, the first anterior chamber depth from the geometric center of the artificial lens on the central axis to the posterior surface of the cornea is calculated; based on the posterior surface curve of the cornea and the anterior surface curve of the natural lens, the second anterior chamber depth from the anterior surface of the natural lens on the central axis to the posterior surface of the cornea is calculated; based on the first anterior chamber depth and the second anterior chamber depth, the sigmoid function is used to calculate the ratio of the anterior chamber depth of the natural lens to the anterior chamber depth of the artificial lens.

[0010] Optionally, in the above method, the anterior segment image to be measured is regionally segmented based on a deep convolutional segmentation network model to obtain a binary image of the anterior segment; edge extraction is performed on the binary image of the anterior segment to obtain an anterior segment structure edge image; coordinate transformation is performed on each pixel of the anterior segment structure edge image based on a homography matrix to obtain a distortion-corrected anterior segment structure edge image; and according to the law of refraction, the positions of virtual image edge points in the distortion-corrected anterior segment structure edge image are restored to real image edge points to obtain a refraction-corrected anterior segment structure edge image.

[0011] Optionally, in the above method, a standard checkerboard plate is placed on the surface of the anterior segment structure edge image to obtain a checkerboard plate image; the corner points of the checkerboard in the checkerboard plate image are detected, and the upper left corner of the image is used as the image coordinate origin to obtain the position coordinates of each corner point; the center position of the checkerboard plate is used as the actual coordinate origin, and the homography matrix is ​​calculated in combination with the actual size of the checkerboard plate and the obtained position coordinates of the corner points; each pixel of the checkerboard plate image is applied to the homography matrix to obtain the distortion-corrected anterior segment structure edge image.

[0012] This scheme is based on Scheimpflug images and uses a deep convolutional segmentation network to first segment the image into regions of various tissue structures. Then, the spatial positions of the intraocular lens and the original lens are accurately located through the homography matrix method and the refraction return algorithm, which can reduce the subsequent measurement error of the arch height.

[0013] Optionally, in the above method, first, a predetermined number of anterior segment images are obtained as training images; then, the cornea, iris, intraocular lens, natural lens and background areas in the training images are manually annotated to obtain a data set containing label data; the data set is divided into a training set, a validation set and a test set according to a predetermined ratio; based on the cross-validation method, the parameters and hyperparameters of the training set and validation set are used to train the model to obtain a trained deep convolutional segmentation network model. Finally, the anterior segment image to be measured is input into the trained deep convolutional segmentation network model for regional segmentation to obtain an anterior segment binary image containing the cornea area, the iris area, the intraocular lens area and the natural lens area.

[0014] Optionally, in the above method, data enhancement processing is performed on the training data in the training set, and the data enhancement processing includes flipping, translation, scale transformation, contrast transformation and Gaussian noise processing; the training set after data enhancement processing is input into the deep convolutional segmentation network model for training, and when the loss value of the set similarity metric loss function is less than a first predetermined threshold, the trained deep convolutional segmentation network model is obtained; the validation set is used to adjust the hyperparameters of the trained deep convolutional segmentation network model, and the hyperparameters include learning rate and number of iterations. When the loss value of the cross entropy loss function is less than a second predetermined threshold, the trained deep convolutional segmentation network model is obtained; the test set is used to evaluate the error of the trained deep convolutional segmentation network model.

[0015] Optionally, in the above method, the deep convolutional segmentation network model includes an input layer, a DenseUnet backbone network and an output layer, the output layer is a softmax layer, which is used to output classification results of multiple organizational structures, the DenseUnet backbone network includes a cascaded first convolutional layer, a first dense block, a first down-sampling unit, a second dense block, a second down-sampling unit, a third dense block, a third down-sampling unit, a fourth dense block, a first up-sampling unit, a fifth dense block, a second up-sampling unit, a sixth dense block, a third up-sampling unit, a sixth dense block, a second convolutional layer, and a third convolutional layer. The backbone network also includes a feature fusion module, the feature fusion module includes a first fusion unit, a second fusion unit and a third fusion unit, the first fusion unit is used to merge the feature map output by the third down-sampling unit with the feature map output by the first up-sampling unit through channels, the second fusion unit is suitable for merging the feature map output by the second down-sampling unit with the feature map output by the second up-sampling unit through channels, and the third fusion unit is suitable for merging the feature map output by the first down-sampling unit with the feature map output by the third up-sampling unit through channels.

[0016] According to another aspect of the present invention, there is provided a computing device, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be suitable for being executed by the at least one processor, and the program instructions include instructions for executing the above method.

[0017] According to another aspect of the present invention, a readable storage medium storing program instructions is provided. When the program instructions are read and executed by a computing device, the computing device executes the above method.

[0018] According to the solution of the present invention, the anterior segment image is segmented into various tissue structure areas through the improved deep convolution segmentation network, and then the segmented image is corrected for distortion through the homography matrix method, and the spatial positions of the artificial lens and the original lens are accurately located through the refraction return algorithm to reduce the error of subsequent arch height measurement; and the arch height distribution curve, arch height area, standardized arch height value and the ratio of the anterior chamber depth of the lens to the anterior chamber depth of the artificial lens under the action of the sigmoid function are quantified, thereby compensating for the instability caused by the single central arch height measurement and providing more accurate and complete statistical features for clinical practice.

[0019] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0021] Figure 1 A block diagram of a computing device 100 according to an embodiment of the present invention is shown;

[0022] Figure 2 A schematic flow chart of a method 200 for quantifying the vault height after intraocular lens implantation according to an embodiment of the present invention is shown;

[0023] Figure 3 A schematic diagram of a Scheimpflug anterior segment image according to an embodiment of the present invention is shown;

[0024] Figure 4 A schematic diagram of the structure of a deep convolutional segmentation network model according to an embodiment of the present invention is shown;

[0025] Figure 5 An image of the edge of the anterior segment structure according to an embodiment of the present invention is shown;

[0026] Figure 6 A schematic diagram of image distortion correction of anterior segment structure edge according to an embodiment of the present invention is shown;

[0027] Figure 7 shows an edge image of the anterior segment structure after distortion correction according to an embodiment of the present invention;

[0028] Figure 8 A schematic diagram showing the relationship between the real image position and the virtual image position of a point on the posterior surface of the cornea according to an embodiment of the present invention is shown;

[0029] Fig. 9 A diagram showing the edge of the anterior segment structure after refraction correction according to one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0030] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0031] Vault height is the distance between the back surface of the ICL lens and the front surface of the natural lens. The size of the vault height is an important indicator for observation after ICL surgery. Before surgery, the depth of the anterior chamber should be checked routinely to ensure that there is a suitable vault height after surgery. Under normal circumstances, the ideal vault height after ICL implantation is 250-750um. If the vault height is too low after surgery, it is easy to cause the position of the ICL artificial lens to move, causing glare, poor vision after surgery, etc. If the vault height after ICL surgery is too high, the anterior chamber will be crowded, the angle of the chamber will be narrow or closed, and the aqueous humor will not flow smoothly, leading to secondary glaucoma, etc. In order to improve the accuracy and stability of vault height measurement, this scheme uses a deep convolutional segmentation network model to first perform tissue structure segmentation on the anterior segment image to be measured, and then accurately locates the spatial position of the intraocular lens and the natural lens through the homography matrix method and the refraction return algorithm; then proposes the vault height distribution curve, vault height area, standardized vault height value and S_ACDR (ratio of the anterior chamber depth of the lens to the anterior chamber depth of the intraocular lens under the action of the sigmoid function) features to compensate for the instability caused by the single central vault height measurement, providing more accurate, sensitive and complete statistical features for clinical practice.

[0032] Figure 1 FIG. 1 shows a block diagram of a computing device 100 according to an embodiment of the present invention. Figure 1 As shown, in a basic configuration 102, computing device 100 typically includes system memory 106 and one or more processors 104. A memory bus 108 may be used for communication between processor 104 and system memory 106.

[0033] Depending on the desired configuration, the processor 104 can be any type of processor, including but not limited to: a microprocessor (μP), a microcontroller (μC), a digital information processor (DSP), or any combination thereof. The processor 104 can include one or more levels of cache such as a first-level cache 110 and a second-level cache 112, a processor core 114, and registers 116. An example processor core 114 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 118 can be used with the processor 104, or in some implementations, the memory controller 118 can be an internal part of the processor 104.

[0034] Depending on the desired configuration, the system memory 106 can be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. The physical memory in the computing device generally refers to the volatile memory RAM, and the data in the disk needs to be loaded into the physical memory before it can be read by the processor 104. The system memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some embodiments, the application 122 may be arranged to execute instructions on the operating system by one or more processors 104 using program data 124. The operating system 120 may be, for example, Linux, Windows, etc., which includes program instructions for processing basic system services and performing tasks that depend on hardware. The application 122 includes program instructions for implementing various user-desired functions, and the application 122 may be, for example, a browser, instant messaging software, a software development tool (such as an integrated development environment IDE, a compiler, etc.), etc., but is not limited thereto. When the application 122 is installed in the computing device 100, a driver module may be added to the operating system 120.

[0035] When the computing device 100 starts running, the processor 104 reads and executes the program instructions of the operating system 120 from the memory 106. The application 122 runs on the operating system 120 and uses the interfaces provided by the operating system 120 and the underlying hardware to implement various functions desired by the user. When the user starts the application 122, the application 122 is loaded into the memory 106, and the processor 104 reads and executes the program instructions of the application 122 from the memory 106.

[0036] Computing device 100 also includes a storage device 132 including a removable storage 136 and a non-removable storage 138 , both of which are connected to storage interface bus 134 .

[0037] The computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to the basic configuration 102 via the bus / interface controller 130. Example output devices 142 include a graphics processing unit 148 and an audio processing unit 150. They can be configured to facilitate communication with various external devices such as a display or speakers via one or more A / V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices) or other peripherals (e.g., printers, scanners, etc.) via one or more I / O ports 158. Example communication devices 146 may include a network controller 160, which may be arranged to facilitate communication with one or more other computing devices 162 via a network communication link via one or more communication ports 164.

[0038] A network communication link can be an example of a communication medium. The communication medium can generally be embodied as a computer-readable instruction, data structure, program module in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A "modulated data signal" can be a signal in which one or more of its data sets or its changes can be performed in a manner that encodes information in the signal. As a non-limiting example, the communication medium may include a wired medium such as a wired network or a dedicated network, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR) or other wireless media. The term computer-readable medium used here may include both storage media and communication media. In the computing device 100 according to the present invention, the application 122 includes instructions for executing the method 200 for quantifying the arch height after intraocular lens implantation of the present invention.

[0039] Figure 2 FIG. 2 is a flow chart of a method 200 for quantifying the vault height after intraocular lens implantation according to an embodiment of the present invention. Figure 2 As shown, the method 200 begins with step S210, where an anterior segment image to be measured is acquired. In an embodiment of the present invention, a Scheimpflug anterior segment image may be acquired based on a Scheimpflug camera system. Figure 3 FIG. 4 shows a schematic diagram of a Scheimpflug anterior segment image according to an embodiment of the present invention. Figure 3 As shown, the Scheimpflug anterior segment image can display a clear anterior chamber structure, thereby obtaining biological structural parameters from the anterior surface of the human cornea to the posterior surface of the lens.

[0040] In order to measure the arch height, it is necessary to perform regional segmentation on the acquired anterior segment image, so as to determine the edge position information of the cornea, iris, artificial lens and natural lens in the anterior segment image. Then, step S220 is performed to determine the edge position information of the cornea, iris, artificial lens and natural lens in the anterior segment image, and obtain the edge image of the anterior segment structure. In an embodiment of the present invention, the anterior segment image to be measured can be regionally segmented based on a deep convolutional segmentation network model to obtain an anterior segment binary image to improve the accuracy and efficiency of tissue region segmentation, and then the edge extraction operator is used to extract the edges of each segmented tissue region. In the process of obtaining the anterior segment binary image, a deep convolutional segmentation network model can be first constructed. The backbone network of the model adopts a DenseUnet network. In order to predict the four tissue regions of the cornea, iris, artificial lens and natural lens and a background region, a softmax layer is used as the output layer to classify the predicted regions. Figure 4 FIG. 4 shows a schematic diagram of the structure of a deep convolutional segmentation network model according to an embodiment of the present invention. Figure 4 As shown, the deep convolutional segmentation network model includes an input layer, a DenseUnet backbone network and an output layer, the output layer is a softmax layer, which is used to output classification results of multiple organizational structures, and the DenseUnet backbone network includes a cascaded first convolutional layer, a first dense block, a first down-sampling unit, a second dense block, a second down-sampling unit, a third dense block, a third down-sampling unit, a fourth dense block, a first up-sampling unit, a fifth dense block, a second up-sampling unit, a sixth dense block, a third up-sampling unit, a sixth dense block, a second convolutional layer, and a third convolutional layer. The backbone network also includes a feature fusion module, which includes a first fusion unit, a second fusion unit and a third fusion unit. The first fusion unit is used to merge the feature map output by the third down-sampling unit with the feature map output by the first up-sampling unit through channels, and the second fusion unit is suitable for merging the feature map output by the second down-sampling unit with the feature map output by the second up-sampling unit through channels, and the third fusion unit is suitable for merging the feature map output by the first down-sampling unit with the feature map output by the third up-sampling unit through channels.

[0041] The input layer image size of the network model is 640x640x3, that is, a three-channel anterior segment image with a resolution of 640x640. After a 3*3 convolution layer, the input image features are roughly extracted and then input into the DenseNet backbone network. DenseNet focuses on the reuse of feature maps of each layer. First, a dense block is used to increase the number of channels of the image. For example, two convolution blocks with an output channel number of 10 are defined in a dense block. When an input with a channel number of 3 is used, an output with a channel number of 3+2x10=23 will be obtained. Then, through the first downsampling->second dense block->second downsampling->third dense block->third downsampling->fourth dense block->first upsampling->fifth dense block->second upsampling->sixth dense block->third upsampling, the encoding and decoding of related parameters are realized. Each convolution layer in a dense block uses the same number of output channels. In each dense block, there is a direct connection between any two layers, that is, the input of each layer of the network is the concatenation of the output of all previous layers, and the feature map learned by this layer will also be directly passed to all subsequent layers as input. Through dense connections, the gradient disappearance problem can be alleviated, feature propagation can be strengthened, feature reuse is encouraged, and the number of parameters is greatly reduced. Each downsampling unit includes two 3*3 convolutional layers, a 2*2 maximum pooling layer and a dropout layer. After each downsampling operation, the number of channels doubles. The pooling layer is used to reduce the dimension of the feature and retain effective information. The maximum pooling layer is connected to a Dropout layer to prevent overfitting and improve the generalization ability of the model. The upsampling unit adopts an interpolation method, and inserts new elements between pixels using a suitable interpolation algorithm based on the original image pixels, thereby achieving image magnification. Each upsampling unit includes a 2*2 convolution layer, two 3*3 convolution layers and a dropout layer. Each fusion unit in the feature fusion module is a concatenation layer, which is used to merge the channels of the high-level features output by the upsampling unit and the low-level features output by the downsampling unit. In one embodiment of the present invention, the backbone network also includes multiple Bottleneck structures, which are arranged at the input and output ends of the dense block in order to reduce the number of parameters. The Bottleneck hourglass structure first passes through a 1x1 convolution kernel, then a 3x3 convolution kernel, and finally a 1x1 convolution kernel. After multiple downsampling and upsampling and feature fusion, it is connected to the sotfmax output layer through two convolution layers, and the image size of the output layer is 640x640x5.

[0042] After the deep convolutional segmentation network model is built, it can be trained. First, a large number of anterior segment images are obtained as training images. The image quality can be enhanced while ensuring that the image detail texture features are retained. For example, adaptive local mean filtering can be used to denoise the image; contrast-limited adaptive histogram equalization can be used to improve the contrast. Then the training images are annotated, including removing privacy information and medical imaging instrument markers, and manually annotating the cornea, iris, artificial lens, natural lens and other structural tissues and background areas under the guidance of a physician. For example, the cornea area is identified as category 1, the iris area is identified as category 2, the artificial lens area is identified as category 3, the lens area is identified as category 4, and the remaining areas are identified as background areas as category 0, to obtain a data set containing labeled data. The data set can be normalized, for example, the image scale is normalized, the image resolution is unified to 640*640, and the labeled data is binarized to normalize the labeled data to the [0,1] interval. Next, the data set is divided into a training set, a validation set, and a test set according to a predetermined ratio, for example, a ratio of 6:2:2, or a ratio of 7:2:1, etc. For example, the data set can be first divided into a training set and a test set, and then a validation set can be divided from the training set again.

[0043] Then, based on the cross-validation method, the parameters and hyperparameters of the training model can be used to train the training set and the validation set to obtain a trained deep convolutional segmentation network model. In one embodiment of the present invention, the training data in the training set is first subjected to data enhancement processing, including flipping, translation, scale transformation, contrast transformation and Gaussian noise processing. The embodiment of the present invention selects the sum of the cross entropy loss function and the Diceloss (set similarity metric) loss function as the multi-classification supervision loss function. During the training process, it is necessary to continuously reduce the two loss functions of the training set and the validation set. The training set after data enhancement processing is input into the deep convolutional segmentation network model for training. When the loss value of the set similarity metric loss function is less than the first predetermined threshold, the trained deep convolutional segmentation network model is obtained. During the training process, the model parameters are continuously adjusted, such as: coefficients in linear regression or logistic regression, support vectors in support vector machines, weights in neural networks, etc. The validation set is used to adjust the hyperparameters of the trained deep convolutional segmentation network model, including the learning rate and the number of iterations. When the loss value of the cross entropy loss function is less than the second predetermined threshold, the trained deep convolutional segmentation network model is obtained; finally, the test set is used to evaluate the generalization ability of the trained deep convolutional segmentation network model. The role of the cross-validation method is to try to use different training set / validation set divisions to perform multiple groups of different training / evaluations on the model. For example, the training set can be divided into k parts to train the model separately. After k training times, k different models are obtained. Then the validation set is used to evaluate the effect of the k models, and the best hyperparameters are selected from them. Using the optimal hyperparameters, the model is retrained using all training sets to obtain the final required model. The anterior segment image to be measured in the test set is input into the trained deep convolutional segmentation network model for regional segmentation to obtain an anterior segment binary image containing the cornea area, iris area, artificial lens area and natural lens area.

[0044] Subsequently, edge extraction is performed on the anterior segment binarized image to obtain an anterior segment structure edge image, for example, edge extraction is performed using Laplacian, Prewitt, Sobel, Laplace, canny operators, etc. The embodiment of the present invention uses the canny operator to grayscale and Gaussian filter the anterior segment binarized image, and uses a double threshold algorithm to detect and connect edges. Figure 5 FIG. 4 shows an image of the edge of the anterior segment structure according to an embodiment of the present invention. Figure 5 As shown, the edges of the cornea area, iris area, artificial lens area and natural lens area are extracted respectively through the deep convolutional segmentation network, thereby preparing for the subsequent arch height measurement.

[0045] The Scheimpflug image has the characteristics of large depth of field and clear imaging, but due to the particularity of the optical path design, the distortion is serious, and effective positive means are needed to eliminate these distortions. Based on the homography matrix, each pixel of the anterior segment structure segmentation map can be perspective transformed to obtain the anterior segment structure edge image after distortion correction. Specifically, a standard checkerboard grid plate is placed on the surface of the anterior segment structure edge image to obtain a checkerboard plate image; the corner points of the checkerboard in the checkerboard plate image (the vertices of the four corners of the checkerboard) are detected, and the upper left corner of the image can be used as the image coordinate origin to obtain the position coordinates of each corner point. Taking the center position of the checkerboard plate as the actual coordinate origin, combined with the actual size of the checkerboard plate and the position coordinates of the corner points obtained, the homography matrix H is calculated. It reflects the position mapping relationship between image coordinates and actual coordinates.

[0046] Figure 6 A schematic diagram of the distortion correction of the edge image of the anterior segment structure according to an embodiment of the present invention is shown. Each pixel of the grid plate image is applied to the homography matrix, that is, coordinate transformation is performed: Among them, (x, y) is the pixel coordinate before correction, (x', y') is the pixel coordinate after correction, and finally the edge image of the anterior segment structure after distortion correction is obtained. Figure 7 FIG. 4 shows an edge map of the anterior segment structure after distortion correction according to an embodiment of the present invention.

[0047] Due to the differences in morphology and refractive index of the anterior segment of the eye, it is generally believed that the refractive index of light in air is 1, the refractive index of the cornea is 1.376, and the refractive index of the aqueous humor is 1.336. Except for the anterior surface structure of the cornea, the cross-sectional imaging of other tissue structures is a virtual image captured by the camera. Therefore, it is also necessary to perform refraction correction on the edge image of the anterior segment structure after distortion correction. Figure 8 FIG. 1 is a schematic diagram showing the relationship between the real image position and the virtual image position of a point on the posterior surface of the cornea according to an embodiment of the present invention. Figure 8 As shown in the figure, o is the position of the principal point of the sheimpflug lens. Taking a certain point p on the posterior surface of the cornea as an example, the relationship between the real image position p and the virtual image position v can be obtained according to the law of refraction. The other tissue structure points except the structure points on the anterior surface of the cornea are restored to the real image position point by point to achieve the purpose of refraction correction, and finally the edge image of the anterior segment structure after refraction correction is obtained. Fig. 9 A diagram showing the edge of the anterior segment structure after refraction correction according to one embodiment of the present invention is shown.

[0048] Then, step S230 can be executed to perform polynomial fitting processing on the edges of the cornea, iris, artificial lens and natural lens in the edge image of the anterior segment structure, and obtain the corneal anterior surface curve, corneal posterior surface curve, artificial lens anterior surface curve, artificial lens posterior surface curve and natural lens anterior surface curve. The anterior segment structure edge after refraction correction is divided into categories, and polynomial fitting processing is performed on each structure curve, that is, the expression of the function y=f(x) is predicted from a set of coordinate data (xi, yi), i=1,2,3....n, so as to fit the edge curves of each tissue structure, namely, the anterior surface of the cornea f1(x), the posterior surface of the cornea f2(x), the anterior surface of the artificial lens f3(x), the posterior surface of the artificial lens f4(x), and the anterior surface of the lens f5(x). The above structure curves are all in the same spatial coordinate system, and according to the actual human eye characteristics, the coordinate value of x is defined as [-6,6]mm.

[0049] Then, step S240 is performed to calculate the arch height distribution curve based on the posterior surface curve of the artificial lens and the anterior surface curve of the natural lens, and the arch height distribution curve is integrated over a predetermined area to calculate the arch height area. The traditional arch height measurement index is the central arch height h = f4(x)-f5(x) (x = 0). In order to improve the stability and sensitivity of the arch height feature in clinical applications, in addition to obtaining the arch height distribution curve f(x) = f4(x)-f5(x), this solution also integrates the arch height distribution curve over a predetermined area to calculate the arch height area. The value of a is determined by the clinical interest region, and a<6, that is, less than the maximum value of x set above.

[0050] Then, step S250 is performed to calculate the standardized arch height value based on the arch height area and the integral area. The standardized arch height value can provide a basis for judging the arch height value measurement result. The standardized arch height value hs=s / (2*a), where s is the arch height area and a is the abscissa value of the integral area.

[0051] Finally, step S260 is performed to calculate the ratio of the anterior chamber depth of the natural lens to the anterior chamber depth of the artificial lens based on the posterior surface curve of the cornea, the anterior surface curve of the artificial lens, the posterior surface curve of the artificial lens, and the anterior surface curve of the natural lens. In an embodiment of the present invention, the first anterior chamber depth from the geometric center of the artificial lens on the central axis to the posterior surface of the cornea is calculated based on the posterior surface curve of the cornea, the anterior surface curve of the artificial lens, and the posterior surface curve of the artificial lens. That is, the anterior chamber depth d1 from the geometric center of the artificial lens on the central axis at x=0 to the posterior surface of the cornea is first calculated, d1=f2(x)-(f3(x)+f4(x)) / 2, (x=0).

[0052] Then, based on the curve of the corneal posterior surface and the curve of the natural lens anterior surface, the second anterior chamber depth from the natural lens anterior surface to the corneal posterior surface on the central axis is calculated, that is, the anterior chamber depth d2 from the natural lens anterior surface to the corneal posterior surface on the central axis at x=0 is calculated, d2=f2(x)-f5(x), (x=0).

[0053] Finally, based on the first anterior chamber depth and the second anterior chamber depth, the sigmoid function is used to calculate the ratio of the natural lens anterior chamber depth to the artificial lens anterior chamber depth. Since the ratio of d2 to d1 is usually greater than 1 and close to 1, the sigmoid inverse function is introduced to improve the feature sensitivity, thereby obtaining the ratio of the natural lens anterior chamber depth to the artificial lens anterior chamber depth S_ACDR:

[0054]

[0055] The above four quantitative indicators of arch height (arch height distribution curve, arch height area, standardized arch height value and S_ACDR) make up for the uncertainty of judging the rationality of intraocular lens implantation by single central arch height in clinical practice, and expand the indicators to correlation quantities related to anterior chamber depth, greatly enriching the evaluation of ICL postoperative effects.

[0056] Through the above scheme, the Scheimpflug anterior segment image is segmented into various tissue structure areas through the improved deep convolutional segmentation network, and then the segmented image is corrected for distortion through the homography matrix method, and the spatial position of the intraocular lens and the original lens is accurately located through the refraction return algorithm to reduce the error of subsequent vault height measurement; and the vault height distribution curve, vault height area, standardized vault height value and the ratio of the anterior chamber depth of the lens to the anterior chamber depth of the intraocular lens under the action of the sigmoid function are quantified, thereby compensating for the instability caused by the single central vault height measurement, and providing more accurate and complete statistical features for clinical practice, so as to achieve the purpose of comprehensively evaluating the effect of ICL implantation.

[0057] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0058] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0059] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0060] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0061] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0062] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.

[0063] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved, and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.

[0064] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative and not restrictive with respect to the scope of the present invention, and the scope of the present invention is defined by the appended claims.

Claims

1. A method for quantifying the vault height after intraocular lens implantation, suitable for execution in a computing device, characterized in that: The method comprises: Acquire an image of the anterior segment of the eye to be measured; Determine the edge position information of the cornea, iris, artificial lens and natural lens in the anterior segment image to obtain an anterior segment structure edge image; Performing polynomial fitting processing on the edges of the cornea, iris, artificial lens and natural lens in the anterior segment structure edge image to obtain a corneal anterior surface curve, a corneal posterior surface curve, an artificial lens anterior surface curve, an artificial lens posterior surface curve and a natural lens anterior surface curve; Based on the posterior surface curve of the artificial lens and the anterior surface curve of the natural lens, a vault distribution curve is calculated, and the vault area is calculated by integrating the vault distribution curve over a predetermined area; Based on the arch height area and the integral area, a standardized arch height value is calculated; The ratio of the anterior chamber depth of the natural lens to the anterior chamber depth of the intraocular lens was calculated based on the posterior surface curve of the cornea, the anterior surface curve of the intraocular lens, the posterior surface curve of the intraocular lens, and the anterior surface curve of the natural lens.

2. The method according to claim 1, characterized in that: The step of calculating the ratio of the natural lens anterior chamber depth to the artificial lens anterior chamber depth comprises: Based on the curve of the posterior surface of the cornea, the curve of the anterior surface of the intraocular lens, and the curve of the posterior surface of the intraocular lens, the first anterior chamber depth from the geometric center of the intraocular lens on the central axis to the posterior surface of the cornea is calculated; Based on the curve of the posterior surface of the cornea and the curve of the anterior surface of the natural lens, the second anterior chamber depth from the anterior surface of the natural lens to the posterior surface of the cornea on the central axis is calculated; Based on the first anterior chamber depth and the second anterior chamber depth, a sigmoid function is used to calculate the ratio of the natural lens anterior chamber depth to the artificial lens anterior chamber depth.

3. The method according to claim 1, characterized in that The anterior segment image is a Scheimpflug anterior segment image acquired by a Scheimpflug camera.

4. The method according to claim 1, characterized in that The step of determining the edge position information of the cornea, the artificial lens and the natural lens in the anterior segment image to obtain the anterior segment structure edge image comprises: Based on the deep convolutional segmentation network model, the anterior segment image to be measured is segmented to obtain a binary image of the anterior segment; Performing edge extraction on the anterior segment of the eye binary image to obtain an anterior segment of the eye structure edge image; Performing coordinate transformation on each pixel of the anterior segment structure edge image based on the homography matrix to obtain a distortion-corrected anterior segment structure edge image; According to the law of refraction, the virtual image edge point position in the distortion-corrected anterior segment structure edge image is restored to the real image edge point position to obtain the refraction-corrected anterior segment structure edge image.

5. The method according to claim 4, characterized in that The step of performing regional segmentation on the anterior segment image to be measured based on the deep convolutional segmentation network model to obtain an anterior segment binary image comprises: acquiring a predetermined number of anterior segment images as training images; Manually annotate the cornea, iris, intraocular lens, natural lens and background areas in the training images to obtain a data set containing labeled data; Dividing the data set into a training set, a validation set and a test set according to a predetermined ratio; Based on the cross-validation method, the parameters and hyperparameters of the model are trained using the training set and the validation set to obtain a trained deep convolutional segmentation network model; The anterior segment image to be measured is input into the trained deep convolutional segmentation network model for region segmentation to obtain an anterior segment binary image including the cornea region, iris region, artificial lens region and natural lens region.

6. The method according to claim 5, characterized in that The step of obtaining a trained deep convolutional segmentation network model by using the training set and the validation set to train the model parameters and hyperparameters based on the cross-validation method comprises: Performing data enhancement processing on the training data in the training set, wherein the data enhancement processing includes flipping, translation, scale transformation, contrast transformation and Gaussian noise processing; Inputting the training set after data enhancement processing into the deep convolutional segmentation network model for training, when the loss value of the set similarity measurement loss function is less than a first predetermined threshold, obtaining the trained deep convolutional segmentation network model; Using the validation set to adjust the hyperparameters of the trained deep convolutional segmentation network model, the hyperparameters include a learning rate and a number of iterations, and when the loss value of the cross entropy loss function is less than a second predetermined threshold, a trained deep convolutional segmentation network model is obtained; Use the test set to evaluate the error of the trained deep convolutional segmentation network model.

7. The method according to claim 6, characterized in that The deep convolutional segmentation network model includes an input layer, a DenseUnet backbone network and an output layer, the output layer is a softmax layer, which is used to output classification results of multiple organizational structures. The DenseUnet backbone network includes a cascaded first convolutional layer, a first dense block, a first downsampling unit, a second dense block, a second downsampling unit, a third dense block, a third downsampling unit, a fourth dense block, a first upsampling unit, a fifth dense block, a second upsampling unit, a sixth dense block, a third upsampling unit, a sixth dense block, a second convolutional layer, and a third convolutional layer. The backbone network also includes a feature fusion module, which includes a first fusion unit, a second fusion unit and a third fusion unit. The first fusion unit is used to merge the feature map output by the third downsampling unit with the feature map output by the first upsampling unit through channels, the second fusion unit is suitable for merging the feature map output by the second downsampling unit with the feature map output by the second upsampling unit through channels, and the third fusion unit is suitable for merging the feature map output by the first downsampling unit with the feature map output by the third upsampling unit through channels.

8. The method according to claim 4, characterized in that The step of performing coordinate transformation on each pixel of the anterior segment structure edge image based on the homography matrix to obtain the distortion-corrected anterior segment structure edge image comprises: A standard checkerboard grid plate is placed on the image surface of the edge of the anterior segment structure to obtain a grid plate image; Detect the corner points of the checkerboard in the checkerboard image, take the upper left corner of the image as the image coordinate origin, and obtain the position coordinates of each corner point; Taking the center position of the grid plate as the actual coordinate origin, the homography matrix is ​​calculated by combining the actual size of the grid plate and the position coordinates of the obtained corner points; Each pixel of the grid plate image is applied to the homography matrix to obtain an edge image of the anterior segment structure after distortion correction.

9. A computing device comprising: at least one processor; and A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method according to any one of claims 1 to 8.

10. A readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, the computing device executes the method according to any one of claims 1 to 8.

Images