Corneal crosslinking operation effect prediction method

Abstract

A corneal cross-linking operation effect prediction method comprises the following steps: acquiring corneal biomechanics dynamic detection video data, and making a data set; extracting the cornea contour of each frame in the video, and after arrangement and integration according to a time axis, calculating the height difference of each pixel point by taking the height of the central cornea of the first frame as a reference to obtain a three-dimensional data matrix; building a corneal crosslinking operation prediction model based on a deep neural network, and predicting a postoperative three-dimensional data matrix of the corneal crosslinking operation by using a three-dimensional data matrix training network; and restoring the predicted data matrix into the front and rear surface contours of the cornea arranged according to time, and calculating biomechanical parameters to obtain predicted cross-linking postoperative biomechanical characteristics. According to the method, the dynamic response characteristics in dynamic detection video data of corneal biomechanics are fully excavated, shallow details and deep semantics are considered, end-to-end deployment of a medical image system is supported, pre-evaluation of a preoperative operation effect is realized, prediction accuracy is improved, and a foundation is laid for regulation and control of personalized operation parameters.

Description

TECHNICAL FIELD

[0001] The present application relates to medical artificial intelligence technology, in particular to a corneal cross-linking operation effect prediction method. BACKGROUND

[0002] Keratoconus is a dilatation disease characterized by progressive thinning and protrusion of the central or para-central cornea, often starting in adolescence, mainly leading to irregular astigmatism and visual acuity decline, and in the late stage, corneal scarring may occur, which seriously affects vision and is one of the major causes of blindness. Although the exact cause is not clear, studies generally believe that it is related to factors such as genetics, biomechanical abnormalities, metabolic disorders, and long-term rubbing of the eyes. Currently, the clinical diagnosis of keratoconus mainly relies on the evaluation of corneal topography on corneal morphology and the analysis of corneal biomechanics by devices such as Corvis ST.

[0003] Severe patients may need corneal transplantation. However, due to the shortage of donors, the actual number of corneal transplants per year is much lower than the clinical demand. Therefore, exploring alternative treatment methods has become a problem that needs to be solved. Corneal cross-linking (CXL) is a key technology currently used in clinical practice to delay or prevent the progression of keratoconus. This surgery uses riboflavin as a photosensitizer to generate active oxygen free radicals under ultraviolet irradiation, promoting the covalent cross-linking between collagen fibers, thereby enhancing the biomechanical strength of the cornea. There are now several methods of corneal cross-linking in clinical practice, including the traditional Dresden protocol, accelerated protocol, trans-epithelial technique, pulsed cross-linking, and topography-guided technique. Among them, the de-epithelialization protocol has a stronger cross-linking effect, but it is associated with intraoperative pain, postoperative infection risk, and longer recovery period, resulting in poor comfort; the trans-epithelial technique has high comfort but is significantly limited in terms of cross-linking depth and strength; to improve cross-linking efficiency and individualized treatment effect, some studies have attempted to use topography-guided CXL technology based on spatial regulation of ultraviolet light dosage or riboflavin distribution to achieve local strengthening of cross-linking.

[0004] Although CXL technology is constantly evolving, postoperative efficacy assessment still relies mainly on long-term follow-up, and how to scientifically select and adjust the surgical protocol before surgery remains a significant challenge, and there is a lack of effective preoperative prediction tools. In addition, there are significant individual differences in the organizational structure, biomechanical properties of the cornea, and the severity of the disease among different patients, and CXL surgical strategies need to be individually designed. Without reliable prediction tools, it is difficult to scientifically adjust the cross-linking protocol before surgery, which affects clinical efficacy. The corneal cross-linking process involves complex photochemical and biomechanical interactions, and existing mechanism models often ignore multiple key factors in simplified solutions, resulting in deviations between model prediction results and clinical outcomes.

[0005] However, with the development of ophthalmic imaging devices, a large amount of corneal image and dynamic data has been accumulated in clinical practice, providing a new path for establishing a data-driven postoperative effect prediction model. Some existing studies have attempted to focus on predicting CXL effect by deep learning or regression model based methods based on preoperative corneal morphology data. However, clinical practice shows that the postoperative corneal morphology changes after corneal crosslinking are not significant, and relying solely on morphological changes to evaluate the effect will also ignore the changes caused by the repair and remodeling of the corneal tissue after the operation, leading to biased evaluation of the effect. Therefore, relying solely on corneal morphology changes as an evaluation basis may not accurately reflect the actual treatment effect of crosslinking. In contrast, after corneal crosslinking, due to the formation of covalent bonds between collagen fibers, the cornea hardens, and the biomechanical properties of the cornea after crosslinking, such as elastic modulus, hardness, etc., usually change more significantly. By predicting the effect of crosslinking through changes in corneal biomechanics, it can be more reasonable, intuitive and have higher clinical relevance. Corvis ST, as a widely used corneal biomechanics detection device, can provide corneal dynamic deformation video data and related static parameters (such as DA, SP-A1, etc.). However, previous studies have only used static parameters to establish prediction models, failing to fully exploit the dynamic response characteristics of corneal morphology contained in the corneal biomechanics dynamic detection video data, such as corneal vertex speed, flattening propagation path, deformation recovery process, etc., resulting in an inability to accurately predict postoperative changes in corneal biomechanics and difficulty in meeting the clinical needs of preoperative evaluation of surgical effect and development of individualized surgical plans.

[0006] It should be noted that the information disclosed in the above BACKGROUND section is only for understanding the background of the present application, and therefore can include information that does not constitute the prior art known to those of ordinary skill in the art. SUMMARY

[0007] The main purpose of the present application is to overcome the defects existing in the background art, and to provide a corneal crosslinking surgery effect prediction method.

[0008] To achieve the above-mentioned purpose, the present application adopts the following technical solutions: A corneal crosslinking surgery effect prediction method, comprising the following steps: S1, obtaining corneal biomechanics dynamic detection video data of multiple patients before and after corneal crosslinking surgery, and making a data set; S2, based on the data set, extracting the corneal contour of each frame of video, arranging and integrating each frame of corneal contour according to the time axis into a whole, taking the height of the center cornea of the first frame as the reference, calculating the height difference of each pixel point, and obtaining a three-dimensional data matrix; S3, building a corneal crosslinking surgery prediction model based on a deep neural network, training the deep neural network using the three-dimensional data matrix, and predicting the three-dimensional data matrix after the corneal crosslinking surgery using the trained network; S4, restoring the predicted three-dimensional data matrix to the time-arranged corneal anterior and posterior surface profiles, calculating the biomechanical parameters according to the profiles, and obtaining the predicted post-crosslinking biomechanical characteristics.

[0009] Further, the step S1 specifically comprises: acquiring corneal biomechanical dynamic detection video data of a keratoconus patient before and after corneal crosslinking surgery; manufacturing a corneal crosslinking surgery sample corneal video data set, the data set comprising data of multiple keratoconus patients, each patient comprising corneal biomechanical dynamic detection video data before crosslinking and corneal biomechanical dynamic detection video data after crosslinking.

[0010] Further, the step S2 specifically comprises: analyzing the input video and extracting multiple frames of images in the video; for each frame of image, automatically extracting a region containing all corneal contour pixel points and clipping the region into an image of the same size; performing superpixel pre-segmentation on each frame of clipped image, and assigning the same semantic label to small regions with the same semantic information; designing a convolutional neural network to classify each pixel of the input image, so that the semantic segmentation result meets the pre-segmentation result, and by iteratively merging small regions with the same semantics, a target semantic segmentation result is obtained; when the loss function value of the segmentation result meets the preset condition, stopping the model training and outputting the image segmentation result; when training the image sequence, the first frame of image uses randomly initialized network parameters, and the subsequent frames of image use the network parameters after training of the previous frame of image as initial parameters for training; extracting the contour of the segmented image, arranging the contours of multiple frames of images in time sequence, and integrating the contours into a three-dimensional data matrix; taking the center height of the corneal anterior surface and posterior surface in the first frame of image as a reference datum, for each pixel point in each subsequent frame of image, calculating the height difference between the pixel point and the corresponding reference datum, and obtaining a three-dimensional data matrix of relative height difference.

[0011] Further, the superpixel pre-segmentation comprises: converting the image from an RGB color space to an LAB color space; setting the number of superpixels and calculating the initial size of each superpixel and the distance between seed points based on the total number of image pixels; calculating the pixel gradient in the neighborhood of each seed point, and moving the seed point to the position with the smallest gradient; for each pixel, calculating the color distance and spatial distance between the pixel and the seed point, and combining the two distances according to preset parameters to calculate the total distance, so as to assign the pixel to the nearest superpixel; Iteratively update the superpixel centers and reassign the pixels until convergence.

[0012] Further, in the semantic segmentation of the training image sequence, the first frame image uses randomly initialized network parameters, and from the second frame, each frame image uses the network parameters trained in the previous frame image as the initial parameters for training to accelerate the training process by utilizing the inter-frame similarity.

[0013] Further, step S3 specifically comprises: Standardize the data input and output formats, and store the paired input matrix and prediction target matrix in batches in a preset format; Normalize the data, scale the data values to a preset interval, and adjust the channel dimension to adapt to the neural network input; Construct a regression neural network model based on an encoder-decoder network structure; Train the neural network model using a supervised learning method, use paired input images and target images as training samples, select mean square error loss as the loss function, update the network parameters through the back propagation algorithm, use the mini-batch training strategy and the stochastic gradient descent optimizer, and iteratively train the network until the loss converges; Preprocess the image to be predicted in the same way as in the training phase, convert it to tensor format and input it into the trained neural network model to obtain the output result.

[0014] Further, in the regression neural network model based on the encoder-decoder network structure, the encoder module gradually extracts features through multiple down-sampling convolution units, each convolution unit includes convolution operation, batch normalization and activation function, and reduces the spatial resolution of the feature map through pooling operation; the decoder module recovers the feature map size through up-sampling operation, and combines the shallow features in the encoder through jump connection; the output module adjusts the channel number through the convolution layer to match the output requirements.

[0015] Further, step S4 specifically comprises: Separate the predicted three-dimensional data matrix by channel to obtain the height images of the anterior and posterior surfaces of the cornea, respectively; Based on the reconstructed corneal deformation sequence, extract key biomechanical parameters at each time; Compare the predicted biomechanical parameters with the actual measurement data to evaluate the structural strengthening effect and the risk of lesion progression after corneal crosslinking.

[0016] Further, the key biomechanical parameters in step S4 include at least one of the maximum flattening length, the first flattening time, and the second flattening time.

[0017] A computer program product comprises a computer program which, when executed by a processor, implements the corneal cross-linking surgery effect prediction method.

[0018] The present application has the following beneficial effects: The present application provides a corneal cross-linking surgery effect prediction method based on corneal biomechanics dynamic detection video data (such as Corvis ST video data), and a deep learning model based on such video data is constructed for predicting the corneal biomechanics changes after corneal cross-linking (CXL). Specifically, the present application obtains a target three-dimensional data matrix by processing and extracting the corneal profile in the corneal biomechanics dynamic detection video frame, and then learns the relationship between the preoperative and postoperative three-dimensional data matrices by using a deep neural network, thereby realizing personalized postoperative corneal biomechanics property prediction.

[0019] The method effectively solves the problem that the corneal cross-linking effect can only be obtained by long-term follow-up in the current clinic, and can directly predict the effect of corneal cross-linking surgery before operation. At the same time, it also breaks through the limitation that the previous corneal cross-linking surgery effect prediction method cannot fully utilize image data and can only predict several postoperative indicators, and provides a more accurate tool for corneal cross-linking surgery prediction. In the preferred scheme of the present application, a regression neural network model based on an encoder-decoder network structure is constructed, which can balance the shallow detail information and the deep semantic features, effectively improve the accuracy of image regression prediction, and the model output is structure-aligned data, which is convenient for end-to-end deployment in a medical image system without complex post-processing operations. In addition, the method also gets rid of the limitation of traditional video frame prediction relying on time series modeling, and can process single image input, greatly simplifying the modeling process.

[0020] From the clinical application value, the present application provides a data-driven simulation mechanism for preoperative corneal cross-linking scheme formulation, and the preoperative effect prediction realized can not only provide a basis for regulating personalized surgical parameters, improving treatment consistency and reducing postoperative uncertainty, but also can realize intelligent prediction of the degree of biomechanics improvement after cross-linking by extracting and utilizing the spatiotemporal features in the corneal deformation process, provide a basis for individualized design of CXL surgery, help to improve the efficiency of efficacy evaluation, and promote the development of precise treatment of keratoconus.

[0021] In summary, the corneal cross-linking surgery effect prediction method based on Corvis ST video data provided by the present application can predict the biomechanics changes of the cornea after corneal cross-linking surgery in different patients before operation, which not only provides a more accurate tool for corneal cross-linking surgery prediction, but also lays a solid foundation for data-driven simulation of preoperative corneal cross-linking scheme formulation and individualized surgical parameter regulation.

[0022] Other beneficial effects in the embodiments of the present application will be further described in the following. Attached Figure Description

[0023] Figure 1 This is an overall flowchart of the corneal cross-linking surgery effect prediction method based on corneal biomechanical dynamic detection video data according to an embodiment of the present invention.

[0024] Figure 2 This is a neural network structure diagram for video frame segmentation to achieve dynamic detection of corneal biomechanics in an embodiment of the present invention.

[0025] Figure 3 This is a neural network structure diagram for predicting the three-dimensional data matrix after corneal cross-linking surgery in an embodiment of the present invention.

[0026] Figure 4 This is a three-dimensional data matrix diagram of the predicted corneal cross-linking procedure in an embodiment of the present invention. Detailed Implementation

[0027] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0028] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0029] This invention aims to address the problem of relying on long-term follow-up to assess the effectiveness of corneal cross-linking surgery in clinical practice. It proposes first acquiring dynamic corneal biomechanical video data (such as Corvis ST video data) to create a dataset. Then, it extracts the corneal contour from each frame of the video and integrates them along the time axis. Using the central corneal height of the first frame as a benchmark, it calculates the pixel height difference to obtain a three-dimensional data matrix. Subsequently, it builds a prediction model based on a deep neural network, trains the model using the three-dimensional data matrix to predict the postoperative three-dimensional data matrix, and finally reconstructs the matrix to represent the anterior and posterior corneal surface contours and calculates biomechanical parameters. This achieves accurate preoperative prediction of postoperative biomechanical characteristics after cross-linking surgery. This approach fully utilizes dynamic video data, providing a precise tool for predicting the effectiveness of corneal cross-linking surgery and laying a data-driven foundation for preoperative planning.

[0030] See Figure 1 This invention provides a method for predicting the outcome of corneal cross-linking surgery, comprising the following steps: Step S1: Obtain dynamic corneal biomechanical data (such as Corvis ST video data) before and after corneal crosslinking surgery for patients (such as multiple keratoconus patients) and create a dataset.

[0031] In some embodiments, step S1 specifically comprises: obtaining corneal biomechanical dynamic detection video data of the corneas of the plurality of keratoconus patients before and after the corneal crosslinking surgery; and preparing a corneal crosslinking surgery sample corneal video data set, which comprises data of the plurality of keratoconus patients, each patient comprising one corneal biomechanical dynamic detection video data before the crosslinking and one corneal biomechanical dynamic detection video data after the crosslinking.

[0032] Step S2 comprises: based on the data set, extracting the corneal contour of each frame in the video, arranging and integrating each corneal contour along the time axis into a whole, taking the height of the center cornea of the first frame as a reference, calculating the height difference of each pixel point, and obtaining a three-dimensional data matrix.

[0033] In some embodiments, step S2 specifically comprises: parsing the input video to extract multiple frames of images in the video; for each frame of image, automatically extracting a region containing all corneal contour pixel points and cropping the region into images of the same size; performing superpixel pre-segmentation on each cropped frame of image to assign the same semantic label to small regions with the same semantic information; designing a convolutional neural network to classify each pixel of the input image, so that the semantic segmentation result meets the pre-segmentation result, and by iteratively merging small regions with the same semantics, a target semantic segmentation result is obtained; when the loss function value of the segmentation result meets a preset condition, the model training is stopped, and the image segmentation result is output (for the structure of the convolutional neural network used to realize the corneal biomechanical dynamic detection video frame semantic segmentation, refer to Figure 2 ); when training the image sequence, the first frame of image uses randomly initialized network parameters, and the subsequent frame of image uses the network parameters after the training of the previous frame of image as the initial parameters for training; extracting the contour of the segmented image, arranging the contours of the multiple frames of image in chronological order, and integrating them into a three-dimensional data matrix; taking the center height of the anterior surface and the posterior surface of the cornea in the first frame of image as the reference datum, calculating the height difference between each pixel point in each subsequent frame of image and the corresponding reference datum, and obtaining a three-dimensional data matrix of the relative height difference.

[0034] In some embodiments, the superpixel pre-segmentation comprises: converting the image from the RGB color space to the LAB color space; setting the number of superpixels and calculating the initial size and seed point spacing of each superpixel based on the total number of image pixels; calculating the pixel gradient in the neighborhood of each seed point and moving the seed point to the position with the smallest gradient; for each pixel, calculating its color distance and spatial distance from the seed point, and combining the two to calculate the total distance according to the preset parameters to assign the pixel to the nearest superpixel; iteratively updating the superpixel center and reassigning the pixels until convergence. The number of superpixels, weight parameters and initial size can be determined in combination with the image resolution or the segmentation accuracy of the validation set to adapt to the corneal contour segmentation requirements.

[0035] In some embodiments, when training the image sequence for semantic segmentation, the first frame image uses randomly initialized network parameters, and starting from the second frame, each frame image uses the network parameters trained from the previous frame image as the initial parameters for training, to accelerate the training process by utilizing the inter-frame similarity.

[0036] Step S3, building a corneal cross-linking surgery prediction model based on a deep neural network, training a deep neural network using the three-dimensional data matrix, and predicting a three-dimensional data matrix after the corneal cross-linking surgery using the trained network.

[0037] In some embodiments, step S3 specifically comprises: standardizing the data input and output formats, and storing the paired input matrix and prediction target matrix in batches in a preset format; normalizing the data, scaling the data values to a preset interval, and adjusting the channel dimension to adapt to the neural network input; constructing a regression neural network model based on an encoder-decoder network structure (see Figure 3 ); training the neural network model in a supervised learning manner, using the paired input image and target image as training samples, selecting mean square error loss as the loss function, updating the network parameters through the back propagation algorithm, using the mini-batch training strategy and the stochastic gradient descent optimizer, and iteratively training the network until the loss converges; preprocessing the image to be predicted in the same way as in the training phase, converting it to a tensor format and inputting it into the trained neural network model to obtain the output result.

[0038] In some embodiments, in the regression neural network model based on the encoder-decoder network structure, the encoder module gradually extracts features through multiple down-sampling convolution units, each convolution unit includes convolution operation, batch normalization and activation function, and reduces the spatial resolution of the feature map through pooling operation; the decoder module restores the feature map size through up-sampling operation, and combines the shallow features in the encoder through skip connection; the output module adjusts the channel number through the convolution layer to match the output requirements.

[0039] Step S4, restoring the predicted three-dimensional data matrix to the corneal anterior and posterior surface profiles arranged according to time, calculating the biomechanical parameters according to the profiles, and obtaining the predicted biomechanical characteristics after the cross-linking surgery.

[0040] In some embodiments, step S4 specifically comprises: separating the predicted three-dimensional data matrix by channel to obtain the height images of the corneal anterior and posterior surfaces respectively; extracting key biomechanical parameters at each time based on the reconstructed corneal deformation sequence; comparing the predicted biomechanical parameters with the actual measurement data to evaluate the structural strengthening effect and the risk of disease progression after the corneal cross-linking surgery.

[0041] In some embodiments, the key biomechanical parameters include at least one of the maximum flattening length, the first flattening time, and the second flattening time.

[0042] The core technical advantages of the corneal cross-linking surgery effect prediction method are as follows: first, the problem that the effect of corneal cross-linking surgery can be evaluated only by long-term follow-up in the clinic is solved, and the postoperative corneal biomechanical characteristics can be predicted through preoperative corneal biomechanical dynamic detection video data (such as Corvis ST video data); second, the limitations of previous prediction models that do not fully utilize image data and can only predict a small number of postoperative indicators are broken through, in particular, a regression neural network model based on an encoder-decoder network structure is constructed, shallow details and deep semantic features are considered, and output structure alignment data are provided to support end-to-end deployment of a medical image system (without complex post-processing), and the dependence on time series modeling for traditional video frame prediction is eliminated, and the modeling process is simplified through single image input; third, a data-driven simulation mechanism is provided for preoperative corneal cross-linking scheme development, preoperative effect evaluation is realized, personalized surgical parameters are laid as a foundation, and the accuracy of surgical effect prediction is improved.

[0043] The specific embodiments of the present application, algorithm examples, and experimental verification are further described below.

[0044] A corneal cross-linking surgery effect prediction method based on corneal biomechanical dynamic detection video data mainly includes the following steps: 1) obtaining Corvis ST video data and making a data set; 2) based on the data set, extracting the corneal contour of each frame in the video, integrating each frame of the corneal contour according to the time axis as a whole, taking the height of the central cornea of the first frame as a reference, calculating the height difference of each pixel point, and obtaining a three-dimensional data matrix; 3) building a corneal cross-linking surgery prediction model based on a deep neural network, training the deep neural network using the three-dimensional data matrix, and predicting the three-dimensional data matrix after the corneal cross-linking surgery using the trained network; and 4) restoring the predicted three-dimensional data matrix to the corneal anterior and posterior surface contours arranged according to the time, calculating the biomechanical parameters according to the contours, and obtaining the predicted postoperative biomechanical characteristics of the cross-linking.

[0045] Referring to Figures 1 to 4 The embodiment of the present application provides a corneal cross-linking surgery effect prediction method based on Corvis ST video data, which includes the following steps: Step S1, data set making: obtaining Corvis ST video data and making a data set.

[0046] In the preferred embodiment, step S1 specifically includes the following substeps: S11: screening patients with keratoconus who have undergone corneal cross-linking surgery from a hospital database, and obtaining the Corvis ST video data of the patients before and after the corneal cross-linking surgery; S12: making a corneal cross-linking surgery sample cornea video data set; the sample cornea video data set includes data of multiple keratoconus patients, each patient including one Corvis ST video data before cross-linking and one Corvis ST video data after cross-linking; Step S2, Corvis ST video data preprocessing: based on the data set obtained in S1, the corneal contour of each frame in the video is extracted, and each frame of corneal contour is arranged and integrated into a whole according to the time axis. Taking the height of the central cornea of the first frame as the reference, the height difference of each pixel point is calculated to obtain an image displayed in two-dimensional form.

[0047] In a preferred embodiment, step S2 specifically includes the following sub-steps: S21: After the frame extraction module receives the input Corvis ST video file, it uses a frame-by-frame decoding method to analyze the video and sequentially extracts all 140 frames of images. Each frame of image corresponds to a time sequence point and contains dynamic deformation information of the corneal anterior surface under the action of air pulse. The frame extraction process can be realized based on the proprietary interface of OpenCV, and the extracted images are uniformly converted to grayscale format.

[0048] S22: For the 140 frames of corneal deformation images extracted in step S21, further image preprocessing operations are performed to remove irrelevant areas and standardize the image size; the useless information in the Corvis ST video has a fixed position. In each frame of image, the corneal contour area is located, a rectangular bounding box containing all the corneal contour pixel points is constructed, and the image is automatically cropped based on this.

[0049] S23: SLIC superpixel pre-segmentation is performed on each frame of cropped image, and the same semantic label is assigned to small areas with obviously same semantic information, to obtain multiple pixel point groups and their corresponding classification labels.

[0050] In a more preferred embodiment, the SLIC superpixel segmentation in step S23 includes the following sub-steps: S231: Convert the image from RGB space to LAB space, set the number of superpixels to K, and divide the image of N pixel points into K superpixels of the same size. The size of each superpixel should be N / K, and the distance between adjacent seed points is approximately .

[0051] S232: Calculate the gradient value of all pixel points in a neighborhood, move the seed point to the place with the smallest gradient in the neighborhood, avoid the seed point falling on the contour boundary with large gradient, and affect the subsequent clustering effect. The difference calculation formula of gradient is as follows:

[0052]

[0053] .

[0054] wherein, is the gradient value of the pixel point, is the two-dimensional coordinate of the pixel point, and are the right adjacent pixel point (horizontal direction adjacent) and the lower adjacent pixel point (vertical direction adjacent) of the pixel point respectively. is the gray value of the pixel point, and are the right adjacent pixel point (horizontal direction adjacent) and the lower adjacent pixel point (vertical direction adjacent) of the pixel point respectively.

[0055] S233: For each searched pixel point, calculate its distance from the seed point respectively, the distance calculation method is as follows, wherein represents the color distance, represents the spatial distance, which is calculated by and two parameters to control the proportion of the two distances:

[0056]

[0057] .

[0058] wherein, are the corresponding components of the LAB color space of the seed point respectively, are the corresponding components of the LAB color space of the pixel point to be allocated respectively, is the two-dimensional coordinate of the seed point in the image, is the two-dimensional coordinate of the pixel point to be allocated in the image, is the comprehensive distance of the pixel point from the seed point.

[0059] S234: Traverse the pixel points in the 2Sx2S region around each superpixel block center point, calculate the distance of each pixel point in the region from the center point of which superpixel block is closest, and divide it into it; After completing an iteration, recalculate the center point coordinates of each superpixel block and reiterate.

[0060] S24: a convolutional neural network is designed, and each pixel of the input image is classified by using the neural network, so that the output semantic segmentation result is as consistent as possible with the SLIC pre-segmentation result; the semantic segmentation result of the neural network will be combined with the same semantic information on the basis of the pre-segmentation result, and a large block is obtained; through multiple iterations, the small blocks are gradually merged, and finally the expected semantic segmentation result is obtained. The network structure is as shown in Figure 2

[0061] The structure of the convolutional neural network is in turn a convolutional layer 1 (kernel 3*3), using BatchNorm and ReLU layer activation; a convolutional layer 2 (kernel 3*3), using BatchNorm and ReLU layer activation; and a convolutional layer 3 (kernel 3*3), using BatchNorm activation. After the convolutional layer 3 output is normalized, the predicted class index is obtained by the maximum value index function, as the final output; at the same time, the class probability distribution is calculated by using the normalized exponential function Softmax, which provides feedback input for the maximum value index function to participate in loss calculation. In the training process, the model parameters are updated by using SDG random gradient descent.

[0062] S25: when the loss function value corresponding to the segmentation result meets the preset value and the segmentation category meets the target value, stop the model training and output the image segmentation result; the loss function value is used to represent the loss value between the predicted clustering label and the pre-classification label of the pixel point group output by the corneal image segmentation module, and the softmax function is used for calculation.

[0063] S26: in this embodiment, since the morphology changes slowly and the structures are similar between adjacent frames in the Corvis ST image sequence, a training strategy of inter-frame parameter inheritance mechanism is designed. When training the first frame image in the corneal image sequence, the convolutional neural network semantic segmentation model is constructed, and the network weight parameters are randomly initialized. From the second frame image, when training each frame image, the network parameters are not reinitialized, but the network weight after the training of the previous frame is loaded as the initial parameter of the current frame, so as to avoid repeated learning of similar features. The above process is executed on the whole 140 frame image sequence, and the network parameters are continuously updated in each frame image training and passed to the next frame, forming a continuous time sequence training path, so as to realize efficient learning of the image sequence segmentation model.

[0064] ​S27: Contour extraction uses a connected domain contour extraction method to perform edge detection and contour tracking operations on each frame of the semantic segmentation image to obtain a continuous contour line segment at the pixel level; the extracted contour line is mapped to a two-dimensional coordinate data according to the image coordinate system, and the position in the horizontal direction of the image and the contour value in the height direction of the image are recorded respectively; two contour lines should be included in each image, corresponding to the front surface and the back surface of the cornea respectively; the contour information of all 140 images is arranged in time sequence, and the front and back surface contour data of each image together form a two-dimensional section (width x channel number), and are stacked along the time axis to form a three-dimensional tensor, and finally a contour three-dimensional data matrix with a size of 140x576x2 is generated, wherein the first dimension corresponds to the image frame sequence, the second dimension corresponds to the sampling point number in the horizontal direction of the image, and the third dimension corresponds to the contour height value of the front surface and the back surface of the cornea respectively.

[0065] S28: Take the first frame of the corneal video sequence as the reference frame, and extract the front and back surface center positions in the image center of the frame, respectively denoted as front surface center height and back surface center height ; that is, for any frame image , the height value of the column pixel is transformed to , ; after normalization, a contour three-dimensional data matrix is obtained which retains the original structure but has relative height difference values, and the size is still 140x576x2.

[0066] Step S3, prediction network building and training: build a corneal cross-linking surgery prediction model based on a deep neural network, train the deep neural network using the images obtained in S2, use the trained network to predict the three-dimensional data matrix after the corneal cross-linking surgery, and process the video sequence processing image to realize visual presentation.

[0067] S31: Standardize the data input and output format, and store the paired input matrix and prediction target matrix in npz format in batches, each file containing an input image tensor and a corresponding target image tensor, corresponding to fields x and y respectively.

[0068] S32: Read the input field x and the output field y in the file, and perform a min-max normalization operation on the values in each tensor to scale all data values to the [0, 1] interval. The normalization calculation formula is , is the original height value, and The minimum and maximum values in the current tensor, respectively; the normalized image tensor is converted to float32 format to ensure numerical accuracy and compatibility with neural network computing requirements; the original data dimension is converted from 140x576x2 to 2x140x576, with the channel dimension placed at the front to adapt to the input requirements of the convolution module in the mainstream deep learning framework Pytorch.

[0069] S33: A regression neural network model based on an encoder-decoder network structure is constructed, and the overall structure includes an encoder module, a feature bottleneck module, a decoder module, and an output module. The encoder module is composed of multiple down-sampling convolution units, each unit containing two convolution operations, respectively connected with BatchNorm and ReLU, gradually reducing the spatial resolution of the feature map through max-pooling operation, while increasing the number of channels, realizing the extraction of high-dimensional semantic features of the input image; the feature bottleneck module is used to connect the encoder and the decoder, and a deeper convolution structure is adopted to enhance the feature expression ability; the feature bottleneck module contains multiple convolution blocks, which are used to integrate multi-scale features extracted by the encoder and compress them into low-dimensional compact representations, enhancing the model's ability to express complex spatial patterns; the decoder module performs step-by-step up-sampling on the bottleneck features to restore the spatial size of the original image (preferably by using transpose convolution to restore the spatial resolution step by step), and after each level of up-sampling, the feature map is connected with the corresponding encoder layer to make up for the loss of spatial details in the up-sampling process; the last layer is a 1x1 convolution, which is used to restore the number of channels of the feature map to 2, corresponding to the predicted values of the anterior and posterior surfaces of the cornea, respectively. Its network structure is shown in Figure 3 .

[0070] S34: The neural network model is trained using a supervised learning method, using paired input images and target images as training samples, and the mean square error loss is selected as the loss function, whose expression is The network parameters are updated through the backpropagation algorithm, and the small batch training strategy is used in combination with the stochastic gradient descent optimizer to iteratively train the network until the loss converges.

[0071] S35: The image to be predicted is preprocessed in the same way as in the training stage, converted to tensor format 2x140x576, and sent into the trained neural network model to obtain the model output result. The output is a prediction tensor with a size of 2x140x576, and the output result is saved. The output result can be saved in npz format for subsequent analysis or visualization.

[0072] Step S4, prediction of biomechanical parameters: the image obtained in S3 is restored to the corneal anterior and posterior surface profiles arranged according to time, and the biomechanical parameters are calculated according to the profiles to obtain the predicted biomechanical characteristics after crosslinking.

[0073] S41: Channel separation is performed on the predicted image tensor obtained in step S3 (size 2x139x576), to correspond to the anterior and posterior surface height images of the cornea respectively.

[0074] S42: Based on the reconstructed corneal deformation sequence, key structure indicators at each time are extracted, including maximum flattening length, first flattening time, second flattening time, etc. Figure 4 As shown in the figure, the three-dimensional data matrix visualization results of the predicted corneal anterior surface (Predict - Upper Surface) and posterior surface (Predict - Lower Surface) after the corneal crosslinking surgery are obtained.

[0075] S43: The predicted biomechanical parameters are compared with the actual measurement data to evaluate the structural strengthening effect after the corneal crosslinking surgery and predict the future risk of disease progression.

[0076] In summary, the present application proposes a corneal crosslinking surgery effect prediction method based on Corvis ST video data, which includes obtaining Corvis ST video data of patients with keratoconus before and after corneal crosslinking surgery, and making a data set; based on the data set, the corneal contour of each frame in the video is extracted, and each frame of the corneal contour is arranged and integrated into a whole according to the time axis, taking the height of the central cornea of the first frame as the reference, calculating the height difference of each pixel point to obtain a three-dimensional data matrix; a corneal crosslinking surgery effect prediction method based on deep neural network is built, the three-dimensional data matrix is used to train the neural network, and the trained network is used to predict the three-dimensional data matrix after the corneal crosslinking surgery; the predicted three-dimensional data matrix is restored to the anterior and posterior surface contours of the cornea arranged according to the time, the biomechanical parameters are calculated according to the contours, and the predicted biomechanical characteristics after the crosslinking surgery are obtained. In summary, this method can predict the biomechanical changes of the cornea after the corneal crosslinking surgery for different patients before the surgery, providing a more accurate tool for predicting the corneal crosslinking surgery; a data-driven simulation mechanism is provided for preoperative plan development of the corneal crosslinking surgery, and the preoperative effect evaluation achieved can lay a foundation for regulating individualized surgical parameters.

[0077] The embodiment of the present application also provides a storage medium for storing a computer program, which is executed to perform at least the method described above.

[0078] The embodiment of the present application also provides a control device, which includes a processor and a storage medium for storing a computer program; wherein the processor is used to execute the computer program to perform at least the method described above.

[0079] The embodiment of the present application also provides a processor, which executes a computer program to perform at least the method described above.

[0080] The storage medium can be implemented by any type of nonvolatile storage device, or a combination thereof. The nonvolatile memory can be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Ferromagnetic Random Access Memory (FRAM), a Flash Memory, a magnetic surface storage, an optical disc or a Compact Disc Read-Only Memory (CD-ROM). The magnetic surface storage can be a magnetic disc memory or a magnetic tape memory. The storage medium described in the embodiments of the present application is intended to include, but is not limited to, these and any other suitable type of memory.

[0081] In several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other ways. The above-described device embodiments are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, alternative division can be used. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or in other forms.

[0082] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0083] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0084] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program performs the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disc or an optical disc and various storage medium capable of storing program codes.

[0085] Alternatively, the integrated unit of the present application can be stored in a computer readable storage medium if it is realized in the form of a software function module and sold or used as an independent product. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes a mobile storage device, a ROM, a RAM, a magnetic disc or an optical disc and various storage medium capable of storing program codes.

[0086] The methods disclosed in the several method embodiments of the present application can be combined arbitrarily without conflict to obtain new method embodiments.

[0087] The features disclosed in the several product embodiments of the present application can be combined arbitrarily without conflict to obtain new product embodiments.

[0088] The features disclosed in the several method or device embodiments of the present application can be combined arbitrarily without conflict to obtain new method embodiments or device embodiments.

[0089] The above is a further detailed description of the present application in combination with specific preferred embodiments, and the specific implementation of the present application cannot be limited to these descriptions. For those skilled in the art, without departing from the concept of the present application, a number of equivalent substitutions or obvious modifications can be made, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present application.

Claims

1. A method for predicting the outcome of corneal cross-linking surgery, characterized in that, Includes the following steps: S1. Obtain dynamic video data of corneal biomechanics before and after corneal cross-linking surgery for multiple patients and create a dataset; S2. Based on the dataset, extract the corneal contour of each frame in the video, arrange the corneal contour of each frame according to the time axis and integrate them into a whole. Taking the height of the cornea at the center of the first frame as the reference, calculate the height difference of each pixel to obtain a three-dimensional data matrix. S3. Build a corneal cross-linking surgery prediction model based on deep neural network, train the deep neural network using the three-dimensional data matrix, and use the trained network to predict the three-dimensional data matrix after corneal cross-linking surgery. S4. The predicted three-dimensional data matrix is ​​restored to the corneal anterior and posterior surface contours arranged according to time. Biomechanical parameters are calculated based on the contours to obtain the predicted biomechanical characteristics after cross-linking surgery.

2. The method for predicting the effect of corneal cross-linking surgery as described in claim 1, characterized in that, Step S1 specifically includes: Acquire dynamic video data of corneal biomechanics in keratoconus patients before and after corneal crosslinking surgery; A corneal video dataset of corneal cross-linking surgery samples was created. The dataset includes data from multiple keratoconus patients, with each patient's data including pre-cross-linking corneal biomechanical dynamic detection video data and post-cross-linking corneal biomechanical dynamic detection video data.

3. The method for predicting the effect of corneal cross-linking surgery as described in claim 1, characterized in that, Step S2 specifically includes: The input video is parsed to extract multiple frames of images from it. For each frame of image, the region containing all corneal contour pixels is automatically extracted and cropped into an image of the same size; Superpixel pre-segmentation is performed on each cropped image, and the same semantic label is assigned to small regions with the same semantic information; Design a convolutional neural network to classify each pixel of the input image so that the semantic segmentation result matches the pre-segmentation result, and obtain the target semantic segmentation result by iteratively merging small regions with the same semantics; When the loss function value of the segmentation result meets the preset condition, the model training is stopped and the image segmentation result is output. When training the image sequence, the first frame image uses randomly initialized network parameters, and subsequent frames image use the network parameters trained on the previous frame image as initial parameters for training. Extract the contours of the segmented images, arrange the contours of multiple frames in chronological order, and integrate them into a three-dimensional data matrix; Using the center height of the anterior and posterior surfaces of the cornea in the first frame as a reference, for each pixel in each subsequent frame, the height difference between its height and the corresponding reference is calculated to obtain a three-dimensional data matrix of relative height difference.

4. The method for predicting the effect of corneal cross-linking surgery as described in claim 3, characterized in that, The superpixel pre-segmentation includes: Convert the image from the RGB color space to the LAB color space; Set the number of superpixels, and calculate the initial size and seed point spacing for each superpixel based on the total number of pixels in the image; Calculate the pixel gradient in the neighborhood of each seed point, and move the seed point to the position with the minimum gradient. For each pixel, calculate its color distance and spatial distance from the seed point, and combine the two according to preset parameters to calculate the total distance, so as to assign the pixel to the nearest superpixel; Iteratively update the superpixel center and redistribute pixels until convergence.

5. The method for predicting the effect of corneal cross-linking surgery as described in claim 3, characterized in that, When performing semantic segmentation on training image sequences, the first frame uses randomly initialized network parameters. Starting from the second frame, each frame uses the network parameters trained on the previous frame as initial parameters for training, in order to accelerate the training process by leveraging inter-frame similarity.

6. The method for predicting the effect of corneal cross-linking surgery as described in claim 1, characterized in that, Step S3 specifically includes: Standardize the data input and output formats, and store pairs of input matrices and prediction target matrices in batches according to preset formats; The data is normalized, the data values ​​are scaled to a preset range, and the channel dimensions are adjusted to fit the neural network input. Construct a regression neural network model based on an encoder-decoder network structure; The neural network model is trained using supervised learning, with paired input and target images as training samples. The mean squared error loss function is selected, and the network parameters are updated through backpropagation. The network is iteratively trained using a mini-batch training strategy and a stochastic gradient descent optimizer until the loss converges. The image to be predicted is preprocessed in the same way as in the training phase, converted into tensor format, and fed into the trained neural network model to obtain the output result.

7. The method for predicting the effect of corneal cross-linking surgery as described in claim 6, characterized in that, In the regression neural network model based on the encoder-decoder network structure, the encoder module extracts features step by step through multiple downsampling convolutional units. Each convolutional unit contains convolution operations, batch normalization, and activation functions, and reduces the spatial resolution of the feature map through pooling operations. The decoder module recovers the feature map size through upsampling operations and combines it with skip connections to fuse shallow features from the encoder; The output module adjusts the number of channels through convolutional layers to match output requirements.

8. The method for predicting the effect of corneal cross-linking surgery as described in claim 1, characterized in that, Step S4 specifically includes: The predicted 3D data matrix is ​​channel-separated to obtain height images of the anterior and posterior surfaces of the cornea, respectively. Based on the reconstructed corneal deformation sequence, key biomechanical parameters at each time point were extracted; By comparing predicted biomechanical parameters with actual measurement data, the structural enhancement effect and risk of disease progression after corneal cross-linking surgery can be assessed.

9. The method for predicting the effect of corneal cross-linking surgery as described in claim 8, characterized in that, The key biomechanical parameters mentioned in step S4 include at least one of the maximum flattening length, the first flattening time, and the second flattening time.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the corneal cross-linking surgery effect prediction method as described in any one of claims 1 to 9.

Images