A method for predicting the outcome of corneal cross-linking surgery

Abstract

A method for predicting the outcome of corneal cross-linking surgery includes: acquiring dynamic corneal biomechanical detection video data and creating a dataset; extracting the corneal contour of each frame in the video, arranging and integrating them along the time axis, and calculating the height difference of each pixel based on the height of the central cornea in the first frame to obtain a three-dimensional data matrix; building a corneal cross-linking surgery prediction model based on a deep neural network, training the network using the three-dimensional data matrix, and predicting the three-dimensional data matrix after corneal cross-linking surgery; restoring the predicted data matrix to the anterior and posterior corneal surface contours arranged along the time axis, and calculating biomechanical parameters to obtain the predicted post-cross-linking post-operative biomechanical characteristics. This invention fully exploits the dynamic response features in dynamic corneal biomechanical detection video data, taking into account both superficial details and deep semantics, supports end-to-end deployment of medical imaging systems, realizes pre-operative surgical outcome assessment, improves prediction accuracy, and lays the foundation for personalized surgical parameter control.

Description

Technical Field

[0001] This invention relates to medical artificial intelligence technology, and in particular to a method for predicting the outcome of corneal cross-linking surgery. Background Technology

[0002] Keratoconus is an ectopic disease characterized by progressive thinning and protrusion of the central or paracentral cornea. It typically begins in adolescence, primarily causing irregular astigmatism and decreased vision. In later stages, corneal scarring may occur, severely impacting vision and making it one of the leading causes of blindness. Although its exact cause is unclear, research generally suggests a link to factors such as genetics, biomechanical abnormalities, metabolic disorders, and prolonged eye rubbing. Currently, the clinical diagnosis of keratoconus mainly relies on corneal topography for morphological assessment and biomechanical analysis using equipment such as the Corvis ST.

[0003] Severe cases may require corneal transplantation. However, due to donor shortages, the actual number of corneal transplants performed annually is far below clinical demand. Therefore, exploring alternative treatments has become an urgent issue. Corneal cross-linking (CXL) is currently a key technique used clinically to delay or halt the progression of keratoconus. This procedure utilizes riboflavin as a photosensitizer, generating reactive oxygen species under ultraviolet irradiation to promote covalent cross-linking between collagen fibers, thereby enhancing the biomechanical strength of the cornea. Currently, various corneal cross-linking methods exist clinically, including the traditional Dresden protocol, accelerated protocols, transepithelial techniques, pulsed cross-linking, and topographic-guided techniques. Among these, the epithelial-ablative protocol offers stronger cross-linking effects but suffers from intraoperative pain, postoperative infection risks, and longer recovery periods, resulting in poor comfort. Transepithelial techniques offer high comfort but significantly limit cross-linking depth and strength. To improve cross-linking efficiency and individualized treatment outcomes, some studies have explored topographic-guided CXL techniques, based on spatial modulation of ultraviolet light dose or riboflavin distribution, to achieve locally enhanced cross-linking.

[0004] Despite the continuous evolution of CXL technology, postoperative efficacy assessment still relies primarily on long-term follow-up. Scientifically selecting and adjusting the surgical plan preoperatively remains a significant challenge, as effective preoperative predictive tools are lacking. Furthermore, substantial individual differences exist in corneal tissue structure, biomechanical properties, and disease severity among patients, necessitating personalized design of CXL surgical strategies. Without reliable predictive methods, it is difficult to scientifically adjust the cross-linking protocol preoperatively, impacting clinical efficacy. The corneal cross-linking process involves complex photochemical and biomechanical interactions; existing mechanistic models, in their simplification, often overlook several key factors, leading to discrepancies between model predictions and clinical outcomes.

[0005] However, with the development of ophthalmic imaging equipment, a large amount of corneal images and dynamic data have been accumulated clinically, providing a new path for establishing data-driven postoperative outcome prediction models. Some existing studies have attempted to predict CXL outcomes based on preoperative corneal morphology data using deep learning or regression models. However, clinical practice shows that corneal morphological changes after corneal cross-linking are not significant. Assessing efficacy solely based on morphological changes may overlook the alterations caused by postoperative corneal tissue repair and remodeling, leading to biased efficacy assessments. Therefore, relying solely on corneal morphological changes as an assessment criterion may not accurately reflect the actual treatment effect of cross-linking. In contrast, after corneal cross-linking, due to the formation of covalent bonds between collagen fibers and corneal hardening, the biomechanical properties of the cornea (such as elastic modulus and hardness) usually undergo more significant changes. Predicting the cross-linking effect through changes in corneal biomechanics is more reasonable, intuitive, and clinically relevant. Corvis ST, as a widely used corneal biomechanical testing device, can provide dynamic corneal deformation video data and related static parameters (such as DA and SP-A1). However, previous studies have only used static parameters to build predictive models, failing to fully explore the dynamic response characteristics of the cornea contained in the dynamic detection video data of corneal biomechanics, such as corneal vertex velocity, flattening propagation path, deformation recovery process, etc., which makes it impossible to accurately predict postoperative corneal biomechanical changes and meet the clinical needs of preoperative assessment of surgical effects and development of personalized surgical plans.

[0006] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The main objective of this invention is to overcome the deficiencies in the aforementioned background technology and provide a method for predicting the outcome of corneal cross-linking surgery.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A method for predicting the outcome of corneal cross-linking surgery includes the following steps:

[0010] S1. Obtain dynamic video data of corneal biomechanics before and after corneal cross-linking surgery for multiple patients and create a dataset;

[0011] S2. Based on the dataset, extract the corneal contour of each frame in the video, arrange the corneal contour of each frame according to the time axis and integrate them into a whole. Taking the height of the cornea at the center of the first frame as the reference, calculate the height difference of each pixel to obtain a three-dimensional data matrix.

[0012] S3. Build a corneal cross-linking surgery prediction model based on deep neural network, train the deep neural network using the three-dimensional data matrix, and use the trained network to predict the three-dimensional data matrix after corneal cross-linking surgery.

[0013] S4. The predicted three-dimensional data matrix is ​​restored to the corneal anterior and posterior surface contours arranged according to time. Biomechanical parameters are calculated based on the contours to obtain the predicted biomechanical characteristics after cross-linking surgery.

[0014] Furthermore, step S1 specifically includes:

[0015] Acquire dynamic video data of corneal biomechanics in keratoconus patients before and after corneal crosslinking surgery;

[0016] A corneal video dataset of corneal cross-linking surgery samples was created. The dataset includes data from multiple keratoconus patients, with each patient's data including pre-cross-linking corneal biomechanical dynamic detection video data and post-cross-linking corneal biomechanical dynamic detection video data.

[0017] Furthermore, step S2 specifically includes:

[0018] The input video is parsed to extract multiple frames of images from it.

[0019] For each frame of image, the region containing all corneal contour pixels is automatically extracted and cropped into an image of the same size;

[0020] Superpixel pre-segmentation is performed on each cropped image, and the same semantic label is assigned to small regions with the same semantic information;

[0021] Design a convolutional neural network to classify each pixel of the input image so that the semantic segmentation result matches the pre-segmentation result, and obtain the target semantic segmentation result by iteratively merging small regions with the same semantics;

[0022] When the loss function value of the segmentation result meets the preset condition, the model training is stopped and the image segmentation result is output.

[0023] When training the image sequence, the first frame image uses randomly initialized network parameters, and subsequent frames image use the network parameters trained on the previous frame image as initial parameters for training.

[0024] Extract the contours of the segmented images, arrange the contours of multiple frames in chronological order, and integrate them into a three-dimensional data matrix;

[0025] Using the center height of the anterior and posterior surfaces of the cornea in the first frame as a reference, for each pixel in each subsequent frame, the height difference between its height and the corresponding reference is calculated to obtain a three-dimensional data matrix of relative height difference.

[0026] Furthermore, the superpixel pre-segmentation includes:

[0027] Convert the image from the RGB color space to the LAB color space;

[0028] Set the number of superpixels, and calculate the initial size and seed point spacing for each superpixel based on the total number of pixels in the image;

[0029] Calculate the pixel gradient in the neighborhood of each seed point, and move the seed point to the position with the minimum gradient.

[0030] For each pixel, calculate its color distance and spatial distance from the seed point, and combine the two according to preset parameters to calculate the total distance, so as to assign the pixel to the nearest superpixel;

[0031] Iteratively update the superpixel center and redistribute pixels until convergence.

[0032] Furthermore, when performing semantic segmentation on the training image sequence, the first frame image uses randomly initialized network parameters. Starting from the second frame, each frame image uses the network parameters trained on the previous frame image as initial parameters for training, in order to accelerate the training process by utilizing inter-frame similarity.

[0033] Furthermore, step S3 specifically includes:

[0034] Standardize the data input and output formats, and store pairs of input matrices and prediction target matrices in batches according to preset formats;

[0035] The data is normalized, the data values ​​are scaled to a preset range, and the channel dimensions are adjusted to fit the neural network input.

[0036] Construct a regression neural network model based on an encoder-decoder network structure;

[0037] The neural network model is trained using supervised learning, with paired input and target images as training samples. The mean squared error loss function is selected, and the network parameters are updated through backpropagation. The network is iteratively trained using a mini-batch training strategy and a stochastic gradient descent optimizer until the loss converges.

[0038] The image to be predicted is preprocessed in the same way as in the training phase, converted into tensor format, and fed into the trained neural network model to obtain the output result.

[0039] Furthermore, in the regression neural network model based on the encoder-decoder network structure, the encoder module extracts features step by step through multiple downsampling convolutional units. Each convolutional unit contains convolution operations, batch normalization, and activation functions, and reduces the spatial resolution of the feature map through pooling operations. The decoder module restores the feature map size through upsampling operations and combines skip connections to fuse shallow features in the encoder. The output module adjusts the number of channels through convolutional layers to match the output requirements.

[0040] Furthermore, step S4 specifically includes:

[0041] The predicted 3D data matrix is ​​channel-separated to obtain height images of the anterior and posterior surfaces of the cornea, respectively.

[0042] Based on the reconstructed corneal deformation sequence, key biomechanical parameters at each time point were extracted;

[0043] By comparing predicted biomechanical parameters with actual measurement data, the structural enhancement effect and risk of disease progression after corneal cross-linking surgery can be assessed.

[0044] Furthermore, the key biomechanical parameters mentioned in step S4 include at least one of the maximum flattening length, the first flattening time, and the second flattening time.

[0045] A computer program product includes a computer program that, when executed by a processor, implements the corneal cross-linking surgery effect prediction method.

[0046] The present invention has the following beneficial effects:

[0047] This invention provides a method for predicting the outcome of corneal crosslinking surgery based on dynamic corneal biomechanical detection video data (such as Corvis ST video data), and constructs a deep learning model based on this type of video data to predict corneal biomechanical changes after corneal crosslinking surgery (CXL). Specifically, this invention obtains a target three-dimensional data matrix by processing and extracting the corneal contour from dynamic corneal biomechanical detection video frames, and then uses a deep neural network to learn the relationship between the preoperative and postoperative three-dimensional data matrices, thereby achieving personalized prediction of postoperative corneal biomechanical properties.

[0048] This method effectively solves the current clinical problem of relying on long-term follow-up to obtain corneal cross-linking effects, enabling direct prediction of corneal cross-linking surgery outcomes preoperatively. Furthermore, it overcomes the limitations of previous methods for predicting corneal cross-linking surgery outcomes, which could not fully utilize image data and could only predict a few postoperative indicators, providing a more accurate tool for predicting corneal cross-linking surgery outcomes. In the preferred embodiment of this invention, a regression neural network model based on an encoder-decoder network structure is constructed, which can take into account both superficial detail information and deep semantic features, effectively improving the accuracy of image regression prediction. Moreover, the model output is structure-aligned data, facilitating end-to-end deployment in medical imaging systems without complex post-processing operations. In addition, this method overcomes the limitations of traditional video frame prediction relying on time-series modeling, enabling the processing of single image inputs and significantly simplifying the modeling process.

[0049] From a clinical application perspective, this invention provides a data-driven simulation mechanism for the formulation of preoperative plans for corneal cross-linking. The preoperative effect pre-assessment achieved can not only provide a basis for adjusting personalized surgical parameters, improving the consistency of efficacy and reducing postoperative uncertainty, but also achieve intelligent prediction of the degree of biomechanical improvement after cross-linking by extracting and utilizing the spatiotemporal characteristics in the corneal deformation process. This provides a basis for individualized design of CXL surgery, helps improve the efficiency of efficacy assessment, and promotes the development of precision treatment for keratoconus.

[0050] In summary, the corneal cross-linking surgery outcome prediction method based on Corvis ST video data provided by this invention can predict the biomechanical changes of the cornea after corneal cross-linking surgery in different patients before surgery. This not only provides a more accurate tool for predicting corneal cross-linking surgery, but also lays a solid foundation for data-driven simulation and personalized surgical parameter control in the formulation of preoperative corneal cross-linking plans.

[0051] Other beneficial effects of the embodiments of the present invention will be further described below. Attached Figure Description

[0052] Figure 1 This is an overall flowchart of the corneal cross-linking surgery effect prediction method based on corneal biomechanical dynamic detection video data according to an embodiment of the present invention.

[0053] Figure 2 This is a neural network structure diagram for video frame segmentation to achieve dynamic detection of corneal biomechanics in an embodiment of the present invention.

[0054] Figure 3 This is a neural network structure diagram for predicting the three-dimensional data matrix after corneal cross-linking surgery in an embodiment of the present invention.

[0055] Figure 4 This is a three-dimensional data matrix diagram of the predicted corneal cross-linking procedure in an embodiment of the present invention. Detailed Implementation

[0056] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0057] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0058] This invention aims to address the problem of relying on long-term follow-up to assess the effectiveness of corneal cross-linking surgery in clinical practice. It proposes first acquiring dynamic corneal biomechanical video data (such as Corvis ST video data) to create a dataset. Then, it extracts the corneal contour from each frame of the video and integrates them along the time axis. Using the central corneal height of the first frame as a benchmark, it calculates the pixel height difference to obtain a three-dimensional data matrix. Subsequently, it builds a prediction model based on a deep neural network, trains the model using the three-dimensional data matrix to predict the postoperative three-dimensional data matrix, and finally reconstructs the matrix to represent the anterior and posterior corneal surface contours and calculates biomechanical parameters. This achieves accurate preoperative prediction of postoperative biomechanical characteristics after cross-linking surgery. This approach fully utilizes dynamic video data, providing a precise tool for predicting the effectiveness of corneal cross-linking surgery and laying a data-driven foundation for preoperative planning.

[0059] See Figure 1 This invention provides a method for predicting the outcome of corneal cross-linking surgery, comprising the following steps:

[0060] Step S1: Obtain dynamic corneal biomechanical data (such as Corvis ST video data) before and after corneal crosslinking surgery for patients (such as multiple keratoconus patients) and create a dataset.

[0061] In some embodiments, step S1 specifically includes: acquiring corneal biomechanical dynamic detection video data of multiple keratoconus patients before and after corneal crosslinking surgery; creating a corneal video dataset of corneal crosslinking surgery samples, wherein the dataset includes data of multiple keratoconus patients, and each patient includes one corneal biomechanical dynamic detection video data before crosslinking and one corneal biomechanical dynamic detection video data after crosslinking.

[0062] Step S2: Based on the dataset, extract the corneal contour of each frame in the video, arrange the corneal contour of each frame according to the time axis and integrate them into a whole. Taking the height of the cornea at the center of the first frame as the reference, calculate the height difference of each pixel to obtain a three-dimensional data matrix.

[0063] In some embodiments, step S2 specifically includes: parsing the input video and extracting multiple frames of images from the video; for each frame, automatically extracting the region containing all corneal contour pixels and cropping it into images of the same size; performing superpixel pre-segmentation on each cropped frame and assigning the same semantic label to small regions with the same semantic information; designing a convolutional neural network to classify each pixel of the input image so that the semantic segmentation result conforms to the pre-segmentation result, and obtaining the target semantic segmentation result by iteratively merging small regions with the same semantic information; stopping model training and outputting the image segmentation result when the loss function value of the segmentation result meets the preset conditions (the convolutional neural network structure used to implement the semantic segmentation of the corneal biomechanical dynamic detection video frame can be found in [reference]). Figure 2 During image training, the first frame uses randomly initialized network parameters, and subsequent frames use the network parameters trained on the previous frame as initial parameters. The contours of the segmented images are extracted, and the contours of multiple frames are arranged in chronological order and integrated into a three-dimensional data matrix. Using the center height of the anterior and posterior surfaces of the cornea in the first frame as a reference, the height difference between each pixel in each subsequent frame and the corresponding reference is calculated to obtain a three-dimensional data matrix of relative height differences.

[0064] In some embodiments, the superpixel pre-segmentation includes: converting the image from the RGB color space to the LAB color space; setting the number of superpixels and calculating the initial size and seed point spacing of each superpixel based on the total number of pixels in the image; calculating the pixel gradient in the neighborhood of each seed point and moving the seed point to the position with the minimum gradient; for each pixel, calculating its color distance and spatial distance from the seed point, and combining the two according to preset parameters to calculate the total distance, so as to assign the pixel to the nearest superpixel; iteratively updating the superpixel center and reallocating pixels until convergence. The number of superpixels, weight parameters, and initial size can be determined in combination with the image resolution or the segmentation accuracy of the validation set to adapt to the corneal contour segmentation requirements.

[0065] In some embodiments, when performing semantic segmentation on training image sequences, the first frame image uses randomly initialized network parameters, and starting from the second frame, each frame image uses the network parameters trained on the previous frame image as initial parameters for training, so as to accelerate the training process by utilizing inter-frame similarity.

[0066] Step S3: Build a corneal cross-linking surgery prediction model based on a deep neural network, train the deep neural network using the three-dimensional data matrix, and use the trained network to predict the three-dimensional data matrix after corneal cross-linking surgery.

[0067] In some embodiments, step S3 specifically includes: standardizing the data input and output format, storing pairs of input matrices and prediction target matrices in batches according to a preset format; normalizing the data, scaling the data values ​​to a preset range, and adjusting the channel dimensions to adapt to the neural network input; and constructing a regression neural network model based on an encoder-decoder network structure (see...). Figure 3 The neural network model is trained using supervised learning, with paired input and target images as training samples. The mean squared error loss function is selected, and the network parameters are updated through backpropagation. A mini-batch training strategy and a stochastic gradient descent optimizer are used to iteratively train the network until the loss converges. The image to be predicted is preprocessed in the same way as in the training phase, converted into tensor format, and fed into the trained neural network model to obtain the output result.

[0068] In some embodiments, in the regressive neural network model based on the encoder-decoder network structure, the encoder module extracts features stepwise through multiple downsampling convolutional units. Each convolutional unit includes a convolution operation, batch normalization, and activation function, and reduces the spatial resolution of the feature map through pooling operations. The decoder module restores the feature map size through upsampling operations and combines skip connections to fuse shallow features from the encoder. The output module adjusts the number of channels through convolutional layers to match output requirements.

[0069] Step S4: Restore the predicted three-dimensional data matrix into the corneal anterior and posterior surface contours arranged according to time, calculate biomechanical parameters based on the contours, and obtain the predicted post-crosslinking biomechanical characteristics.

[0070] In some embodiments, step S4 specifically includes: performing channel separation on the predicted three-dimensional data matrix to obtain height images of the anterior and posterior corneal surfaces respectively; extracting key biomechanical parameters at each time step based on the reconstructed corneal deformation sequence; and comparing the predicted biomechanical parameters with the actual measurement data to evaluate the structural enhancement effect and disease progression risk after corneal cross-linking surgery.

[0071] In some embodiments, the key biomechanical parameters include at least one of the maximum flattening length, the first flattening time, and the second flattening time.

[0072] The corneal cross-linking surgery outcome prediction method proposed in this invention has the following core advantages: First, it solves the problem of relying on long-term follow-up to assess the outcome of corneal cross-linking surgery in clinical practice, and can predict postoperative corneal biomechanical characteristics through preoperative corneal biomechanical dynamic detection video data (such as Corvis ST video data); Second, it overcomes the limitations of previous prediction models that did not fully utilize image data and could only predict a small number of postoperative indicators. In particular, it constructs a regression neural network model based on an encoder-decoder network structure, which takes into account both superficial details and deep semantic features, and outputs structure-aligned data to support end-to-end deployment of medical imaging systems (without complex post-processing), and eliminates the dependence of traditional video frame prediction on time series modeling, simplifying the modeling process through single image input; Third, it provides a data-driven simulation mechanism for the formulation of preoperative corneal cross-linking surgery plans, realizes preoperative outcome pre-assessment, lays the foundation for adjusting personalized surgical parameters, and improves the accuracy of surgical outcome prediction.

[0073] The following further describes specific embodiments of the present invention, algorithm examples, and experimental verification.

[0074] A method for predicting the effect of corneal cross-linking surgery based on dynamic detection video data of corneal biomechanics comprises the following main steps: 1) Acquire Corvis ST video data and create a dataset; 2) Based on the dataset, extract the corneal contour of each frame in the video, integrate the corneal contours of each frame into a whole according to the time axis, and calculate the height difference of each pixel point with the height of the central cornea in the first frame as the benchmark to obtain a three-dimensional data matrix; 3) Build a corneal cross-linking surgery prediction model based on a deep neural network, train the deep neural network using the three-dimensional data matrix, and use the trained network to predict the three-dimensional data matrix after corneal cross-linking surgery; 4) Restore the predicted three-dimensional data matrix to the anterior and posterior corneal surface contours arranged according to time, calculate biomechanical parameters based on the contours, and obtain the predicted biomechanical characteristics after cross-linking surgery.

[0075] See Figures 1 to 4 This invention provides a method for predicting the outcome of corneal crosslinking surgery based on Corvis ST video data, comprising the following steps:

[0076] Step S1: Dataset Creation: Obtain Corvis ST video data and create a dataset.

[0077] In a preferred embodiment, step S1 specifically includes the following sub-steps:

[0078] S11: Screen keratoconus patients who have undergone corneal crosslinking surgery from the hospital database and obtain their Corvis ST video data before and after the corneal crosslinking surgery;

[0079] S12: Create a corneal video dataset of corneal cross-linking surgery samples; the sample corneal video dataset includes data from multiple keratoconus patients, with each patient including one pre-cross-linking Corvis ST video and one post-cross-linking Corvis ST video;

[0080] Step S2, Corvis ST video data preprocessing: Based on the dataset obtained in S1, extract the corneal contour of each frame in the video, arrange the corneal contour of each frame according to the time axis and integrate them into a whole. Taking the height of the center cornea of ​​the first frame as the reference, calculate the height difference of each pixel point to obtain an image displayed in two-dimensional form.

[0081] In a preferred embodiment, step S2 specifically includes the following sub-steps:

[0082] S21: After receiving the input Corvis ST video file, the frame extraction module parses the video using a frame-by-frame decoding method, sequentially extracting all 140 frames. Each frame corresponds to a time point, containing dynamic deformation information of the anterior corneal surface under the action of air pulses. This frame extraction process can be implemented based on OpenCV's proprietary interface, and the extracted images are uniformly converted to grayscale format.

[0083] S22: For the 140 frames of corneal deformation images extracted in step S21, further image preprocessing operations are performed to remove irrelevant areas and standardize the image size; the useless information in the Corvis ST video has a fixed position. The corneal contour area is located in each frame of the image, a minimum rectangular bounding box containing all corneal contour pixels is constructed, and the image is automatically cropped based on this.

[0084] S23: Perform SLIC superpixel pre-segmentation on each cropped image frame, assign the same semantic label to small regions with obviously the same semantic information, and obtain multiple pixel groups and their corresponding classification labels.

[0085] In a more preferred embodiment, step S23, SLIC superpixel segmentation, includes the following sub-steps:

[0086] S231: Convert the image from RGB space to LAB space. Set the number of superpixels to K. Divide the image with N pixels into K superpixels of the same size. The size of each superpixel should be N / K, and the distance between adjacent seed points is approximately... .

[0087] S232: Calculate the gradient values ​​of all pixels in a neighborhood, and move the seed point to the location with the smallest gradient in that neighborhood to avoid the seed point falling on the contour boundary with a large gradient, which would affect the subsequent clustering effect. The formula for calculating the gradient by difference is as follows:

[0088]

[0089]

[0090] .

[0091] in, The gradient value of a pixel. The two-dimensional coordinates of the pixel. and Each pixel The difference in grayscale values ​​in the horizontal and vertical directions. The grayscale value of a pixel. and Each pixel The gray values ​​of the right-hand adjacent pixels (horizontally adjacent) and the bottom adjacent pixels (vertically adjacent).

[0092] S233: For each searched pixel, calculate its distance to the seed point. The distance calculation method is as follows: Represents color distance. Represents spatial distance, through and Two parameters control the proportional distribution of the two distances:

[0093]

[0094]

[0095] .

[0096] in, These are the corresponding components of the LAB color space for the seed point. These are the corresponding components of the LAB color space for the pixels to be assigned. The seed point has two-dimensional coordinates in the image. The two-dimensional coordinates of the pixel to be assigned in the image. This is the combined distance between the pixel and the seed point.

[0097] S234: Traverse the pixels within a 2S×2S region surrounding the center point of each superpixel block, calculate which superpixel block's center point each pixel in the region is closest to, and assign it to that superpixel block; after completing one iteration, recalculate the center point coordinates of each superpixel block and iterate again.

[0098] S24: Design a convolutional neural network to classify each pixel of the input image, making the output semantic segmentation result as similar as possible to the SLIC pre-segmentation result. Based on the pre-segmentation result, the neural network merges small blocks with the same semantic information to obtain larger blocks. Through multiple iterations, the merging of small blocks is gradually achieved, finally obtaining the expected semantic segmentation result. Its network structure is as follows: Figure 2 As shown.

[0099] The convolutional neural network (CNN) structure consists of three layers: Convolutional Layer 1 (3×3 kernel) activated by BatchNorm and ReLU layers; Convolutional Layer 2 (3×3 kernel) activated by BatchNorm and ReLU layers; and Convolutional Layer 3 (3×3 kernel) activated by BatchNorm. The output of Convolutional Layer 3 is batch normalized and then used to obtain the predicted class index via a maximum index function, which serves as the final output. Simultaneously, the normalized exponential function Softmax is used to calculate the class probability distribution, providing feedback input to the maximum index function for loss calculation. During training, SDG stochastic gradient descent is used to update the model parameters.

[0100] S25: When the loss function value corresponding to the segmentation result meets the preset value and the segmentation category meets the target value, stop the model training and output the image segmentation result; the loss function value is used to characterize the loss value between the predicted clustering label and the pre-classification label of the pixel group output by the corneal image segmentation module, and is calculated using the softmax function.

[0101] S26: In this embodiment, due to the slow morphological changes and structural similarities between adjacent frames in the Corvis ST image sequence, a training strategy with an inter-frame parameter inheritance mechanism is designed. When training the first frame of the corneal image sequence, the convolutional neural network semantic segmentation model is constructed, and the network weight parameters are randomly initialized. From the second frame onwards, during training for each frame, the network parameters are not re-initialized; instead, the network weights from the previous frame are loaded as the initial parameters for the current frame, avoiding repeated learning of similar features. This process is executed cyclically throughout the entire 140-frame image sequence. The network parameters are continuously updated and passed to the next frame during training in each frame, forming a continuous temporal training path, thereby achieving efficient learning of the image sequence segmentation model.

[0102] S27: Contour extraction uses a connected component contour extraction method. Edge detection and contour tracking are performed on each frame of semantic segmentation image to obtain pixel-level continuous contour lines. The extracted contour lines are mapped to two-dimensional coordinate data according to the image coordinate system, and their positions in the horizontal direction and contour values ​​in the height direction are recorded respectively. Each frame of image should contain two contour lines, corresponding to the anterior and posterior surfaces of the cornea respectively. The contour information of all 140 frames of images is arranged in chronological order. The contour data of the anterior and posterior surfaces of each frame of image together form a two-dimensional cross section (width × number of channels), and are stacked along the time axis to form a three-dimensional tensor. The final generated contour three-dimensional data matrix has a size of 140×576×2, where the first dimension corresponds to the image frame order, the second dimension corresponds to the number of sampling points in the horizontal direction of the image, and the third dimension corresponds to the contour height values ​​of the anterior and posterior surfaces of the cornea respectively.

[0103] S28: Using the first frame of the corneal video sequence as the reference frame, extract the positions of the anterior and posterior surfaces at the center of the image in that frame, and record them as the center height of the anterior surface. and rear surface center height That is, for any 1st Frame Image , its first The height value of the column pixel is transformed into , After normalization, a three-dimensional contour data matrix is ​​obtained that retains the original structure but has a relative height difference, with a size of 140×576×2.

[0104] Step S3, Prediction Network Construction and Training: Build a corneal cross-linking surgery prediction model based on a deep neural network, use the images obtained in S2 to train the deep neural network, use the trained network to predict the three-dimensional data matrix after corneal cross-linking surgery, and process it into video sequence images for visualization.

[0105] S31: Standardize the data input and output format, and store pairs of input matrices and prediction target matrices in batches in npz format. Each file contains an input image tensor and a corresponding target image tensor, corresponding to fields x and y, respectively.

[0106] S32: Read the input field x and output field y from the file, and perform min-max normalization on the values ​​in each tensor to scale all data values ​​to the range [0, 1]. The normalization calculation formula is as follows: , This is the original height value. and These are the minimum and maximum values ​​in the current tensor, respectively. The normalized image tensor is converted to floating-point (float32) format to ensure numerical accuracy and compatibility with neural network computation requirements. The original data dimension is converted from 140×576×2 to 2×140×576, with the channel dimension placed first to adapt to the input requirements of the convolution module in the mainstream deep learning framework PyTorch.

[0107] S33: Construct a regression neural network model based on an encoder-decoder network structure. The overall structure includes an encoder module, a feature bottleneck module, a decoder module, and an output module. The encoder module consists of multiple downsampling convolutional units, each containing two convolutional layers connected to BatchNorm and ReLU respectively. Max pooling is used to gradually reduce the spatial resolution of the feature map while increasing the number of channels, thereby achieving high-dimensional semantic feature extraction from the input image. The feature bottleneck module connects the encoder and decoder, employing a deeper convolutional structure to enhance feature representation. The feature bottleneck module contains multiple convolutional blocks to integrate the multi-scale features extracted by the encoder and compress them into a low-dimensional compact representation, enhancing the model's ability to express complex spatial patterns. The decoder module upsamples the bottleneck features step by step to restore the spatial size of the original image (preferably by upsampling through transposed convolution to restore spatial resolution). After each upsampling, a skip connection is made with the feature map of the corresponding encoder layer to compensate for the spatial details lost during upsampling. The last layer is a 1×1 convolution, used to restore the number of channels in the feature map to 2, corresponding to the predicted values ​​of the anterior and posterior surfaces of the cornea, respectively. Its network structure is as follows Figure 3 As shown.

[0108] S34: The neural network model is trained using supervised learning, with paired input and target images as training samples. The mean squared error loss function is selected, and its expression is: The network parameters are updated through backpropagation, a mini-batch training strategy is used, and a stochastic gradient descent optimizer is combined to iteratively train the network until the loss converges.

[0109] S35: Preprocess the image to be predicted in the same way as in the training phase, convert it into a tensor format of 2×140×576 and feed it into the trained neural network model to obtain the model output. The output is a prediction tensor of size 2×140×576 and the output is saved. The output can be saved in npz format for subsequent analysis or visualization.

[0110] Step S4: Predicted Biomechanical Parameter Calculation: The image obtained in S3 is restored to the anterior and posterior corneal surface contours arranged in time. Biomechanical parameters are calculated based on the contours to obtain the predicted post-crosslinking biomechanical characteristics.

[0111] S41: Perform channel separation on the predicted image tensor (size 2×139×576) obtained in step S3, corresponding to the anterior and posterior surface height images of the cornea respectively.

[0112] S42: Based on the reconstructed corneal deformation sequence, key structural indicators at each time point are extracted, including maximum flattening length, first flattening time, and second flattening time; such as... Figure 4 As shown, this is a visualization of the three-dimensional data matrix of the anterior (Predict - Upper Surface) and posterior (Predict - Lower Surface) corneal surfaces after corneal cross-linking surgery, as predicted in the experiment.

[0113] S43: Compare the predicted biomechanical parameters with the actual measurement data to evaluate the structural enhancement effect after corneal cross-linking and predict the risk of future disease progression.

[0114] In summary, this invention proposes a method for predicting the outcome of corneal cross-linking surgery based on Corvis ST video data. The method includes acquiring Corvis ST video data of keratoconus patients before and after corneal cross-linking surgery and creating a dataset. Based on the dataset, the corneal contour of each frame in the video is extracted, and the corneal contours of each frame are integrated into a whole according to the time axis. Using the height of the central cornea in the first frame as a benchmark, the height difference of each pixel is calculated to obtain a three-dimensional data matrix. A method for predicting the outcome of corneal cross-linking surgery based on a deep neural network is constructed. The neural network is trained using the three-dimensional data matrix, and the trained network is used to predict the three-dimensional data matrix after corneal cross-linking surgery. The predicted three-dimensional data matrix is ​​then restored to the corneal surface contours arranged according to time, and biomechanical parameters are calculated based on the contours to obtain the predicted biomechanical characteristics after cross-linking surgery. In conclusion, this method can predict the biomechanical changes of the cornea in different patients after corneal cross-linking surgery, providing a more accurate tool for predicting corneal cross-linking surgery. It provides a data-driven simulation mechanism for the formulation of preoperative corneal cross-linking surgery plans, and the achieved preoperative outcome pre-assessment can lay the foundation for adjusting personalized surgical parameters.

[0115] This invention also provides a storage medium for storing a computer program, which, when executed, performs at least the methods described above.

[0116] This invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein the processor executes the computer program by performing at least the method described above.

[0117] This invention also provides a processor that executes a computer program, at least performing the methods described above.

[0118] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc or CD-ROM; magnetic surface memory can be disk storage or magnetic tape storage. The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0119] In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0120] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0121] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0122] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0124] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0125] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0126] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0127] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or application, should be considered within the scope of protection of the present invention.

Claims

1. A method for predicting the outcome of corneal cross-linking surgery, characterized in that, Includes the following steps: S1. Obtain dynamic video data of corneal biomechanics before and after corneal cross-linking surgery for multiple patients and create a dataset; S2. Based on the dataset, extract the corneal contour of each frame in the video, arrange the corneal contour of each frame according to the time axis and integrate them into a whole. Taking the height of the cornea at the center of the first frame as the reference, calculate the height difference of each pixel to obtain a three-dimensional data matrix. S3. Build a corneal cross-linking surgery prediction model based on deep neural network, train the deep neural network using the three-dimensional data matrix, and use the trained network to predict the three-dimensional data matrix after corneal cross-linking surgery. S4. Restore the predicted three-dimensional data matrix into the corneal anterior and posterior surface contours arranged according to time, calculate biomechanical parameters based on the contours, and obtain the predicted post-crosslinking biomechanical characteristics. Step S2 specifically includes: The input video is parsed to extract multiple frames of images from it. For each frame of image, the region containing all corneal contour pixels is automatically extracted and cropped into an image of the same size; Superpixel pre-segmentation is performed on each cropped image, and the same semantic label is assigned to small regions with the same semantic information; Design a convolutional neural network to classify each pixel of the input image so that the semantic segmentation result matches the pre-segmentation result, and obtain the target semantic segmentation result by iteratively merging small regions with the same semantics; When the loss function value of the segmentation result meets the preset condition, the model training is stopped and the image segmentation result is output. When training the image sequence, the first frame image uses randomly initialized network parameters, and subsequent frames image use the network parameters trained on the previous frame image as initial parameters for training. Extract the contours of the segmented images, arrange the contours of multiple frames in chronological order, and integrate them into a three-dimensional data matrix; Using the center height of the anterior and posterior surfaces of the cornea in the first frame as a reference, for each pixel in each subsequent frame, the height difference between its height and the corresponding reference is calculated to obtain a three-dimensional data matrix of relative height difference.

2. The method for predicting the effect of corneal cross-linking surgery as described in claim 1, characterized in that, Step S1 specifically includes: Acquire dynamic video data of corneal biomechanics in keratoconus patients before and after corneal crosslinking surgery; A corneal video dataset of corneal cross-linking surgery samples was created. The dataset includes data from multiple keratoconus patients, with each patient's data including pre-cross-linking corneal biomechanical dynamic detection video data and post-cross-linking corneal biomechanical dynamic detection video data.

3. The method for predicting the effect of corneal cross-linking surgery as described in claim 1, characterized in that, The superpixel pre-segmentation includes: Convert the image from the RGB color space to the LAB color space; Set the number of superpixels, and calculate the initial size and seed point spacing for each superpixel based on the total number of pixels in the image; Calculate the pixel gradient in the neighborhood of each seed point, and move the seed point to the position with the minimum gradient. For each pixel, calculate its color distance and spatial distance from the seed point, and combine the two according to preset parameters to calculate the total distance, so as to assign the pixel to the nearest superpixel; Iteratively update the superpixel center and redistribute pixels until convergence.

4. The method for predicting the effect of corneal cross-linking surgery as described in claim 1, characterized in that, When performing semantic segmentation on training image sequences, the first frame uses randomly initialized network parameters. Starting from the second frame, each frame uses the network parameters trained on the previous frame as initial parameters for training, in order to accelerate the training process by utilizing inter-frame similarity.

5. The method for predicting the effect of corneal cross-linking surgery as described in claim 1, characterized in that, Step S3 specifically includes: Standardize the data input and output formats, and store pairs of input matrices and prediction target matrices in batches according to preset formats; The data is normalized, the data values ​​are scaled to a preset range, and the channel dimensions are adjusted to fit the neural network input. Construct a regression neural network model based on an encoder-decoder network structure; The neural network model is trained using supervised learning, with paired input and target images as training samples. The mean squared error loss function is selected, and the network parameters are updated through backpropagation. The network is iteratively trained using a mini-batch training strategy and a stochastic gradient descent optimizer until the loss converges. The image to be predicted is preprocessed in the same way as in the training phase, converted into tensor format, and fed into the trained neural network model to obtain the output result.

6. The method for predicting the effect of corneal cross-linking surgery as described in claim 5, characterized in that, In the regression neural network model based on the encoder-decoder network structure, the encoder module extracts features step by step through multiple downsampling convolutional units. Each convolutional unit contains convolution operations, batch normalization, and activation functions, and reduces the spatial resolution of the feature map through pooling operations. The decoder module recovers the feature map size through upsampling operations and combines it with skip connections to fuse shallow features from the encoder; The output module adjusts the number of channels through convolutional layers to match output requirements.

7. The method for predicting the effect of corneal cross-linking surgery as described in claim 1, characterized in that, Step S4 specifically includes: The predicted 3D data matrix is ​​channel-separated to obtain height images of the anterior and posterior surfaces of the cornea, respectively. Based on the reconstructed corneal deformation sequence, key biomechanical parameters at each time point were extracted; By comparing predicted biomechanical parameters with actual measurement data, the structural enhancement effect and risk of disease progression after corneal cross-linking surgery can be assessed.

8. The method for predicting the effect of corneal cross-linking surgery as described in claim 7, characterized in that, The key biomechanical parameters mentioned in step S4 include at least one of the maximum flattening length, the first flattening time, and the second flattening time.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the corneal cross-linking surgery effect prediction method as described in any one of claims 1 to 8.

Images