Intraoperative and postoperative prediction method, device, medium and program product based on MMC-DM for SMILE

Abstract

The application discloses a SMILE intraoperative and postoperative prediction method and device based on MMC-DM, a medium and a program product, and the method comprises the following steps: using a fusion encoder to perform feature extraction and double-layer fusion mechanism fusion on preprocessed corneal topography and clinical baseline data, taking the output multi-modal feature vector as a condition, first constructing a three-stage cascade diffusion generation model based on a latent diffusion model, combining Controlnet to generate an attraction frame, an edge cutting completion frame and a laser scanning completion frame, then combining the generated visual features of the attraction frame to construct a postoperative numerical prediction model, and respectively outputting a postoperative equivalent spherical numerical value and a lens actual cutting thickness numerical value; a double-dimension quantitative evaluation system conforming to a clinical visual angle is established, quantitative evaluation is performed on the generated image and the real image of the laser scanning completion frame, and quantitative indexes of the lens edge trajectory dimension and the OBL shape dimension are calculated. The application can simulate the dynamic process of the operation, fully fuse multi-modal data, improve the postoperative prediction accuracy, and provide a quantitative basis.

Description

Technical Field

[0001] This invention relates to the field of ophthalmic medical image analysis technology, and in particular to methods, equipment, media and program products for intraoperative and postoperative prediction of SMILE (Small Incision Lenticule Extraction) based on MMC-DM (Multi-Modal Cascade-Diffusion Model). Background Technology

[0002] SMILE, as an advanced flapless minimally invasive refractive correction procedure, encompasses a comprehensive clinical process including preoperative multidimensional examination, intraoperative laser ablation, and postoperative outcome evaluation. Preoperatively, a thorough screening of the patient's eyes is conducted, and the acquired clinical baseline data is precisely input into the femtosecond laser surgery system. During the procedure, an arc-shaped negative pressure suction ring is used to fix the eyeball, and a femtosecond laser scans within the corneal stroma to create a microlens. The surgeon then separates and removes the microlens through a tiny 2mm-4mm incision, thereby achieving refractive correction by altering corneal curvature. Postoperatively, follow-up is conducted using multiple indicators such as visual acuity, refractive error, and visual quality to quantitatively evaluate the final surgical outcome.

[0003] However, due to individual differences in corneal biomechanics and the complex effects of laser physics, intraoperative complications often include abnormal accumulation of OBL (Opaque Bubble Layer) obstructing the surgical field, increasing the difficulty of lenticule separation, and the risk of tissue residue or tearing due to rough lenticule edges. Therefore, the ability to visually predict these potential intraoperative morphological features before surgery is of significant clinical importance in assisting surgeons to avoid operational risks and optimize surgical planning.

[0004] With the rapid development of artificial intelligence (AI) technology, its application in ophthalmology has expanded from basic auxiliary functions to complex and in-depth diagnosis and treatment. In visual analysis and image generation, existing deep learning technologies, such as CNNs (Convolutional Neural Networks) and Transformers, are widely used for fundus lesion identification, anatomical structure segmentation, and disease grading diagnosis. Meanwhile, generative techniques based on GANs (Generative Adversarial Networks) and DMs (Diffusion Models) are frequently used for medical image data augmentation and quality enhancement. In clinical numerical prediction, relying on the accumulation of massive amounts of clinical data, machine learning algorithms can now achieve regression prediction of postoperative visual acuity and corneal biomechanical parameters, as well as efficient screening for keratoconus. These technologies provide a certain degree of objective quantitative decision-making basis for clinical diagnosis and treatment.

[0005] Nevertheless, existing research primarily focuses on "image-to-label" assisted diagnosis or "data-to-numerical" prognostic assessment, lacking an integrated technical solution capable of deeply fusing preoperative multimodal data to generate high-fidelity intraoperative keyframe microscopic images across modalities and simultaneously and accurately predict postoperative visual quality. The limitations of current technology make it difficult to simulate the deformation patterns of the eyeball under negative pressure and the physical texture characteristics after laser scanning, restricting surgeons' accurate prediction of lenticule separation and potential visual interference risks, and to some extent hindering further improvements in the predictability and safety of surgical procedures.

[0006] Meanwhile, existing technologies suffer from a significant disconnect between postoperative prediction and intraoperative morphological prediction: they separate intraoperative morphological analysis and postoperative outcome prediction into two independent tasks, failing to establish an intrinsic link between them. Key morphological features such as intraoperative lens edge morphology and OBL distribution directly affect postoperative refractive correction and visual quality, but existing methods do not incorporate this intraoperative morphological information into the postoperative prediction model, relying solely on preoperative data for postoperative numerical regression. This results in a lack of intraoperative morphological support for postoperative prediction, hindering further improvement in prediction accuracy and preventing the achievement of a closed-loop process encompassing "preoperative prediction - intraoperative simulation - postoperative prediction."

[0007] This invention addresses the aforementioned core deficiencies by constructing a full-process collaborative mechanism through modular design and feature reuse, fully realizing closed-loop support for "preoperative prediction - intraoperative simulation - postoperative prediction," and providing an integrated solution for the entire surgical process planning. Summary of the Invention

[0008] This invention provides a method, device, medium, and program product for intraoperative and postoperative prediction of SMILE based on MMC-DM, which solves the technical problems of existing technologies such as difficulty in simulating the dynamic process of surgery, insufficient fusion of multimodal data, and disconnect between postoperative prediction and intraoperative morphological prediction.

[0009] In a first aspect, the present invention provides a method for intraoperative and postoperative prediction of SMILE based on MMC-DM, comprising: After preprocessing the corneal topography and clinical baseline data of patients before SMILE surgery, a fusion encoder is used to extract visual feature maps from the preprocessed corneal topography and extract numerical feature vectors from the clinical baseline data. After fusion through a two-layer fusion mechanism of shallow adaptive modulation and deep cross-attention fusion, a multimodal feature vector is output. A three-stage cascaded diffusion generation model is constructed based on the potential diffusion model. Using multimodal feature vectors as conditions, and combined with Controlnet, the image generated in the previous stage is used as a spatial constraint to guide the image generation in the next stage. Intraoperative key frames that conform to the SMILE surgical timing logic are generated in stages, including suction frames, edge cutting completion frames, and laser scanning completion frames. By reusing the multimodal feature vector and combining it with the visual features of the generated attraction frame, a postoperative numerical prediction model is constructed, which outputs the postoperative equivalent spherical lens value and the actual lens cutting thickness value, respectively. A two-dimensional quantitative evaluation system that conforms to the clinical perspective is established to quantitatively evaluate the generated image and the real image of the completed laser scan frame. First, the root mean square error and edge roughness difference of the lens edge trajectory dimension are calculated, and then the Dice similarity coefficient and local SSIM of the OBL morphology dimension are calculated.

[0010] Optionally, when the fusion encoder extracts visual feature maps, it uses the Swin Transformer as the backbone network to perform multi-scale feature encoding on the preprocessed corneal topography map, extracts the spatial feature maps of encoder residual layer 3, encoder residual layer 4 and encoder residual layer 5, and establishes a hierarchical visual feature map of the corneal topography map. When the fusion encoder extracts numerical feature vectors, it uses a random Gaussian matrix with fixed weights to project the preprocessed clinical baseline data into a high-dimensional space, and then performs nonlinear mapping through a multilayer perceptron to generate numerical feature vectors with high-frequency response characteristics. The shallow adaptive modulation process of the fusion encoder includes: aggregating and compressing numerical feature vectors to generate global physical prior vectors; using adaptive group normalization to predict affine transformation parameters through the global physical prior vectors; and injecting the parameters layer by layer into visual feature maps at different levels to complete the recalibration of image feature distribution. The deep cross-attention fusion process of the fusion encoder includes: initializing a learnable visual query vector, using the visual query vector as Q, and the modulated visual feature map as K and V; retrieving and aggregating key visual information in the visual feature map through a cross-attention mechanism; concatenating the aggregated visual query vector with the numerical feature vector, and then completing cross-modal information exchange through a self-attention mechanism; the continuous process of cross-attention aggregation and self-attention interaction constitutes a Q-transformer block, and after iterative fusion of N Q-transformer blocks, a multimodal feature vector is output.

[0011] Optionally, when constructing the three-stage cascaded diffusion generation model, a stable diffusion model is used as the baseline model. Its pre-trained variational autoencoder is used to map the image to a low-dimensional latent space for denoising, and the encoder and decoder parameters of the variational autoencoder are kept frozen throughout the process. The three-stage generation process adopts a differentiated training strategy. In the first stage, no ControlNet is involved, the fusion encoder is trained, and the parameters of the UNet backbone network are fine-tuned. In the second and third stages, the corresponding ControlNets are introduced respectively, and only the ControlNet parameters of the current stage are trained, while all model parameters of the previous stages are frozen.

[0012] Optionally, the specific implementation process of the postoperative numerical prediction model includes: Obtain the multimodal feature vector output by the fusion encoder; The generated attraction frames are input into the ResNet network to extract high-dimensional visual feature maps and unfold them. The unfolded one-dimensional visual feature vectors are concatenated with the multimodal feature vectors and used as input features for the postoperative numerical prediction model. The postoperative numerical prediction model is based on a multilayer perceptron and outputs the postoperative equivalent spherical lens value and the actual lens cutting thickness value through two independent decoupled prediction heads.

[0013] Optionally, before calculating the index of the lens edge trajectory dimension, the laser scanning completion frame is first converted from the Cartesian coordinate system to the polar coordinate system, and the lens edge trajectory is extracted by the sub-pixel gradient detection algorithm based on guide line constraint; before calculating the index of the OBL morphology dimension, the texture contrast of the OBL region is first enhanced by the contrast-limited adaptive histogram equalization algorithm, and then the OBL region is binarized to obtain the OBL binarized mask image and its morphological features are extracted.

[0014] Optionally, the formulas for calculating the root mean square error and the edge roughness difference are as follows: , , , In the above formula, This represents the root mean square error. This represents the total number of sampling angles in polar coordinates. This is the current angle index. and These represent the angle index as The sub-pixel radial distance of the generated edge trajectory. This indicates the roughness of a single edge. These represent the angle index as The subpixel-level radial distance of the real edge trajectory. Indicates that at the angle index is The original edge radial coordinate sequence extracted at that time. Indicates that at the angle index is The macroscopic contour sequence after smoothing and fitting. Indicates the difference in edge roughness. and These represent the roughness of the generated edge and the real edge, respectively.

[0015] Optionally, the formulas for calculating the Dice similarity coefficient and local SSIM are as follows: , , In the above formula, Represents the Dice similarity coefficient. and These represent the OBL mask sets obtained after binarizing the real image and the generated image, respectively. This represents the total number of pixels in the set with a value of 1. This represents the total number of pixels in the spatially overlapping area between the actual bubble region and the generated bubble region. and These represent the real OBL image and the generated OBL image after enhancement using the contrast-limited adaptive histogram equalization algorithm, respectively. Indicates local SSIM, and Image used to reflect brightness and The local pixel mean, and Represents an image used to reflect contrast. and The local pixel variance, Images used to reflect the degree of structural similarity and Local covariance, and The first and second constants are very small and introduced to maintain the stability of the denominator.

[0016] Secondly, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the aforementioned SMILE intraoperative and postoperative prediction method based on MMC-DM.

[0017] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the aforementioned SMILE intraoperative and postoperative prediction method based on MMC-DM.

[0018] Fourthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned MMC-DM-based SMILE intraoperative and postoperative prediction method.

[0019] One or more technical solutions provided by this invention have at least the following technical effects or advantages: 1. Constructing a unified multimodal feature fusion framework for multi-task reuse to achieve deep fusion of heterogeneous multimodal data and cross-task feature reuse. Addressing the limitations of existing technologies in fusing unstructured images (corneal topography) and structured clinical data (such as refractive power and biomechanical parameters), this invention proposes a fusion encoder (FusedEncode). Through a two-layer fusion mechanism of shallow adaptive modulation and deep cross-attention fusion, it achieves deep fusion and feature interaction of heterogeneous multimodal data, innovatively realizing unified extraction and cross-task reuse of underlying representations: In the intraoperative generation task, the multimodal feature vector output by the fusion encoder serves as a multimodal conditional injection diffusion model, accurately guiding image synthesis; in the postoperative prediction task, this multimodal feature vector can be seamlessly reused and directly regressed to relevant postoperative values ​​through the prediction head. This design not only effectively reduces model parameter redundancy but also establishes deep collaboration between visual features and physical parameters. By transferring fine-grained morphological semantics learned in the generation task to assist numerical regression, it effectively solves the technical problem of the disconnect between postoperative prediction and intraoperative morphological prediction, thereby improving postoperative prediction accuracy.

[0020] 2. Achieving high-fidelity temporal image generation of the surgical process based on a cascaded architecture combined with ControlNet technology. Addressing the continuous dynamic characteristics of ocular negative pressure adsorption deformation and laser physical ablation during SMILE surgery, this invention overcomes the limitations of traditional single-frame independent generation by constructing a three-stage cascaded diffusion generation model. This model uses ControlNet, taking the image generated in the previous stage as a spatial constraint to guide the generation of the image in the next stage. This design strictly follows the causal logic of the surgical operation, ensuring the temporal continuity of corneal tissue in terms of geometric morphology and texture details, solving the problem that existing generation technologies struggle to simulate the dynamic process of surgery, and achieving high-fidelity visual simulation of the entire surgical process.

[0021] 3. Establish a two-dimensional quantitative evaluation system for key morphological features of SMILE surgery to achieve image quality evaluation from a clinical perspective. Given that general image metrics such as PSNR (Peak Signal-to-Noise Ratio) and Global SSIM (Global Structural Similarity Index) are insufficient to accurately reflect key clinical details during surgery, this invention establishes a two-dimensional quantitative evaluation system consistent with a clinical perspective. For lens edge trajectories, a gradient sub-pixel edge detection algorithm based on guide line constraints is proposed. Edge detail similarity is evaluated by calculating RMSE (Root Mean Square Error) and edge roughness differences. For OBL morphology, morphological features are extracted through image enhancement and segmentation algorithms. The Dice Similarity Coefficient and Local SSIM (Local Structural Similarity Index) of the OBL region are comprehensively calculated, providing objective and accurate quantitative evidence for verifying the clinical reference value of the generated images. Attached Figure Description

[0022] Figure 1 The flowchart below shows a method for intraoperative and postoperative prediction of SMILE based on MMC-DM according to the present invention. Figure 2 This is a processing logic diagram of the fusion encoder of the present invention; Figure 3 The processing logic diagram for the three-stage cascaded diffusion generation model; Figure 4 This is a processing logic diagram for the postoperative numerical prediction model; Figure 5 A comparison of real and generated images of the attraction frame, the edge-cutting completed frame, and the laser scanning completed frame; Figure 6 This is a schematic diagram for evaluating the lens edge trajectory in a specific example; Figure 7 This is a schematic diagram of OBL morphology evaluation in a specific example. Detailed Implementation

[0023] This invention provides a method, device, medium, and program product for intraoperative and postoperative prediction of SMILE based on MMC-DM, which solves the technical problems of existing technologies such as difficulty in simulating the dynamic process of surgery, insufficient fusion of multimodal data, and disconnect between postoperative prediction and intraoperative morphological prediction.

[0024] To better understand, a detailed description will be provided below with reference to the accompanying drawings and specific embodiments. Obviously, the embodiments described in this invention are only a part of the embodiments of this invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0025] This invention provides a method for intraoperative and postoperative prediction of SMILE surgery based on MMC-DM. Based on an artificial intelligence deep learning architecture, it deeply fuses visual feature maps and numerical feature vectors. It extracts and fuses heterogeneous features through a fusion encoder and combines a three-stage cascaded diffusion generation model to generate key frames in SMILE surgery in a temporal sequence. At the same time, it reuses multimodal feature vectors to achieve accurate prediction of relevant postoperative values ​​and establishes a two-dimensional quantitative evaluation system that conforms to the clinical perspective to verify the quality of the generated images. This provides comprehensive technical support for preoperative planning and risk avoidance of SMILE surgery.

[0026] This method comprises five core steps. Using a fusion encoder as the core feature extraction module, it generates a multimodal feature vector containing both physical and visual semantics by deeply fusing visual feature maps and numerical feature vectors. This multimodal feature vector is then injected into a three-stage cascaded diffusion generation model to generate three temporally coherent intraoperative keyframes: the attraction frame, the edge cutting completion frame, and the laser scanning completion frame. Furthermore, this multimodal feature vector is reused in a postoperative numerical prediction model. Combined with the generated attraction frame visual features, it predicts the postoperative spherical equivalent (SE) and the actual lens ablation thickness, thus achieving a complete closed-loop process of "preoperative prediction - intraoperative simulation - postoperative prediction," enabling synergistic linkage and efficient reuse of each stage.

[0027] Step S1: Multimodal data preprocessing.

[0028] The core function of this step is to standardize the heterogeneous multimodal data before SMILE, eliminate differences in data units and formats, and construct unified model input data, laying the foundation for subsequent feature fusion. The specific implementation steps are as follows: Step S11, Visual modality data processing: Obtain three preoperative corneal topography maps (axial curvature map, tangential curvature map, and corneal thickness map) of the patient, perform channel stitching processing on them, and then uniformly adjust the image resolution to obtain a standardized corneal topography map, which is used as the visual modality input.

[0029] Step S12, Construction of Structured Clinical Baseline Data: Construct clinical baseline data containing 20 dimensions of features, divided into the following three categories according to feature attributes, comprehensively covering the core influencing factors before SMILE surgery: The first category is demographic characteristics, which includes two characteristics: gender and age. The second category includes preoperative clinical physiological parameters, which include corneal diameter, anterior chamber depth, corneal aspheric coefficient, thinnest point corneal thickness, non-contact intraocular pressure, flat curvature, steep curvature, meridian direction of steep curvature, preoperative spherical power, preoperative cylindrical power, and preoperative astigmatic axis, totaling 11 features. The third category is surgical design and input parameters, which include seven features: the corrected spherical power, corrected cylindrical power, corrected astigmatism axis, corneal cap thickness, optical zone diameter, remaining corneal stromal bed thickness, and lenticule ablation thickness of the femtosecond laser surgery system.

[0030] Step S13, Numerical Feature Standardization: The clinical baseline data with the above 20 features are standardized using Z-scores to obtain standardized 20-dimensional structured clinical baseline data. This eliminates differences between different physical units and ensures the stability of model training. The calculation formula is: In the formula These are the standardized eigenvalues. These are the original eigenvalues. The characteristic mean, The characteristic standard deviation is denoted as .

[0031] Step S2: Feature extraction and feature fusion.

[0032] The fusion encoder in this step is the core feature extraction module of this invention. Its core function is to achieve deep fusion of unstructured corneal topographic image features and structured clinical baseline numerical features, generating a multimodal feature vector that includes visual semantics of corneal morphology and clinical physiology, as well as physical semantics of surgical parameters. This provides a unified feature representation for subsequent intraoperative image generation and postoperative numerical prediction. Figure 2 As shown, the implementation of this module consists of three sub-steps: visual feature encoding, numerical feature encoding, and heterogeneous feature fusion, as detailed below: Step S21, Visual Feature Encoding: For the standardized corneal topography processed in Step S1, SwinTransformer is used as the backbone network to encode its multi-scale features. Spatial feature maps containing shallow texture, mid-level structure and deep semantics are extracted respectively. Corresponding to the levels of encoder residual layer 3, encoder residual layer 4 and encoder residual layer 5 of SwinTransformer, a hierarchical visual feature map of corneal topography is established, that is, a multi-scale visual feature vector.

[0033] Step S22, Numerical Feature Encoding: For the 20-dimensional structured clinical baseline data standardized in Step S1, a Fourier high-frequency mapping algorithm is designed to realize high-dimensional mapping and nonlinear encoding of numerical features, enhancing the model's sensitivity to small numerical changes. Specifically, firstly, a random Gaussian matrix with fixed weights is used to project the standardized 20-dimensional structured clinical baseline data into a high-dimensional space, converting it into a high-dimensional feature vector containing high-frequency sine and cosine components; then, this high-dimensional feature vector is input into an MLP (Multi-Layer Perceptron) for nonlinear mapping to generate a numerical feature vector with high-frequency response characteristics.

[0034] Step S23, Heterogeneous Feature Fusion: This is divided into two levels: shallow fusion and deep fusion, to achieve comprehensive interaction and fusion between visual feature maps and numerical feature vectors, and finally output multimodal feature vectors.

[0035] Step S231, shallow fusion: The above numerical feature vectors are aggregated and compressed to generate a global physical prior vector; then, the AdaGN (Adaptive Group Normalization) method is used to predict the affine transformation parameters (scaling factor and offset factor) using the global physical prior vector, and these parameters are injected layer by layer into the visual feature maps of encoder residual layer 3, encoder residual layer 4 and encoder residual layer 5 extracted by SwinTransformer. By recalibrating the image feature distribution, the numerical features are used to achieve shallow modulation of the visual feature map.

[0036] Step S232, Deep Fusion: First, a set of learnable visual query vectors is initialized, with the visual query vectors as Q (query) and the modulated visual feature maps as K (key) and V (value). The key visual information in the visual feature maps is retrieved and aggregated using a cross-attention mechanism. Then, the aggregated visual query vectors are concatenated with the numerical feature vectors, and cross-modal information exchange is completed through a self-attention mechanism. The continuous process of cross-attention aggregation and self-attention interaction constitutes a Q-transformer block. After iterative fusion of N Q-transformer blocks, the final output is a feature vector containing multimodal semantics, i.e., a multimodal feature vector.

[0037] Step S3: Generate cascaded surgical timing keyframes.

[0038] The core function of this step is to construct a three-stage cascaded diffusion generation model based on the latent diffusion model. Using the multimodal feature vector output from step S2 as a condition, it generates intraoperative keyframes conforming to the SMILE surgical time-series logic in stages, achieving high-fidelity visual simulation of the dynamic surgical process. The latent diffusion model is the core underlying technical framework of this invention's three-stage cascaded diffusion generation model, providing the core logic of "gradually denoising and generating images from random noise." The baseline model is the specific implementation carrier selected under this framework. This scheme uses the Stable Diffusion (SD) model as the baseline model, utilizing its pre-trained VAE (Variational Autoencoder) to map the image to a low-dimensional latent space for efficient denoising. The encoder and decoder parameters of the VAE are always kept frozen to avoid the loss of pre-training knowledge. Figure 3 As shown, this step is specifically divided into three stages: suction frame generation, edge cutting completion frame generation, and laser scanning completion frame generation. Each subsequent stage uses the image generated in the previous stage as a spatial constraint, strictly following the causal logic of the surgical operation. The specific implementation is as follows: Step S31, Attract Frame Generation Stage.

[0039] Step S311: Starting with random noise, the initial noise state in the latent space is obtained by mapping through a VAE encoder. The multimodal feature vectors extracted in step S2 are used as conditions and injected into the backbone network of Unet through a cross-attention mechanism.

[0040] Step S312, in the low-dimensional latent space of the VAE, Unet combines the condition pairs Predictive denoising yields features from the previous time step. After that The process involves iterative denoising to gradually remove noise, ultimately yielding fully denoised latent features. This potential feature Finally, the VAE decoder reconstructs and generates the intraoperative suction frame of the SMILE surgery (the morphological image of the eyeball after being suctioned by negative pressure).

[0041] Step S313: This stage focuses on training the fusion encoder and fine-tuning the backbone parameters of Unet to adapt it to the visual distribution characteristics of images under the SMILE surgical microscope field of view.

[0042] Step S32, edge cutting completes the frame generation stage.

[0043] Step S321: Freeze all model parameters trained in step S31, including all parameters of the fusion encoder and Unet, to preserve the pre-trained and fine-tuned feature extraction and basic generation capabilities.

[0044] Step S322: Introduce Controlnet1 network, input the attraction frame generated in step S31 as spatial constraint condition into Controlnet1, and combine it with the multimodal feature vector output in step S2 to guide the model to generate the edge-cutting completion frame (morphological image after corneal stromal lenticule edge-cutting operation).

[0045] Step S323: In this stage, only the parameters of Controlnet1 are trained without changing the parameters of other models, so as to ensure the spatial constraints and morphological consistency of the generated results.

[0046] ControlNet Technical Description: ControlNet is a neural network adaptation architecture used to add spatial control conditions to pre-trained diffusion models. Its core mechanism is to build a trainable branch by cloning the encoder layer of the original diffusion model, while keeping the parameters of the original model frozen to maintain its learned generative priors. The two parallel branches are bridged and feature fused through zero-weight initialization convolutional layers, enabling the network to gradually learn and accept new spatial constraint information (such as the fused eyeball contour) without destroying the original generative ability, thereby achieving precise control over the structure of the generated image.

[0047] Step S33: Laser scanning completes the frame generation stage.

[0048] Step S331: Freeze all model parameters trained in step S32, including all parameters of Controlnet1, while preserving the generation features of the edge-cutting completed frames.

[0049] Step S332: Introduce the Controlnet2 network and input the edge-cutting completion frame generated in step S32 as a new spatial constraint into Controlnet2. Combined with the multimodal feature vector output in step S2, guide the model to generate the laser scanning completion frame (the morphological image after the femtosecond laser scans and cuts the corneal stroma lenticule). This stage focuses on simulating the edge morphology of the lens after laser scanning and the texture distribution of the OBL.

[0050] Step S333: In this stage, only the parameters of Controlnet2 are trained to ensure that the generated results are consistent with the timing of the surgical procedure, and finally, the intraoperative keyframes of three types of SMILE surgery are obtained.

[0051] Step S4: Prediction of postoperative core clinical indicators.

[0052] The core function of this step is to reuse the multimodal feature vectors of the fusion encoder and combine them with the visual features of the generated attraction frames (visual features extracted from the generated attraction frames) to construct a postoperative numerical prediction model. This enables accurate regression prediction of core clinical indicators after SMILE surgery, achieving "feature reuse and multi-task operation with a single instrument," reducing model parameter redundancy. Figure 4 As shown, the specific implementation is as follows: Step S41: Reuse the fusion encoder trained in step S2 to obtain a multimodal feature vector containing preoperative topographic map visual semantics and clinical baseline physical semantics.

[0053] Step S42: Input the attraction frame generated in step S31 into the ResNet network, extract a high-dimensional visual feature map reflecting the initial state of the eyeball after negative pressure adsorption, and expand the high-dimensional visual feature map to obtain a one-dimensional visual feature vector.

[0054] Step S43: The multimodal feature vectors are concatenated with the expanded one-dimensional visual feature vectors to obtain the fused prediction feature vectors.

[0055] In step S44, the predicted feature vector is input into the MLP for feature mapping, and then connected to two independent decoupled prediction heads, namely the Spherical Equivalent (SE) prediction head and the Lens Ablation Thickness prediction head. The two prediction heads output the postoperative Spherical Equivalent (SE) value and the Lens Ablation Thickness (Ablation) value, respectively, to achieve accurate prediction of postoperative core clinical indicators.

[0056] Step S5: Quantitative evaluation of the completed laser scan frame.

[0057] The core function of this step is to establish a two-dimensional quantitative evaluation system that aligns with a clinical perspective. This involves quality verification of the laser scan completion frames generated in step S33, with a focus on evaluating two dimensions closely related to the clinical operation of SMILE surgery: lens edge trajectory and OBL morphology. This ensures that the generated images have clinical reference value. The specific implementation is as follows: Step S51, Lens edge trajectory evaluation.

[0058] The accuracy of the lens edge trajectory directly affects the safety of intraoperative lens separation. This dimension extracts the lens edge trajectory using a sub-pixel gradient detection algorithm based on guide line constraints, and calculates quantitative indicators to evaluate the similarity between the generated trajectory and the true trajectory. The specific steps are as follows: Step S511: Convert the completed laser scan frame from Cartesian coordinates to polar coordinates to reduce the complexity of edge extraction.

[0059] Step S512: Extract the lens edge trajectory using a sub-pixel gradient detection algorithm based on guide line constraints. First, use a step response filter to search for gray-level abrupt change points in the radial direction, and remove outlier noise points by median filtering to fit a smooth global guide line. Then, use this global guide line as a reference to extract radial gray-level strips of fixed width. Next, calculate the first-order gradient of the one-dimensional gray-level signal of each strip, lock the position of the maximum gradient value, and use the weighted centroid method of neighborhood gradients to calculate the precise sub-pixel edge coordinates.

[0060] Step S513: Based on the extracted lens edge trajectories of the generated image and the real image, calculate two quantitative evaluation metrics: (1) Root Mean Square Error (RMSE): The error is calculated by comparing the generated edge trajectory with the actual edge trajectory. The distance deviation at a radial angle quantifies the overall fit of the cutting position; a smaller value indicates a higher fit. The formula for calculating RMSE is as follows: , In the above formula, Indicates the root mean square error; This represents the total number of sampling angles in polar coordinates, ranging from 1 to 360. It is the current angle index; and These represent the angle index as At that time, the sub-pixel radial distance between the generated edge trajectory and the real edge trajectory is calculated.

[0061] (2) Edge roughness difference: The macroscopic contour of the edge is fitted using a smoothing filter, and the microscopic residual between the original edge and the fitted contour is calculated to define the roughness of a single edge. The absolute difference in roughness between the generated image and the real image is then calculated to assess the realism of the incision's smoothness; a smaller value indicates smoother edges and lower surgical risk. This difference in edge roughness... The calculation formula is as follows: , , In the above formula, This represents the total number of sampling angles in polar coordinates, ranging from 1 to 360. It is the current angle index; Indicates that at the angle index is The original edge radial coordinate sequence extracted at that time; Indicates that at the angle index is The macroscopic contour sequence after smoothing and filtering; and These represent the roughness of the generated edge and the real edge, respectively.

[0062] Step S52, OBL morphology assessment.

[0063] The distribution and morphology of OBLs directly affect the surgical field of view and laser ablation accuracy during surgery. This dimension extracts OBL regions through image enhancement and segmentation algorithms, and calculates quantitative evaluation indicators to obtain the similarity between the generated region and the real region. The specific steps are as follows: Step S521: Targeted processing is performed on the OBL region in the completed laser scanning frame: First, the Limit Contrast Adaptive Histogram Equalization (CLAHE) algorithm is used to enhance the texture contrast of the tiny bubbles in the OBL region and improve the recognizability of the bubble region; then, the enhanced OBL region is binarized and segmented to obtain the OBL binarized mask image and extract its morphological features.

[0064] Step S522: Based on the OBL binarized mask images of the generated image and the real image, calculate two quantitative evaluation metrics: (1) Local Structural Similarity Index (SSIM): This evaluates the perceptual similarity of bubble textures in terms of brightness, contrast, and structure, reflecting the visual realism of the generated bubbles. The closer the value is to 1, the more realistic the visual texture. To eliminate background interference, Local SSIM is only performed in the Region of Interest (ROI) enhanced by the CLAH algorithm, i.e., the OBL grayscale region. The formula for Local SSIM is as follows: , In the above formula, Indicates local SSIM, and These represent the real OBL image and the generated OBL image after enhancement using the CLAHE algorithm, respectively. and Image used to reflect brightness and The local pixel mean, and Represents an image used to reflect contrast. and The local pixel variance, Images used to reflect the degree of structural similarity and Local covariance, and The first and second constants are very small and introduced to maintain the stability of the denominator.

[0065] (2) Dice Similarity Coefficient: This measures the degree of spatial overlap between the generated bubble region and the real bubble region, reflecting the accuracy of the model's prediction of the bubble distribution location. The closer the value is to 1, the higher the degree of overlap. The Dice similarity coefficient is calculated based on the OBL mask after binarization segmentation, as follows: , In the above formula, Represents the Dice similarity coefficient. and These represent the OBL mask sets obtained after binarizing the real image and the generated image, respectively. This represents the total number of pixels with a value of 1 within the set, i.e., the area of ​​the region; This represents the total number of pixels in the spatially overlapping area between the actual bubble region and the generated bubble region.

[0066] To verify the technical effects of the present invention, the SMILE surgical clinical dataset was selected for model training and testing. The dataset includes preoperative multimodal data collected from multiple clinical centers, intraoperative microscope keyframe images, and postoperative follow-up values. The training set accounted for 80% and the test set accounted for 20%. After multiple iterations of training, the temporal generation effect of intraoperative keyframes, the prediction effect of postoperative equivalent spherical and lens actual cutting thickness, and the clinical quantitative evaluation effect of laser scanning completion frames were verified. All quantitative indicators are the average results on the test set.

[0067] Verification 1: Timing generation effect of intraoperative keyframes.

[0068] The three types of intraoperative keyframes generated in step S3—suction frame, edge incision completion frame, and laser scan completion frame—have their real and generated images as follows: Figure 5 As shown. From Figure 5 It can be seen that the generated images are highly consistent with the real images in terms of color, lighting, and overall structure, exhibiting extremely high visual fidelity. This demonstrates that the three-stage cascaded diffusion generation model can not only accurately distinguish and generate intraoperative keyframes at three different stages of SMILE surgery, but also precisely reproduce changes in key physical features such as bubble texture. Therefore, it fully proves that this method can effectively learn and reproduce, with high fidelity, the detailed changes and visual distribution patterns under the microscopic field of view of real clinical surgery.

[0069] right Figure 5 Intraoperative keyframes were evaluated using common image quality assessment metrics: PSNR and global SSIM. A higher PSNR value indicates less image distortion, and a higher global SSIM value indicates higher image structural similarity. The specific results are shown in Table 1.

[0070] Table 1. PSNR and Global SSIM of Intraoperative Keyframes

[0071] As shown in Table 1, the three types of intraoperative keyframes generated by this invention all have high image quality. The PSNR and global SSIM of the attraction frame are the best, which is consistent with the characteristics of the cascaded generation architecture. The subsequent frames have slightly lower indicators due to the introduction of more complex surgical morphology (lens side cutting, OBL generation) but still remain within a reasonable range, thus achieving high-fidelity generation of intraoperative keyframes.

[0072] Verification 2: Predictive effect of postoperative equivalent spherical lens and lens actual cutting thickness.

[0073] For the two core clinical indicators output in step S4, postoperative equivalent spherical diameter (SE) and actual lens ablation thickness (Ablation), the average error was used as a quantitative evaluation indicator. The smaller the average error, the higher the prediction accuracy. The specific results are shown in Table 2.

[0074] Table 2. Average Error of Postoperative Core Clinical Indicators

[0075] As shown in Table 2, the present invention has high prediction accuracy for postoperative core clinical indicators, and the average error is controlled within the clinically acceptable error range of SMILE surgery. This proves that the design of multimodal feature fusion and feature cross-task reuse effectively improves the accuracy of postoperative prediction and achieves synergistic optimization of intraoperative morphological generation and postoperative numerical prediction.

[0076] Verification 3: Laser scanning completes the clinical quantitative assessment of the effect.

[0077] Based on the clinical dual-dimensional quantitative assessment system established in step S5, the laser scan completion frames (i.e., generated images) and real laser scan completion frames (i.e., real images) in the test set were compared and evaluated. The focus was on analyzing the indicators of two dimensions: lens edge trajectory and OBL morphology. The specific results are as follows: (1) Lens edge trajectory evaluation.

[0078] like Figure 6 As shown, Figure 6 The left image in the image is the original image. Figure 6 The middle image is a diagram of the lens edge trajectory in Cartesian coordinates. Figure 6 The right image in the diagram is a display under polar coordinates, where the X-axis represents the angle and the Y-axis represents the pixel value. According to the polar coordinates, the RMSE is 1.457 and the edge roughness difference is 0.104.

[0079] The above-mentioned index values ​​are all at a low level, which proves that the lens edge trajectory generated by the present invention has a high degree of consistency with the real trajectory and the edge is smooth. It can accurately simulate the lens edge morphology of laser cutting in SMILE surgery, and provide a reliable visual reference for clinicians to predict the difficulty of lens separation.

[0080] (2) OBL morphology assessment.

[0081] like Figure 7 As shown, Figure 7 The left image in the image is the original image. Figure 7 The middle image shows the OBL grayscale region enhanced using the CLAHE algorithm; other irrelevant local areas are masked. Figure 7 The right image shows the OBL mask after binarization segmentation based on the enhanced OBL grayscale region. Local SSIM is calculated based on the OBL grayscale region to evaluate the reproduction of texture details; simultaneously, the Dice similarity coefficient is calculated based on the binarized OBL mask to quantify the overlap between the real and generated bubbles in spatial distribution. The final calculation results are: local SSIM = 0.821, and Dice similarity coefficient = 0.610.

[0082] The results above show that the spatial overlap and visual texture of the OBL area are both good, proving that the present invention can accurately simulate the distribution location and texture characteristics of OBL, providing effective visual basis for clinicians to predict the risk of surgical field obstruction during surgery.

[0083] Overall Results: The MMC-DM-based intraoperative and postoperative prediction method for SMILE surgery proposed in this invention successfully achieves deep fusion of preoperative multimodal data, high-fidelity temporal generation of intraoperative keyframes, accurate prediction of postoperative core clinical indicators, and clinically specific quantitative evaluation of generated images. All quantitative evaluation indicators reach clinical application-level standards. Crucially, this invention fully realizes a closed-loop process of "preoperative prediction - intraoperative simulation - postoperative prediction" through cross-task reuse of core features, collaborative linkage of modules, and an evaluation feedback mechanism. Using a fusion encoder as the core hub, it connects three core stages: the foundational construction of preoperative multimodal data prediction, dynamic simulation of key intraoperative morphology, and accurate prediction of postoperative outcomes. Simultaneously, a dual-dimensional quantitative evaluation system verifies the reliability of intraoperative simulation, feeding back the evaluation results to the preoperative prediction stage to optimize prediction accuracy. This forms a closed-loop mechanism where each stage supports and optimizes the others, completely resolving the shortcomings of disconnected stages in existing technologies. It provides comprehensive AI-assisted support for preoperative planning and risk avoidance in SMILE surgery, effectively improving the predictability and safety of surgical plans.

[0084] In a second aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a SMILE intraoperative and postoperative prediction method based on MMC-DM.

[0085] The memory can be volatile or non-volatile, or a combination of both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory of this invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0086] The processor can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0087] The method steps of this invention can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof.

[0088] Software implementation can be achieved by executing functional modules (such as procedures, functions, etc.). Software code can be stored in memory and executed by the processor. Memory can be implemented in the processor or outside the processor.

[0089] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a SMILE intraoperative and postoperative prediction method based on MMC-DM.

[0090] Computer storage media can include various media that can store program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0091] Fourthly, the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of an MMC-DM-based SMILE intraoperative and postoperative prediction method.

[0092] Specifically, computer program products include: data signals, data signals embodied in a carrier wave, or computer-readable storage media.

[0093] It should be noted that the technical solutions described in this invention can be combined arbitrarily without conflict.

[0094] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention also includes such modifications and variations.

Claims

1. A method for intraoperative and postoperative prediction of SMILE based on MMC-DM, characterized in that, include: After preprocessing the corneal topography and clinical baseline data of patients before SMILE surgery, a fusion encoder is used to extract visual feature maps from the preprocessed corneal topography and extract numerical feature vectors from the clinical baseline data. After fusion through a two-layer fusion mechanism of shallow adaptive modulation and deep cross-attention fusion, a multimodal feature vector is output. A three-stage cascaded diffusion generation model is constructed based on the potential diffusion model. Using multimodal feature vectors as conditions, and combined with Controlnet, the image generated in the previous stage is used as a spatial constraint to guide the image generation in the next stage. Intraoperative key frames that conform to the SMILE surgical timing logic are generated in stages, including suction frames, edge cutting completion frames, and laser scanning completion frames. By reusing the multimodal feature vector and combining it with the visual features of the generated attraction frame, a postoperative numerical prediction model is constructed, which outputs the postoperative equivalent spherical lens value and the actual lens cutting thickness value, respectively. A two-dimensional quantitative evaluation system that conforms to the clinical perspective is established to quantitatively evaluate the generated image and the real image of the completed laser scan frame. First, the root mean square error and edge roughness difference of the lens edge trajectory dimension are calculated, and then the Dice similarity coefficient and local SSIM of the OBL morphology dimension are calculated.

2. The method as described in claim 1, characterized in that, When the fusion encoder extracts visual feature maps, it uses the Swin Transformer as the backbone network to perform multi-scale feature encoding on the preprocessed corneal topography map, extracting spatial feature maps of encoder residual layer 3, encoder residual layer 4 and encoder residual layer 5, and establishing a hierarchical visual feature map of the corneal topography map. When the fusion encoder extracts numerical feature vectors, it uses a random Gaussian matrix with fixed weights to project the preprocessed clinical baseline data into a high-dimensional space, and then performs nonlinear mapping through a multilayer perceptron to generate numerical feature vectors with high-frequency response characteristics. The shallow adaptive modulation process of the fusion encoder includes: aggregating and compressing numerical feature vectors to generate global physical prior vectors; using adaptive group normalization to predict affine transformation parameters through the global physical prior vectors; and injecting the parameters layer by layer into visual feature maps at different levels to complete the recalibration of image feature distribution. The deep cross-attention fusion process of the fusion encoder includes: initializing a learnable visual query vector, using the visual query vector as Q, and the modulated visual feature map as K and V; retrieving and aggregating key visual information in the visual feature map through a cross-attention mechanism; concatenating the aggregated visual query vector with the numerical feature vector, and then completing cross-modal information exchange through a self-attention mechanism; the continuous process of cross-attention aggregation and self-attention interaction constitutes a Q-transformer block, and after iterative fusion of N Q-transformer blocks, a multimodal feature vector is output.

3. The method as described in claim 1, characterized in that, When constructing a three-stage cascaded diffusion generation model, a stable diffusion model is used as the baseline model. Its pre-trained variational autoencoder is used to map the image to a low-dimensional latent space for denoising. The encoder and decoder parameters of the variational autoencoder are kept frozen throughout the process. The three-stage generation process employs a differentiated training strategy. In the first stage, no ControlNet is involved; the fusion encoder is trained, and the parameters of the UNet backbone network are fine-tuned. The second and third stages introduce corresponding ControlNets, and training is performed only on the ControlNet parameters of the current stage, while freezing all model parameters from the previous stages.

4. The method as described in claim 1, characterized in that, The specific implementation process of the postoperative numerical prediction model includes: Obtain the multimodal feature vector output by the fusion encoder; The generated attraction frames are input into the ResNet network to extract high-dimensional visual feature maps and unfold them. The unfolded one-dimensional visual feature vectors are concatenated with the multimodal feature vectors and used as input features for the postoperative numerical prediction model. The postoperative numerical prediction model is based on a multilayer perceptron and outputs the postoperative equivalent spherical lens value and the actual lens cutting thickness value through two independent decoupled prediction heads.

5. The method as described in claim 1, characterized in that, Before calculating the index of the lens edge trajectory dimension, the laser scanning completion frame is first converted from the Cartesian coordinate system to the polar coordinate system, and the lens edge trajectory is extracted by the sub-pixel gradient detection algorithm based on guide line constraint. Before calculating the index of the OBL morphology dimension, the texture contrast of the OBL region is first enhanced by the contrast-limited adaptive histogram equalization algorithm, and then the OBL region is binarized to obtain the OBL binarized mask image and its morphological features are extracted.

6. The method as described in claim 1, characterized in that, The formulas for calculating the root mean square error and the edge roughness difference are as follows: ， ， ， In the above formula, This represents the root mean square error. This represents the total number of sampling angles in polar coordinates. This is the current angle index. and These represent the angle index as The sub-pixel radial distance of the generated edge trajectory. This indicates the roughness of a single edge. These represent the angle index as The subpixel-level radial distance of the real edge trajectory. Indicates that at the angle index is The original edge radial coordinate sequence extracted at that time. Indicates that at the angle index is The macroscopic contour sequence after smoothing and fitting. Indicates the difference in edge roughness. and These represent the roughness of the generated edge and the real edge, respectively.

7. The method as described in claim 1, characterized in that, The formulas for calculating the Dice similarity coefficient and local SSIM are as follows: ， ， In the above formula, Represents the Dice similarity coefficient. and These represent the OBL mask sets obtained after binarizing the real image and the generated image, respectively. This represents the total number of pixels in the set with a value of 1. This represents the total number of pixels in the spatially overlapping area between the actual bubble region and the generated bubble region. and These represent the real OBL image and the generated OBL image after enhancement using the contrast-limited adaptive histogram equalization algorithm, respectively. Indicates local SSIM, and Image used to reflect brightness and The local pixel mean, and Represents an image used to reflect contrast. and The local pixel variance, Images used to reflect the degree of structural similarity and Local covariance, and The first and second constants are very small and introduced to maintain the stability of the denominator.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

Images