An ophthalmic surgery robot generalization training method and system based on a mechanical model

Abstract

The application discloses an ophthalmic surgery robot generalization training method and system based on a mechanical model, and comprises the following steps: first, a layered super-elastic refined mechanical model is established according to eyeball anatomical structure; then, multi-dimensional mechanical parameter directional disturbance is carried out in combination with clinical pathological characteristics and surgery interaction rules, and a large-scale simulation sample library is constructed; subsequently, multi-modal surgery data collected in the clinic are preprocessed, and high-quality real samples are screened relying on mechanical characteristics and surgery task association rules; a multi-modal perception model is constructed in a simulation sample pre-training, screened real sample migration fine-tuning mode; finally, dynamic parameter adjustment is formed through test feedback, full-process closed-loop iterative optimization is formed, and the generalization perception ability of the model across patients, across pathologies and across surgical procedures is improved.

Description

Technical Field

[0001] This invention belongs to the field of medical robot training technology, and specifically relates to a generalization training method and system for ophthalmic surgical robots based on a mechanical model. Background Technology

[0002] With the development of technology, medical robots have been introduced into ophthalmic surgery. Ophthalmic surgery presents challenges such as limited operating space, extremely fragile soft tissues of the eyeball, delicate anatomical structures, and a very low margin for error. Therefore, ophthalmic surgical robots rely on intelligent sensing modules to accurately perceive changes in eyeball tissue, the interaction force between instruments and tissues, and the location and depth of lesions, thereby completing automated and intelligent minimally invasive ophthalmic surgery.

[0003] Currently, the training of perception models for existing ophthalmic surgical robots mainly relies on two methods: real clinical surgical data and simulated virtual data. Among them, real clinical ophthalmic surgical data is costly to collect, has scarce effective samples, and is difficult to annotate. Different patients have significant differences in age, eye shape, soft tissue elasticity, and pathological condition. Training with only a small amount of real data is prone to overfitting, resulting in poor model generalization ability and an inability to adapt to clinical scenarios with multiple patients, multiple surgical procedures, and multiple pathologies.

[0004] However, existing simulation virtual data generation methods mostly use rigid homogeneous eyeball modeling, which generally simplifies the eyeball into an overall homogeneous rigid structure. They do not perform layered mechanical characterization of the cornea, sclera, vitreous body, and retina, and cannot characterize the nonlinear, viscoelastic, and anisotropic biomechanical behavior of the eyeball's soft tissues. This results in a significant deviation between the virtual data generated by simulation and the real physiological characteristics.

[0005] Some studies have attempted to introduce mechanical parameters for generalization and expansion, but these only involve simple, random perturbations of the elastic modulus and Poisson's ratio. The generalization dimension is singular, lacks constraints on pathological mechanisms and surgical interactions, fails to distinguish the differences in the layered structure of the eyeball, and cannot simulate real pathological mechanical behaviors such as vitreous liquefaction, retinal diseases, and age-related tissue degeneration. At the same time, the parameter generalization does not combine the contact constraints between ophthalmic surgical instruments and tissues, or the deformation damage threshold for targeted adjustment, resulting in highly homogeneous simulation samples and incomplete coverage of lesion mechanical responses and tissue deformation patterns.

[0006] Meanwhile, there is a significant domain offset between the existing virtual simulation domain and the real clinical domain, and there is a lack of cross-domain alignment mechanism based on mechanical features; conventional data screening only relies on superficial signal indicators such as image clarity and signal-to-noise ratio, without combining ocular biomechanical features and surgical perception tasks for quantitative screening. The utilization rate of a small number of real samples is low, the quality of effective samples is poor, and there are many dirty samples, which cannot support the training requirements of high-precision perception models. Summary of the Invention

[0007] To overcome the problems existing in the prior art, the present invention provides a generalization training method and system for ophthalmic surgical robots based on a mechanical model.

[0008] Therefore, the technical solution of the present invention is: a generalization training method for ophthalmic surgical robots based on a mechanical model, comprising the following steps: S1. Perform fine-grained stratification of the soft tissues of the eyeball and construct a hyperelastic mechanical model: S1.1 Based on the actual anatomical structure of the eyeball, the cornea, sclera, vitreous body, and retina are divided into mutually independent layered structures; S1.2 Collect the elastic modulus, Poisson's ratio, viscosity coefficient, and basic stress-strain parameters of various tissues under normal physiological and typical pathological conditions. S1.3. The constitutive relations of each soft tissue were constructed using the Ogden hyperelastic model combined with the Kelvin-Voigt viscoelastic model. S1.4. Establish a three-dimensional layered mesh model of the eyeball using the finite element method, set the contact constraints, deformation range, and tissue damage threshold between the surgical instruments and the eyeball tissue, and characterize the tissue deformation, stress distribution, and mechanical behavior of the interaction between the instruments and soft tissue during the operation. S2. Generalize multi-dimensional mechanical parameters in a targeted manner and construct a simulation sample library: Combining common clinical pathological features with the interactive characteristics of surgical instruments, multi-dimensional joint directional perturbation is performed on the overall size of the eyeball, the mechanical parameters of soft tissues in each layer, the location and extent of lesions, the posture of surgical instruments, the operating force, and environmental noise; multimodal data are collected and labeled simultaneously to construct a large-scale simulation sample library covering different individuals, different pathologies, and different surgical methods. S3. Collect real clinical surgical data and perform preprocessing; S4. Screen clinical data based on correlations of mechanical characteristics; A comprehensive evaluation system based on ocular biomechanical characteristics and aligned with surgical perception tasks was established. Evaluation dimensions included the effectiveness of tissue deformation, quality of interactive force signals, salience of lesion features, similarity between simulated and real biomechanical characteristics, and matching degree of surgical scenario. A weighted entropy weighting method was used to assign weights and scores to each evaluation indicator, and high-quality clinical samples with comprehensive scores higher than a set threshold were selected to form a subset of real and effective samples. This achieved alignment of biomechanical characteristics between the simulation domain and the clinical real domain, reducing the accuracy loss caused by domain offset. S5. Perform pre-training and transfer fine-tuning of the multimodal generalized perception model; A multimodal perception model with a CNN-Transformer hybrid architecture was built. It takes visual, depth, and force multimodal data as input and outputs perception results such as tissue deformation, lesion location coordinates, instrument interaction force, and safe instrument operation distance. S6. Conduct model generalization performance testing and closed-loop iterative optimization; A test dataset covering different ages, pathological types, and surgical methods was constructed to test the accuracy and robustness of the fine-tuned perception model. If the test indicators did not meet the clinical preset standards, the mechanical model parameters, the generalization range of mechanical parameters, and the data screening threshold were adaptively adjusted. The modeling, sample generation, data screening, and model training processes were executed cyclically to form a complete closed-loop iterative optimization mechanism, and finally a generalized perception model for ophthalmic surgical robots that meets the requirements of clinical application was obtained.

[0009] Based on the above scheme and as a preferred embodiment of the above scheme: In step S1.3, the Ogden hyperelastic model and the Kelvin Voigt viscoelastic model are used to construct coupled constitutive equations: In the formula: s For stress tensor; m Shear modulus; α These are the constitutive parameters of the material; , , Main elongation ratio; I Unit tensor; or The viscosity coefficient; Let be the strain rate tensor.

[0010] Based on the above scheme and as a preferred embodiment of the above scheme: In step S1.4, the tissue deformation constraint conditions that ensure surgical safety are as follows: In the formula: u To organize displacement, The maximum safe deformation threshold; And, the contact force constraints between the robotic device and the eye tissue: In the formula: F c For the contact force of the instrument, F max For maximum safe interaction.

[0011] Based on the above scheme and as a preferred embodiment of the above scheme: in step S2, the multimodal data collected includes collected visual images, depth information, instrument interaction forces, tissue deformation displacement, and stress field.

[0012] Based on the above scheme and as a preferred embodiment of the above scheme: In step S2, the various parameters of the eyeball are corrected by multi-dimensional joint directional perturbation. In the formula: E The original elastic modulus, This is the corrected elastic modulus; or The original viscosity coefficient, The corrected viscosity coefficient, R The original geometric dimensions of the eyeball. The corrected eyeball geometry; c The random perturbation coefficient is the value within the interval [0,1]. , , This represents the corresponding pathological disturbance coefficient.

[0013] Based on the above scheme and as a preferred scheme: In step S3, visual, force, and depth multimodal raw data are collected during ophthalmic clinical surgery. Signal denoising, multimodal data registration, and temporal-spatial alignment are performed sequentially to remove invalid samples with blurred images, abnormal force signals, and missing data, thereby obtaining a regular and usable real clinical dataset. The formula for calculating the alignment error during temporal and spatiotemporal alignment is as follows: In the formula: E a This refers to the spatiotemporal alignment error of multimodal data. , , The spatial coordinates and timestamps for the first mode; , , These are the spatial coordinates and timestamps for the second mode.

[0014] Based on the above scheme and as a preferred option: In step S4, the information entropy of a single indicator is first calculated: In the formula: e i For the first i Information entropy of the indicator; K The normalization coefficient is... K =1 / ln( n ), n The total number of samples; For the first j The sample at the th i Standardized percentage of each indicator; ln is the natural logarithm; Calculate the weights of each indicator based on information entropy: In the formula: For the firsti The weight of each indicator; For the first i Information entropy of the indicator; This is the sum of the information entropy redundancy of all indicators; Calculate the comprehensive score for a single sample by combining the indicator weights: In the formula: Score is the overall score of the sample; , , , , The weights of the five evaluation indicators; , , , , The standardized scores of a single sample across the five metrics.

[0015] Based on the above scheme and as a preferred embodiment of the above scheme: In step S5, the total loss function expression of the multimodal sensing model is: In the formula: For location perception loss, To perceive loss, To avoid loss of safe distance, , This is the loss balance coefficient; The training process uses an exponential decay method to adjust the learning rate, and the learning rate update formula is: In the formula: For the first The learning rate of the round; The initial learning rate; This is the current training round; This is the learning rate decay coefficient.

[0016] Based on the above scheme and as a preferred option: In step S6, generalization performance testing is performed on clinical data, and the position perception error is calculated using the following formula: In the formula: This is due to position perception error; The total number of test samples; , The coordinates of the instrument position predicted by the model; , These are the actual coordinates of the instruments during clinical surgery. Formula for calculating force sensing error: In the formula: For force sensing error; The total number of test samples; The instrument contact force predicted by the model; This refers to the actual contact force of instruments during clinical surgery.

[0017] Another technical solution of the present invention is: a generalization training system for ophthalmic surgical robots based on a mechanical model, comprising: Ocular soft tissue biomechanical modeling module: responsible for inputting biomechanical parameters of each layer of tissue, completing hyperelastic constitutive modeling, finite element mesh generation, and defining contact constraints and deformation thresholds; Simulation sample generation module: Based on the mechanical model and combined with the constraints of pathological and surgical conditions, it completes the directional generalization of multi-dimensional mechanical parameters and generates a standardized multimodal simulation sample library; Real data acquisition and preprocessing module: Connects to the vision, depth, and force sensing devices of ophthalmic surgical robots to complete noise reduction, registration, alignment, and abnormal data removal of raw data; Real data intelligent screening module: Establishes a mechanical feature evaluation index system, uses weighted entropy weighting method to complete sample quantitative scoring and high-quality sample screening, and realizes the matching and alignment of mechanical features in the virtual and real domains; Generalized perception model training module: Build a multimodal hybrid network model and perform pre-training on simulation data and fine-tuning on high-quality clinical samples; Model generalization iterative optimization module: responsible for model performance testing and error analysis, and adjusting mechanical parameters, generalization range and screening threshold in a coordinated manner to achieve closed-loop iteration throughout the entire process; Human-computer interaction visualization module: centrally displays the eye biomechanics simulation effect, sample selection results, model training process and perception output results, and supports manual viewing and adjustment of key parameters.

[0018] Compared with the prior art, the beneficial effects of the present invention are: 1. By adopting a multi-tissue layered structure of the eyeball combined with hyperelastic modeling, the limitations of traditional rigid homogeneous simplified modeling are overcome. This approach can realistically reproduce the nonlinear and viscoelastic mechanical properties of various tissues in the eyeball, making the simulation data more consistent with actual human physiological data and improving the accuracy of training samples.

[0019] 2. By coupling common clinical pathological features with surgical conditions, multi-dimensional mechanical parameters are generalized in a targeted manner, abandoning the crude mode of random value selection of single parameters. This allows simulation samples to cover more individual differences and pathological scenarios, improving the diversity and effectiveness of training simulation samples.

[0020] 3. Using biomechanical characteristics as the core, clinical samples are screened to select high-quality clinical samples, forming a subset of real and effective samples. This breaks away from the superficial screening logic that relies solely on image and signal quality, and achieves feature alignment between simulation data and clinical data, significantly improving the utilization value of a small amount of clinical data.

[0021] 4. An integrated closed-loop system was constructed, encompassing mechanical modeling, simulation sample generation, clinical data screening, model training, and iterative optimization. Each link is interconnected and adjustable, solving the problem of fragmented modules and lack of collaborative optimization in traditional technologies. This significantly improves the generalization accuracy and stability of the model in unknown patients and complex pathological scenarios.

[0022] 5. The trained model is applicable to mainstream ophthalmic surgical scenarios such as cataract surgery, vitrectomy, and retinal repair. It has a wide range of applications and can directly support the clinical training and practical application of ophthalmic surgical robots. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system module architecture diagram of the present invention; Figure 3 This is a structural diagram of the multimodal CNN-Transformer perception model of the present invention. Detailed Implementation

[0024] Example 1 The generalization training method for ophthalmic surgical robots based on mechanical models described in this embodiment first establishes a layered, hyperelastic, refined mechanical model according to the anatomical structure of the eyeball; then, combining clinical pathological features and surgical interaction rules, it conducts multi-dimensional mechanical parameter directional perturbations to construct a large-scale simulation sample library; subsequently, it preprocesses the multimodal surgical data collected clinically and selects high-quality real samples based on the association rules between mechanical features and surgical tasks; it constructs a multimodal perception model by using simulation sample pre-training and real sample transfer fine-tuning after screening; finally, it dynamically adjusts parameters through test feedback to form a closed-loop iterative optimization throughout the entire process, improving the model's generalization perception ability across patients, pathologies, and surgical procedures.

[0025] The specific steps are as follows: S1. Perform fine-grained stratification of the soft tissues of the eyeball and construct a hyperelastic mechanical model: Based on the actual anatomical structure of the eyeball, the cornea, sclera, vitreous body, and retina are divided into mutually independent layered structures.

[0026] Collect the elastic modulus, Poisson's ratio, viscosity coefficient, and basic stress-strain parameters of various tissues under normal physiological and typical pathological conditions; for example: corneal elastic modulus 0.3-0.8 MPa, scleral elastic modulus 0.1-0.3 MPa, vitreous elastic modulus 0.01-0.05 MPa, retinal elastic modulus 0.05-0.15 MPa, and viscosity coefficient of each tissue 0.005-0.05 Pa·s.

[0027] Coupled constitutive equations were constructed using the Ogden hyperelastic model and the Kelvin Voigt viscoelastic model: In the formula: s For stress tensor; m Shear modulus; α These are the constitutive parameters of the material; , , Main elongation ratio; I Unit tensor; or The viscosity coefficient; Let be the strain rate tensor.

[0028] To ensure surgical safety, tissue deformation constraints are set: In the formula: u To organize displacement, The maximum safe deformation threshold is defined as 0.1-0.2 mm.

[0029] Set contact force constraints between the robotic instrument and the eye tissue: In the formula: F c For the contact force of the instrument, F max The maximum safe interaction force is set to 0.5 N.

[0030] A three-dimensional layered mesh model of the eyeball was constructed using the finite element method, with the mesh element size set to 0.1-0.3 mm. Combined with the above constraints, the simulation of tissue deformation, stress distribution and interaction forces during robot operation was completed.

[0031] S2. Generalize multi-dimensional mechanical parameters in a targeted manner and construct a simulation sample library: Combining common clinical pathological features with the interactive characteristics of surgical instruments, multi-dimensional joint directional perturbation is performed on the overall size of the eyeball, the mechanical parameters of soft tissues in each layer, the location and extent of lesions, the posture of surgical instruments, the operating force, and environmental noise; multimodal data are collected and labeled simultaneously to construct a large-scale simulation sample library covering different individuals, different pathologies, and different surgical methods.

[0032] For example, by combining two typical ocular pathological features—vitreous liquefaction and retinal detachment—multi-dimensional joint directional perturbations are applied to correct parameters such as eyeball geometry, soft tissue elastic modulus, and viscosity coefficient. In the formula: E The original elastic modulus, or The original viscosity coefficient, R The original geometric dimensions of the eyeball; c The random perturbation coefficient is the value within the interval [0,1]. , , This represents the corresponding pathological disturbance coefficient.

[0033] This experiment generated 100,000 sets of simulation samples, and simultaneously collected visual images, depth images, robot interaction force time-series signals, tissue deformation displacement and stress field data to construct a multimodal simulation sample library.

[0034] S3. Collect real clinical surgical data and perform preprocessing; We collect raw visual, force, and depth multimodal data during ophthalmic clinical surgery, and sequentially perform signal denoising, multimodal data registration, and temporal and spatiotemporal alignment processing. Invalid samples such as blurred images, abnormal force signals, and missing data are removed to obtain a well-organized and usable real clinical dataset.

[0035] The raw clinical data was denoised using Gaussian filtering. The Gaussian filter kernel formula is as follows: in: The coordinates of the pixels within the kernel relative to the kernel center; The standard deviation is Gaussian to control the degree of fuzziness; The normalization coefficient ensures that the sum of all weights is 1, and the image brightness remains unchanged.

[0036] After noise reduction, spatiotemporal alignment of multimodal data is performed. The alignment error is calculated using the following formula: In the formula: Ea This refers to the spatiotemporal alignment error of multimodal data. , , The spatial coordinates and timestamps for the first mode; , , These are the spatial coordinates and timestamps for the second mode.

[0037] This study collected data from 20 clinical vitrectomy surgeries. After noise reduction, spatiotemporal alignment, and removal of outlier data, 800 sets of valid and real data were obtained.

[0038] S4. Screen clinical data based on correlations of mechanical characteristics; A comprehensive evaluation system centered on ocular biomechanical characteristics and aligned with surgical perception tasks was established. Evaluation dimensions included the effectiveness of tissue deformation, quality of interactive force signals, salience of lesion features, similarity between simulated and real biomechanical characteristics, and matching degree of surgical scenario. A weighted entropy weighting method was used to assign weights and scores to each evaluation indicator, and high-quality clinical samples with comprehensive scores higher than a set threshold were selected to form a subset of real and effective samples. This achieved alignment of biomechanical characteristics between the simulation domain and the clinical real domain, reducing the accuracy loss caused by domain offset.

[0039] First, calculate the information entropy of each individual indicator: In the formula: e i For the first i Information entropy of the indicator; K The normalization coefficient is... K =1 / ln( n )( n (total number of samples) For the first j The sample at the th i Standardized percentage of each indicator; ln is the natural logarithm.

[0040] Calculate the weights of each indicator based on information entropy: In the formula: For the first i The weight of each indicator; For the first i Information entropy of the indicator; It is the sum of the information entropy redundancy of all indicators.

[0041] Calculate the comprehensive score for a single sample by combining the indicator weights: In the formula: Score is the overall score of the sample; , , , , The weights of the five evaluation indicators; , , , , The standardized scores of a single sample across the five metrics.

[0042] The screening threshold was set to 0.7, and only samples with a comprehensive score greater than the threshold were retained. In the end, 320 high-quality real samples were selected to achieve the alignment of mechanical characteristics between the simulation domain and the real clinical domain.

[0043] S5. Perform pre-training and transfer fine-tuning of the multimodal generalized perception model; A multimodal perception model with a CNN-Transformer hybrid architecture was constructed. It takes visual, depth, and force multimodal data as input and outputs perception results such as tissue deformation, lesion location coordinates, instrument interaction force, and safe instrument operation distance. First, the model was pre-trained using large-scale simulation samples to learn the general mechanical characteristics of the eyeball and the basic perception characteristics of surgery. Then, a small number of high-quality clinical samples were selected and fine-tuned to adapt the pre-trained model to the perception task requirements of real clinical physiological scenarios.

[0044] Construct a CNN-Transformer hybrid architecture multimodal perception model. The overall loss function of the model is expressed as follows: In the formula: For location perception loss, To perceive loss, For the loss of safe distance, , This is the loss balance coefficient.

[0045] The training process uses an exponential decay method to adjust the learning rate, and the learning rate update formula is: In the formula: For the first The learning rate of the round; The initial learning rate; This is the current training round; This is the learning rate decay coefficient.

[0046] The training process is divided into two stages: the first stage uses simulated samples for 100 rounds of pre-training with an initial learning rate of 0.001 and a batch size of 32; the second stage uses selected real samples for 30 rounds of fine-tuning with an initial learning rate of 0.0001 and a batch size of 16.

[0047] S6. Conduct model generalization performance testing and closed-loop iterative optimization; A test dataset covering different ages, pathological types, and surgical methods was constructed to test the accuracy and robustness of the fine-tuned perception model. If the test indicators did not meet the clinical preset standards, the mechanical model parameters, the generalization range of mechanical parameters, and the data screening threshold were adaptively adjusted. The modeling, sample generation, data screening, and model training processes were executed cyclically to form a complete closed-loop iterative optimization mechanism, and finally a generalized perception model for ophthalmic surgical robots that meets the requirements of clinical application was obtained.

[0048] Generalization performance was tested using clinical data from 10 unknown patients. The formula for calculating position perception error is as follows: In the formula: This is due to position perception error; The total number of test samples; , The coordinates of the instrument position predicted by the model; , These are the actual coordinates of the instruments during clinical surgery.

[0049] Force sensing error calculation formula: In the formula: For force sensing error; The total number of test samples; The instrument contact force predicted by the model; This refers to the actual contact force of instruments during clinical surgery.

[0050] Test results show that the position sensing error is less than 0.2 mm and the force sensing error is less than 0.1 N, meeting the requirements for clinical use. For samples with errors exceeding the limits, the perturbation range of mechanical parameters and the data screening threshold are automatically adjusted, and the entire process is repeated cyclically until the model performance converges.

[0051] Example 2 This embodiment relates to a generalization training system for ophthalmic surgical robots based on a mechanical model, including: ① Ocular soft tissue biomechanical modeling module: Responsible for inputting biomechanical parameters of each layer of tissue, completing hyperelastic constitutive modeling, finite element mesh generation, and defining contact constraints and deformation thresholds.

[0052] Calculate the strain of ocular soft tissue engineering: In the formula: To organize the original length, The length after tissue deformation.

[0053] Based on viscoelastic properties, a stress time-domain integral expression is established: In the formula: for Organizational stress at any given moment; It is the relaxation modulus function; For strain rate; t The current moment; t It is the integral variable.

[0054] The ocular soft tissue mechanical modeling module can construct an Ogden-Kelvin-Voigt coupled finite element model based on the above formula, defining the contact constraints, deformation constraints, and damage thresholds between the robotic instrument and the ocular tissue.

[0055] ② Simulation Sample Generation Module: Based on the mechanical model and combined with the constraints of pathological and surgical conditions, the directional generalization of multi-dimensional mechanical parameters is completed, and a standardized multimodal simulation sample library is generated.

[0056] By combining clinical information such as age and lesion type, a mapping relationship of mechanical parameters under pathological conditions is established: In the formula: The elastic modulus of tissue under pathological conditions; This represents the elastic modulus of the tissue under normal conditions. This is a correction function related to age and lesion type.

[0057] Based on this formula, the module completes multi-dimensional directional generalization of eyeball geometry, layered mechanical parameters, robot instrument posture, and contact force, generating a simulation sample library covering multiple pathologies and surgical procedures.

[0058] ③ Real data acquisition and preprocessing module: It interfaces with the vision, depth, and force sensors of ophthalmic surgical robots to perform noise reduction, registration, alignment, and removal of abnormal data from raw data.

[0059] Calculate the signal-to-noise ratio of intraoperative force signals and image signals: In the formula: SNR Signal-to-noise ratio; Signal power; This represents noise power.

[0060] The module connects to the robot's endoscope vision unit, six-dimensional force sensor, and depth sensing unit to complete the noise reduction, registration, temporal alignment, and abnormal data removal of raw data.

[0061] ④ Intelligent filtering module for real data: A mechanical feature evaluation index system was established, and the weighted entropy weighting method was used to complete the quantitative scoring of samples and the selection of high-quality samples, so as to achieve the matching and alignment of mechanical features in the virtual and real domains.

[0062] Calculate the similarity of mechanical characteristics between simulation data and real data: In the formula: Similarity in mechanical features; For simulation stress field data; This is real surgical stress field data.

[0063] Using mechanical similarity, deformation effectiveness, force signal quality, and surgical task matching as evaluation indicators, and combining entropy weight method to complete sample quantification and scoring, high-quality real samples are selected.

[0064] ⑤ Generalized Perception Model Training Module: A multimodal hybrid network model was built, and simulation data pre-training and fine-tuning of high-quality clinical sample transfer were performed.

[0065] CNN spatial feature extraction calculation formula: In the formula: For extracted visual features; is the activation function; W is the convolution kernel weight matrix; X is the input image data; b is the bias term.

[0066] The calculation formula for Transformer self-attention mechanism: In the formula: Attention(Q,K,V) is the self-attention output; Q is the query matrix; K is the key matrix; V is the value matrix; To query the dot product of the key; is the scaling factor; softmax is the normalized exponential function.

[0067] The module integrates visual, depth, and force-sensing multimodal features, uses simulated samples for pre-training and real samples for transfer fine-tuning, and outputs perception results such as lesion location, tissue deformation, and instrument interaction force.

[0068] like Figure 3 As shown, the input includes visual images, depth images, and temporal force data from the ophthalmic surgical robot during surgery. The visual and depth images are fed into the CNN spatial feature extraction module, while the temporal force data is fed into the Transformer temporal feature modeling module. The extracted features from both sources are then fed into the multimodal feature fusion module. The multimodal feature fusion module outputs four perception results in parallel: lesion location coordinates, ocular tissue deformation, interactive force between the robot and ocular soft tissue, and safe operating distance of the robot surgical instruments, meeting the requirements for intelligent perception and safe operation of ophthalmic surgical robots during surgery.

[0069] ⑥ Model generalization iterative optimization module: Responsible for model performance testing and error analysis, and coordinating adjustments to mechanical parameters, generalization range, and screening thresholds to achieve a closed-loop iteration throughout the entire process.

[0070] Set the convergence criteria for iteration: In the formula: For the first t The loss value of each training round.

[0071] The module performs cross-age, cross-pathology, and cross-operative tests on the model, and adjusts the mechanical model parameters, parameter generalization range, and data screening threshold based on the feedback of changes in loss values, forming an integrated closed-loop iterative system.

[0072] ⑦ Human-computer interaction visualization module: It displays the real-time effects of eye biomechanics simulation, the interaction status between robotic instruments and tissues, sample screening results, model training curves, and perception prediction results, and supports staff to manually adjust key parameters.

[0073] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A generalization training method for ophthalmic surgical robots based on a mechanical model, characterized in that: Includes the following steps: S1. Perform fine-grained stratification of the soft tissues of the eyeball and construct a hyperelastic mechanical model: S1.1 Based on the actual anatomical structure of the eyeball, the cornea, sclera, vitreous body, and retina are divided into mutually independent layered structures; S1.2 Collect the elastic modulus, Poisson's ratio, viscosity coefficient, and basic stress-strain parameters of various tissues under normal physiological and typical pathological conditions. S1.

3. The constitutive relations of each soft tissue were constructed using the Ogden hyperelastic model combined with the Kelvin-Voigt viscoelastic model. S1.

4. Establish a three-dimensional layered mesh model of the eyeball using the finite element method, set the contact constraints, deformation range, and tissue damage threshold between the surgical instruments and the eyeball tissue, and characterize the tissue deformation, stress distribution, and mechanical behavior of the interaction between the instruments and soft tissue during the operation. S2. Generalize multi-dimensional mechanical parameters in a targeted manner and construct a simulation sample library: Combining common clinical pathological features with the interactive characteristics of surgical instruments, multi-dimensional joint directional perturbation is performed on the overall size of the eyeball, the mechanical parameters of soft tissues in each layer, the location and extent of lesions, the posture of surgical instruments, the operating force, and environmental noise; multimodal data are collected and labeled simultaneously to construct a large-scale simulation sample library covering different individuals, different pathologies, and different surgical methods. S3. Collect real clinical surgical data and perform preprocessing; S4. Screen clinical data based on correlations of mechanical characteristics; Establish a comprehensive evaluation system centered on ocular biomechanical characteristics and aligned with surgical perception tasks; evaluation dimensions include the effectiveness of tissue deformation, quality of interactive force signals, salience of lesion features, similarity between simulated and real biomechanical characteristics, and matching degree of surgical scenario; We use a weighted entropy weighting method to assign weights and scores to each evaluation index, and select high-quality clinical samples with comprehensive scores higher than a set threshold to form a subset of real and effective samples. This achieves alignment of the mechanical features of the simulation domain with the real clinical domain, reducing the accuracy loss caused by domain offset. S5. Perform pre-training and transfer fine-tuning of the multimodal generalized perception model; A multimodal perception model with a CNN-Transformer hybrid architecture was built. It takes visual, depth, and force multimodal data as input and outputs perception results such as tissue deformation, lesion location coordinates, instrument interaction force, and safe instrument operation distance. S6. Conduct model generalization performance testing and closed-loop iterative optimization; We constructed a test dataset covering different ages, pathological types, and surgical methods to test the accuracy and robustness of the fine-tuned perception model. If the test indicators do not meet the clinical preset standards, the mechanical model parameters, the generalization range of mechanical parameters, and the data screening threshold are adaptively adjusted. The modeling, sample generation, data screening, and model training processes are executed in a loop to form a complete closed-loop iterative optimization mechanism, and finally a generalized perception model of ophthalmic surgical robot that meets the requirements of clinical application is obtained.

2. The generalization training method for an ophthalmic surgical robot based on a mechanical model as described in claim 1, characterized in that: In step S1.3, the coupled constitutive equations are constructed using the Ogden hyperelastic model and the Kelvin Voigt viscoelastic model: In the formula: σ For stress tensor; μ Shear modulus; α These are the constitutive parameters of the material; , , Main elongation ratio; I Unit tensor; η The viscosity coefficient; Let be the strain rate tensor.

3. The generalization training method for an ophthalmic surgical robot based on a mechanical model as described in claim 1, characterized in that: In step S1.4, the tissue deformation constraints that ensure surgical safety are as follows: In the formula: u To organize displacement, The maximum safe deformation threshold; And, the contact force constraints between the robotic device and the eye tissue: In the formula: F c For the contact force of the instrument, F max For maximum safe interaction.

4. The generalization training method for an ophthalmic surgical robot based on a mechanical model as described in claim 1, characterized in that: In step S2, the acquired multimodal data includes acquired visual images, depth information, instrument interaction forces, tissue deformation displacement, and stress field.

5. The generalization training method for an ophthalmic surgical robot based on a mechanical model as described in claim 1, characterized in that: In step S2, the various parameters of the eyeball are corrected by multi-dimensional joint directional perturbation: In the formula: E The original elastic modulus, This is the corrected elastic modulus; η The original viscosity coefficient, The corrected viscosity coefficient, R The original geometric dimensions of the eyeball. The corrected eyeball geometry; γ The random perturbation coefficient is the value within the interval [0,1]. , , This represents the corresponding pathological disturbance coefficient.

6. The generalization training method for an ophthalmic surgical robot based on a mechanical model as described in claim 1, characterized in that: In step S3, visual, force, and depth multimodal raw data are collected during ophthalmic clinical surgery. Signal denoising, multimodal data registration, and temporal-spatial alignment are performed sequentially to remove invalid samples with blurred images, abnormal force signals, and missing data, resulting in a well-organized and usable real clinical dataset. The formula for calculating the alignment error during temporal and spatiotemporal alignment is as follows: In the formula: E a This refers to the spatiotemporal alignment error of multimodal data. , , The spatial coordinates and timestamps for the first mode; , , These are the spatial coordinates and timestamps for the second mode.

7. The generalization training method for an ophthalmic surgical robot based on a mechanical model as described in claim 1, characterized in that: In step S4, the information entropy of a single indicator is first calculated: In the formula: e i For the first i Information entropy of the indicator; K The normalization coefficient is... K =1 / ln( n ), n The total number of samples; For the first j The sample at the th i Standardized percentage of each indicator; ln is the natural logarithm; Calculate the weights of each indicator based on information entropy: In the formula: For the first i The weight of each indicator; For the first i Information entropy of the indicator; This is the sum of the information entropy redundancy of all indicators; Calculate the comprehensive score for a single sample by combining the indicator weights: In the formula: Score is the overall score of the sample; , , , , The weights of the five evaluation indicators; , , , , The standardized scores of a single sample across the five metrics.

8. The generalization training method for an ophthalmic surgical robot based on a mechanical model as described in claim 1, characterized in that: In step S5, the total loss function expression of the multimodal perception model is: In the formula: For location perception loss, To perceive loss, For the loss of safe distance, , This is the loss balance coefficient; The training process uses an exponential decay method to adjust the learning rate, and the learning rate update formula is: In the formula: For the first The learning rate of the round; The initial learning rate; This is the current training round; This is the learning rate decay coefficient.

9. The generalization training method for an ophthalmic surgical robot based on a mechanical model as described in claim 1, characterized in that: In step S6, generalization performance testing is performed on the clinical data, and the location perception error is calculated using the following formula: In the formula: This is due to position perception error; The total number of test samples; , The coordinates of the instrument position predicted by the model; , These are the actual coordinates of the instruments during clinical surgery. Force sensing error calculation formula: In the formula: For force sensing error; The total number of test samples; The instrument contact force predicted by the model; This refers to the actual contact force of instruments during clinical surgery.

10. A generalization training system for an ophthalmic surgical robot based on a mechanical model, based on the training method described in any one of claims 1 to 9, characterized in that: include: Ocular soft tissue biomechanical modeling module: responsible for inputting biomechanical parameters of each layer of tissue, completing hyperelastic constitutive modeling, finite element mesh generation, and defining contact constraints and deformation thresholds; Simulation sample generation module: Based on the mechanical model and combined with the constraints of pathological and surgical conditions, it completes the directional generalization of multi-dimensional mechanical parameters and generates a standardized multimodal simulation sample library; Real data acquisition and preprocessing module: Connects to the vision, depth, and force sensing devices of ophthalmic surgical robots to complete noise reduction, registration, alignment, and abnormal data removal of raw data; Real data intelligent screening module: Establishes a mechanical feature evaluation index system, uses weighted entropy weighting method to complete sample quantitative scoring and high-quality sample screening, and realizes the matching and alignment of mechanical features in the virtual and real domains; Generalized perception model training module: Build a multimodal hybrid network model and perform pre-training on simulation data and fine-tuning on high-quality clinical samples; Model generalization iterative optimization module: responsible for model performance testing and error analysis, and adjusting mechanical parameters, generalization range and screening threshold in a coordinated manner to achieve closed-loop iteration throughout the entire process; Human-computer interaction visualization module: centrally displays the eye biomechanics simulation effect, sample selection results, model training process and perception output results, and supports manual viewing and adjustment of key parameters.

Images