Image-based ophthalmic robotic control methods, systems, devices, and media

Abstract

The application provides an image-based ophthalmic robot control method, system, device and medium, wherein the method comprises: acquiring an intraoperative three-dimensional OCT image sequence and a visual evoked potential signal, segmenting an optic nerve region to generate an optic nerve spatial model; performing dynamic registration on the optic nerve spatial model and the visual evoked potential signal according to an eye movement compensation algorithm to generate a fusion data body; calculating a visual function damage risk coefficient and a postoperative visual acuity recovery prediction value through a pre-trained neural network prediction model based on the fusion data body and real-time operation parameters of a robot end instrument; and dynamically generating a robot operation correction control instruction according to the damage risk coefficient and the postoperative visual acuity recovery prediction value. The method can improve the individualization accuracy of a surgical plan and the postoperative visual function protection effect.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of ophthalmic robot control, and particularly relates to an image-based ophthalmic robot control method, system, device and medium. BACKGROUND

[0002] With the development of ophthalmic surgery robots and medical imaging technology, the control system based on OCT, intraoperative microscope and other images can realize accurate positioning of surgical instruments in intraocular tissues. By segmenting key structures such as the retina and optic nerve, and combining robot motion planning algorithms, the operation safety is improved. In the traditional method, the postoperative clinical follow-up is completely dependent on the evaluation of the surgical effect, and the surgeon only adjusts the instrument parameters according to the anatomical position and operating experience during the operation, lacking real-time prediction ability of the postoperative visual function recovery effect of the patient.

[0003] However, the damage to the optic nerve function is often caused by irreversible operations in the operation (such as excessive compression and energy accumulation), and the existing technology cannot dynamically capture the optic nerve electrophysiological signals (such as visual evoked potentials), nor can it establish a predictive correlation between intraoperative operation parameters and postoperative vision, making it difficult to optimize the operation strategy in real time according to the individualized functional recovery needs. SUMMARY

[0004] Therefore, it is necessary to provide an image-based ophthalmic robot control method, system, device and medium, which can construct a predictive correlation between operation parameters and postoperative visual recovery effect through real-time fusion analysis of intraoperative optic nerve bioelectric signals and multi-modal images, so as to improve the individualized accuracy of the operation plan and the postoperative visual function protection effect.

[0005] In a first aspect, the application provides an image-based ophthalmic robot control method, comprising:

[0006] Obtaining an intraoperative three-dimensional OCT image sequence and a visual evoked potential signal, segmenting the optic nerve region to generate an optic nerve spatial model;

[0007] According to the eye movement compensation algorithm, the optic nerve spatial model and the visual evoked potential signal are dynamically registered to generate a fusion data body;

[0008] Based on the fusion data body and the real-time operation parameters of the robot end instrument, the visual function damage risk coefficient and the postoperative visual recovery prediction value are calculated through a pre-trained neural network prediction model;

[0009] According to the damage risk coefficient and the postoperative visual recovery prediction value, a robot operation correction control instruction is dynamically generated.

[0010] In one of the embodiments, the optic nerve spatial model and the visual evoked potential signal are dynamically registered according to an eye movement compensation algorithm to generate a fusion data body, including:

[0011] The three-dimensional OCT image sequence of adjacent frames is subjected to optical flow field analysis to generate a tissue displacement vector field;

[0012] A dynamic correction matrix is constructed according to the tissue displacement vector field;

[0013] The collection point coordinates of the visual evoked potential signal are mapped to the corresponding nodes of the optic nerve spatial model through the dynamic correction matrix;

[0014] The amplitude parameter and the latency offset parameter of the visual evoked potential signal are bound to the mapped nodes to generate the fusion data body.

[0015] In one of the embodiments, based on the fusion data body and the real-time operation parameters of the robot end instrument, the visual function damage risk coefficient and the postoperative visual acuity recovery prediction value are calculated through a pre-trained neural network prediction model, including:

[0016] Based on the amplitude parameter, the latency offset parameter, and the real-time pressure change rate of the robot end instrument, a dynamic optic nerve function index is calculated;

[0017] The dynamic optic nerve function index and the real-time pressure are input into a pre-trained convolutional recurrent neural network to output the visual function damage risk coefficient;

[0018] Based on the functional relationship between the visual function damage risk coefficient and the preoperative baseline visual acuity, the postoperative visual acuity recovery prediction value is calculated.

[0019] In one of the embodiments, the robot operation correction control instruction is dynamically generated according to the damage risk coefficient and the postoperative visual acuity recovery prediction value, including:

[0020] Based on the real-time position coordinates of the robot end instrument in the three-dimensional OCT image sequence, a current surgical stage identifier is generated through a pre-set surgical stage division rule;

[0021] It is judged whether the current surgical stage identifier belongs to a pre-set high-risk stage set;

[0022] If the current surgical stage identifier belongs to the pre-set high-risk stage set, the maximum allowable moving speed and the pressure safety threshold are calculated according to the visual function damage risk coefficient;

[0023] If the postoperative visual acuity recovery prediction value is less than a pre-set recovery threshold, a motion pause instruction is generated and an alarm signal is triggered.

[0024] In one of the embodiments, the dynamic correction matrix is constructed according to the tissue displacement vector field, further including:

[0025] affine transformation modeling on the tissue displacement vector field to generate a spatial compensation parameter matrix, the spatial compensation parameter matrix comprising translation, rotation and scaling parameters;

[0026] performing spatial transformation operation on the original coordinates of the visual evoked potential signal collection points by using the spatial compensation parameter matrix to generate corrected spatial coordinates;

[0027] screening an effective mapping node set according to the Euclidean distance between the corrected spatial coordinates and the nodes in the optic nerve spatial model;

[0028] updating the spatial coordinate distribution of the fusion data volume based on the effective mapping node set.

[0029] In one embodiment, the training method of the convolutional recurrent neural network comprises:

[0030] constructing a training data set, the training data set containing preoperative three-dimensional OCT image features, intraoperative instrument operation parameter sequences and postoperative clinical visual function evaluation results;

[0031] generating instrument operation parameter and visual function damage correlation simulation data by the generative adversarial network, the correlation simulation data being used to expand the training data set;

[0032] extracting the time sequence dependent features of the instrument operation parameters using the long short-term memory module, and weighting and fusing the time sequence dependent features and the image features through the attention mechanism;

[0033] optimizing the network weight parameters through the back propagation algorithm, so that the KL divergence between the visual function damage risk coefficient output by the convolutional recurrent neural network and the postoperative clinical evaluation result converges to a set range.

[0034] In one embodiment, the image-based ophthalmic robot control method further comprises:

[0035] In the postoperative verification stage, the visual function damage risk coefficient sequence recorded in the historical operation process and the postoperative visual recovery prediction value of the key operation points are obtained;

[0036] performing Gaussian mixture model clustering analysis on the visual function damage risk coefficient sequence and the postoperative visual recovery prediction value to generate a high-risk operation mode feature set;

[0037] based on the high-risk operation mode feature set and the anatomical structure features of the preoperative three-dimensional OCT image, generating a prognosis optimization score report through a random forest algorithm, the prognosis optimization score report including surgery path planning suggestions and risk avoidance strategies.

[0038] In a second aspect, the application also provides an image-based ophthalmic robot control system, comprising:

[0039] An image segmentation module is configured to acquire an intraoperative three-dimensional OCT image sequence and a visual evoked potential signal, segment an optic nerve region to generate an optic nerve spatial model;

[0040] A data fusion module is configured to perform dynamic registration on the optic nerve spatial model and the visual evoked potential signal according to an eye movement compensation algorithm to generate a fusion data volume;

[0041] A visual function prediction analysis module is configured to calculate a visual function damage risk coefficient and a postoperative visual acuity recovery prediction value based on the fusion data volume and real-time operation parameters of a robot end instrument through a pre-trained neural network prediction model;

[0042] A control instruction generation module is configured to dynamically generate a robot operation correction control instruction according to the damage risk coefficient and the postoperative visual acuity recovery prediction value.

[0043] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned image-based ophthalmic robot control method when executing the computer program.

[0044] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the above-mentioned image-based ophthalmic robot control method.

[0045] The above-mentioned image-based ophthalmic robot control method, system, device and medium generate an optic nerve spatial model by acquiring an intraoperative three-dimensional OCT image sequence and a visual evoked potential signal, perform dynamic registration on the model and a bioelectric signal to generate a fusion data volume based on an eye movement compensation algorithm, then calculate a visual function damage risk coefficient and a postoperative visual acuity recovery prediction value by using a pre-trained neural network prediction model combined with real-time operation parameters of an instrument, and finally dynamically generate a robot operation correction control instruction according to the prediction indexes. This technical solution solves the problem of intraoperative optic nerve function dynamic monitoring, realizes quantitative correlation prediction of surgical operation parameters and postoperative visual acuity recovery effect by establishing a real-time fusion analysis mechanism of anatomical structure and bioelectric signal, forms an intraoperative closed-loop control oriented to prognosis, and thus improves the individualized precision of surgical operation and the visual function protection effect. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments or the related art, the following will briefly introduce the drawings needed to be used in the embodiment or the related art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0047] Figure 1 A flowchart of an image-based ophthalmic robot control method provided by an embodiment of the present application is shown in FIG. 1.

[0048] Figure 2 A structural diagram of an image-based ophthalmic robot control system provided by an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0049] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0050] First, the terms involved in the embodiments of the present application are briefly introduced.

[0051] Intraoperative three-dimensional OCT image sequence refers to a three-dimensional image data set of ocular tissue continuously collected by an optical coherence tomography (OCT) device during surgery. The image sequence presents the cross-sectional and longitudinal sectional morphology of structures such as the retina and optic nerve in layers with a resolution of microns, providing real-time dynamic change information of anatomical structures during surgery. Unlike static images, its time continuity supports tissue displacement tracking, providing a spatial reference for motion compensation algorithms. In ophthalmic robot surgery, this sequence is the basic data source for constructing an optic nerve spatial model, ensuring the operation accuracy of fine anatomical structures.

[0052] Visual evoked potential signal (VEP) refers to a cortical bioelectric signal induced by visual stimulation, reflecting the functional integrity of the optic nerve pathway. Intraoperative VEP is collected by placing electrodes at specific points on the patient's scalp. Its core parameters include amplitude (reflecting the degree of synchronous excitation of nerve cells) and latency (reflecting the speed of nerve conduction). In the technical solution, VEP signal is used as a real-time functional monitoring indicator to quantify the instantaneous impact of surgical operation on the optic nerve function, making up for the deficiency of pure anatomical images in assessing nerve activity.

[0053] Optic nerve spatial model refers to a digital three-dimensional geometric model of the optic nerve generated based on three-dimensional OCT image segmentation. The model extracts the boundary of the optic nerve and reconstructs its three-dimensional topological structure through image segmentation algorithms, and can contain the coordinate spatial position information of key areas such as the optic disc and nerve fiber layer. Its value lies in abstracting anatomical structures into computer-processable mathematical entities, providing a stable reference system for dynamic registration and supporting collision avoidance calculations in robot path planning.

[0054] According to the above glossary, the implementation environment of the image-based ophthalmic robot control method provided in the embodiments of the present application is described. The implementation environment includes a terminal, a sensor array, a processor, and a storage device. The sensor array includes but is not limited to an OCT imaging sensor, a visual evoked potential electrode sensor, a robot end force / position sensor, an eye movement tracking sensor, an optical positioning tracking system, etc. The processor can be a central processing unit, a graphics processing unit, a multi-core processor, or an artificial intelligence chip, etc. The storage device can be a distributed storage device or a centralized storage device, which is not limited here.

[0055] In combination with the above glossary and implementation environment, the application scenarios of the embodiments of the present application are described. The image-based ophthalmic robot control method provided in the embodiments of the present application can be applied to the following scenarios, including but not limited to:

[0056] In the field of ophthalmic minimally invasive surgery, such as glaucoma minimally invasive surgery, the technical solution also has significant application value. Although minimally invasive surgery has small trauma, the operation space is limited, and the operation precision is extremely high. In the surgical process, the multi-modal data fusion analysis result obtained by using the technical solution can help the doctor to more accurately locate the target region of the operation, such as the structure of the trabecular meshwork, and to monitor the interaction between the surgical instrument and the surrounding tissue in real time. Through dynamically generating robot operation correction control instructions, the robot can automatically adjust the motion trajectory and operation parameters of the surgical instrument according to the predicted visual function damage risk and postoperative visual recovery, to ensure the precision and safety of the operation. This not only helps to improve the success rate of the operation, but also reduces the occurrence of postoperative complications and promotes the rapid recovery of the visual function of the patient after the operation.

[0057] Illustratively, the coal mine personnel safety situation dynamic perception method provided in the embodiments of the present application can also be applied to other application scenarios, which are only exemplified here and are not limited to specific application scenarios.

[0058] In an exemplary embodiment, as shown in Figure 1 An image-based ophthalmic robot control method is provided. The method is applied to the terminal in the implementation environment described above. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and can be realized through the interaction of the terminal and the server. The method includes the following steps 101 to 104:

[0059] Step 101: acquiring an intraoperative three-dimensional OCT image sequence and a visual evoked potential signal, and segmenting a optic nerve region to generate an optic nerve spatial model.

[0060] Exemplarily, the intraoperative three-dimensional OCT image sequence is collected in real time by a high-resolution optical coherence tomography (OCT) device, which can provide high-precision anatomical structure information of the intraocular tissue, including detailed images of key structures such as the retina and optic nerve; at the same time, visual evoked potential signals are synchronously collected by professional electrophysiological monitoring equipment, which reflect the real-time functional state of the optic nerve during the operation and are an important indicator for evaluating whether the optic nerve is damaged. Specifically, the optic nerve region can be identified by using an improved U-Net segmentation network on the OCT image sequence: after inputting the original OCT image, multi-scale features are extracted by the encoder, spatial attention mechanism is embedded in the skip connection to enhance the optic disc edge feature response, and the decoder outputs the pixel-level segmentation mask. Further, the mask sequence is converted into a three-dimensional optic nerve mesh model with topological connection relationship by using the Poisson surface reconstruction algorithm, which contains the node coordinates and normal vector information of the optic nerve fiber layer. For example, the segmentation network uses a multi-center data set to enhance generalization in the training stage, and adds a motion artifact adversarial training module to eliminate the interference of intraoperative shaking, and the generated optic nerve spatial model provides a spatial reference coordinate system for dynamic registration.

[0061] Step 102, according to the eye movement compensation algorithm, the optic nerve spatial model and the visual evoked potential signal are dynamically registered to generate a fusion data body.

[0062] Specifically, since the eyeball may move slightly during the operation, which may cause a spatio-temporal mismatch between the OCT image and the VEP signal. In order to eliminate this mismatch, the eye movement compensation algorithm can monitor the movement state of the eyeball in real time and make corresponding adjustments to the optic nerve spatial model. Exemplarily, this algorithm can be based on eye tracking technology, and by monitoring the movement trajectory of the eyeball, the optic nerve spatial model is dynamically corrected. Further, the compensated optic nerve spatial model is fused with the visual evoked potential signal to generate a fusion data body. The fusion data body integrates the anatomical structure and functional state information of the optic nerve, providing more comprehensive data support for the subsequent prediction model. For example, multi-modal data fusion techniques such as feature fusion or decision fusion based methods can be used to effectively integrate the optic nerve spatial model and the visual evoked potential signal. Through this dynamic registration and fusion process, the method can ensure the accuracy and consistency of the data, providing a reliable basis for subsequent prediction analysis.

[0063] Step 103, based on the fusion data body and the real-time operation parameters of the robot end instrument, the pre-trained neural network prediction model is used to calculate the visual function damage risk coefficient and the postoperative visual acuity recovery prediction value.

[0064] Specifically, the real-time operation parameters of the robotic end-effector instrument include the position, angle, force, and other information of the instrument, which directly affect the visual function status during the operation. By inputting the fused data volume and these real-time operation parameters into the pre-trained neural network prediction model, the model can calculate the visual function damage risk coefficient and postoperative visual acuity recovery prediction value based on a large amount of historical data and complex nonlinear relationships. For example, the neural network prediction model can use a deep learning architecture such as long short-term memory (LSTM) or convolutional neural network (CNN) to process time series data and spatial data. Further, the training process of the model can utilize a large amount of clinical data, including preoperative, intraoperative, and postoperative visual function evaluation results, to improve the accuracy and generalization ability of the model. For example, during the training process, data augmentation techniques such as random noise addition or data cropping can be used to improve the robustness of the model. Through this deep learning-based prediction method, the method can real-time assess the visual function risk during the operation and predict the postoperative visual acuity recovery, providing a scientific basis for the operation.

[0065] Step 104, dynamically generating robot operation correction control instructions according to the damage risk coefficient and the postoperative visual acuity recovery prediction value.

[0066] Specifically, according to the preset risk threshold and prediction target, the risk level and expected effect of the current operation can be real-time evaluated. If the damage risk coefficient exceeds the set threshold or the postoperative visual acuity recovery prediction value is lower than the expected target, the method can automatically adjust the operation parameters of the robotic end-effector instrument and generate correction control instructions. For example, the correction control instructions can include adjusting the motion trajectory, force, or operation speed of the instrument to reduce the risk of visual function damage and improve the possibility of postoperative visual acuity recovery. Further, these correction control instructions will be fed back to the robot control system in real time to ensure that the operation can be dynamically adjusted according to the real-time evaluation results. For example, if the prediction model finds that the current operation may cause excessive compression of the optic nerve, the force or position of the instrument can be automatically adjusted to avoid potential damage. Through this dynamic adjustment mechanism, the method can real-time optimize the operation and improve the safety and effectiveness of the operation.

[0067] The image-based ophthalmic robot control method, system, device and medium described above generate a optic nerve spatial model by obtaining an intraoperative three-dimensional OCT image sequence and a visual evoked potential signal, dynamically register the model and the bioelectric signal based on an eye movement compensation algorithm to generate a fusion data body, then use a pre-trained neural network prediction model combined with real-time instrument operation parameters to calculate a visual function damage risk coefficient and a postoperative visual acuity recovery prediction value, and finally generate a robot operation correction control instruction dynamically according to the prediction indexes. The technical solution solves the problem of the lack of dynamic monitoring of the optic nerve function during surgery, realizes the quantitative correlation prediction of the surgical operation parameters and the postoperative visual acuity recovery effect by establishing a real-time fusion analysis mechanism of the anatomical structure and the bioelectric signal, forms an intraoperative closed-loop control oriented to the prognosis, and thus improves the individualized precision of the surgical operation and the protection effect of the visual function.

[0068] In one embodiment, the optic nerve spatial model and the visual evoked potential signal are dynamically registered according to the eye movement compensation algorithm to generate a fusion data body, including:

[0069] Optical flow field analysis is performed on the three-dimensional OCT image sequence of adjacent frames to generate a tissue displacement vector field.

[0070] Illustratively, the optical flow field analysis is performed on the three-dimensional OCT image sequence of adjacent frames to generate a tissue displacement vector field. Optical flow field analysis is a computer vision technique used to estimate the direction and speed of pixel motion in an image sequence. In this embodiment, by analyzing the three-dimensional OCT image sequence of adjacent frames, the displacement of intraocular tissues can be accurately calculated to generate a tissue displacement vector field to capture the micro-movement of the eyeball during surgery and provide a basis for subsequent dynamic correction. Illustratively, classical optical flow algorithms such as the Lucas-Kanade algorithm or the Farneback algorithm can be used, which can effectively process three-dimensional image sequences to generate accurate displacement vector fields. The real-time deformation of tissues caused by physiological tremor or instrument operation is accurately quantified.

[0071] A dynamic correction matrix is constructed according to the tissue displacement vector field.

[0072] Specifically, the role of the dynamic correction matrix is to adjust the coordinates of the visual evoked potential signal acquisition points to keep them consistent with the coordinate system of the optic nerve spatial model. Specifically, the displacement information in the tissue displacement vector field is converted into coordinate adjustment parameters through matrix operations to construct the dynamic correction matrix. This process ensures the accurate alignment of the visual evoked potential signal and the optic nerve spatial model in space, providing a basis for subsequent data fusion. For example, the dynamic correction matrix can be constructed through linear transformation or nonlinear transformation, depending on the complexity and accuracy requirements of the tissue displacement.

[0073] The collection point coordinates of the visual evoked potential signals are mapped to the corresponding nodes of the optic nerve spatial model through the dynamic correction matrix.

[0074] Specifically, after the adjustment of the dynamic correction matrix, the collection point coordinates of the visual evoked potential signals can be accurately mapped to the corresponding positions in the optic nerve spatial model, so as to spatially bind the functional signals with the anatomical structure, so that the visual evoked potential signals of each point can correspond to the specific position in the optic nerve spatial model. For example, the collection point coordinates can be accurately mapped to the nodes of the optic nerve spatial model through an interpolation algorithm or a nearest neighbor algorithm, ensuring the accuracy and reliability of the mapping.

[0075] The amplitude parameter and the latency offset parameter of the visual evoked potential signals are bound to the mapped nodes to generate a fusion data body.

[0076] Specifically, the amplitude parameter and the latency offset parameter of the visual evoked potential signals are bound to the corresponding nodes of the optic nerve spatial model to form a fusion data body containing anatomical structure and functional state. The amplitude parameter reflects the intensity of the optic nerve signal, and the latency offset parameter reflects the delay of the signal, both of which are important indicators for evaluating the functional state of the optic nerve. By binding these parameters to the nodes of the optic nerve spatial model, the generated fusion data body can provide comprehensive information of the intraocular tissue, providing more abundant data support for subsequent analysis and prediction. For example, the amplitude parameter can be determined by the peak value of the signal, and the latency offset parameter can be calculated by the difference between the starting time of the signal and the standard time. Through this binding method, the fusion data body not only contains the anatomical structure of the optic nerve, but also contains the real-time information of its functional state, providing more comprehensive guidance for surgical operation. The above embodiments can realize accurate dynamic registration and fusion of intraoperative optic nerve bioelectric signals and multi-modal images, improve the safety and effectiveness of the operation, and also provide a data basis for real-time evaluation of the functional state of the optic nerve, enhancing the postoperative visual function protection effect.

[0077] In one of the embodiments, based on the fusion data body and the real-time operation parameters of the robot end instrument, the visual function damage risk coefficient and the postoperative visual acuity recovery prediction value are calculated through a pre-trained neural network prediction model, including:

[0078] Based on the amplitude parameter, the latency offset parameter, and the real-time pressure change rate of the robot end instrument, a dynamic optic nerve function index is calculated.

[0079] Specifically, the core bioelectric parameters in the fusion data body are continuously acquired during the operation, including the amplitude parameter of the visual evoked potential signal (reflecting the synchronous discharge intensity of the neural cell group) and the latency shift parameter (indicating the change in neural conduction velocity), while the pressure rate of change of the robot end instrument in the XYZ axial direction (characterizing the instantaneous mechanical stimulation intensity of the instrument on the tissue) is extracted from the six-dimensional force sensor. Based on the above three groups of dynamic parameters, a dynamic optic nerve function index is constructed, including: normalizing the amplitude parameter to eliminate individual potential amplitude base differences; calculating the first derivative of the latency shift to capture the deterioration trend of neural conduction block; and nonlinearly weighting the pressure rate of change and the aforementioned bioelectric parameters to form a scalar index that comprehensively reflects the real-time functional status of the optic nerve. This index can sensitively identify the decline in neural conduction efficiency caused by optic disc compression in glaucoma surgery, such as the abnormal increase in latency derivative when the pressure rate of change increases sharply. Further, the dynamic optic nerve function index can be calculated by weighted summation, in which the amplitude parameter, the latency shift parameter, and the real-time pressure rate of change are respectively assigned different weights to reflect their importance in the evaluation of optic nerve function. For example, the weight of the amplitude parameter can be set to 0.4, the weight of the latency shift parameter can be set to 0.3, and the weight of the real-time pressure rate of change can be set to 0.3, thereby obtaining a comprehensive dynamic optic nerve function index for subsequent prediction analysis.

[0080] The dynamic optic nerve function index and the real-time pressure are input into a pre-trained convolutional recurrent neural network, and the output is a visual function damage risk coefficient.

[0081] Specifically, the convolutional recurrent neural network (CRNN) model pre-trained in this embodiment undertakes the core prediction task. The convolutional recurrent neural network (Convolutional Recurrent Neural Network, CRNN) is a deep learning architecture that combines the characteristics of convolutional neural network (CNN) and recurrent neural network (RNN). It combines the advantages of CNN in processing spatial data and the advantages of RNN in processing time series data, and can effectively process complex data containing both spatial information and time information. In this application, CRNN is used to process the spatial features in the fusion data body and the time series features of the real-time operation parameters of the robot end instrument to achieve accurate prediction of the risk of visual function damage. Exemplarily, in this embodiment, the input layer of the network is designed as a dual-channel architecture, in which: the first channel receives the time series flow data of the DNFI, extracts the feature decay pattern through two layers of long short-term memory (LSTM) units, and pays special attention to the gradient mutation features within a 15 ms time window; the second channel inputs the spatial distribution map of the instrument operation pressure, and uses a three-dimensional convolution kernel to scan the pressure conduction hot spot on the optic nerve spatial model. After the dual-channel output is fused through the attention gate mechanism, the visual function damage risk coefficient (0-1 continuous value) is output through the regression layer. During model training, a multi-center surgery data set is used to enhance robustness, and a gradient penalty mechanism is introduced to prevent overfitting, ensuring that in complex scenarios such as diabetic retinopathy, high-risk operation modes (such as when the ultrasonic emulsification probe approaches the optic nerve, the DNFI decays exponentially) can still be identified stably.

[0082] Based on the functional relationship between the visual function damage risk coefficient and the preoperative baseline visual acuity, the postoperative visual acuity recovery prediction value is calculated.

[0083] Specifically, the postoperative visual recovery prediction is achieved based on the dynamic correlation of the damage risk and the preoperative baseline. Exemplarily, by reading the key parameters in the preoperative visual function profile of the patient, including the mean defect value (MD) of static visual field and the baseline amplitude of pattern visual evoked potential P100 wave, the real-time damage risk coefficient during the operation is input into the prognosis function model: a bivariate interaction equation is established to consider the buffering effect of preoperative optic nerve compensation on intraoperative damage; a time integral operation is introduced to accumulate the exposure time of high-risk operation in the entire surgical stage; and a postoperative visual recovery prediction value (Snellen visual acuity percentage) is output. This mechanism can predict the prognosis difference of different dissection paths in macular surgery, for example, when the cumulative risk value exceeds the compensation threshold, it automatically prompts to choose the temporal approach to avoid central foveal function damage. This embodiment constructs a high-sensitivity functional index by dynamically weighting and fusing multiple source parameters, analyzes the complex mapping relationship between mechanical stimulation and neural response by using a double-channel spatiotemporal joint modeling, and realizes individualized prognosis prediction based on preoperative and intraoperative parameter coupling algorithm. In clinical application, the visual function evaluation time point can be advanced from postoperative to intraoperative decision-making link, so that the doctor can adjust the surgical procedure according to the prediction value during the key operation stage such as retinal peeling, thereby reducing the risk of irreversible optic nerve damage.

[0084] In one embodiment, the robot operation correction control instruction is dynamically generated according to the damage risk coefficient and the postoperative visual recovery prediction value, including:

[0085] Based on the real-time position coordinates of the robot end instrument in the three-dimensional OCT image sequence, the current surgical stage identifier is generated by a preset surgical stage division rule.

[0086] Specifically, based on the real-time spatial coordinates of the robot end instrument in the three-dimensional OCT image sequence (obtained by registering the instrument retroreflective marker point and the OCT voxel coordinates), the stage intelligent division is performed in combination with the surgical procedure knowledge base. For example, the preset surgical stage division rule can include multi-dimensional condition judgment: when the instrument tip enters the space range of 0.5 mm from the optic disc, it is marked as "optic nerve adjacent stage"; when the change rate of vitreous proliferation membrane traction force detected exceeds 0.3 N / s, it is marked as "proliferation membrane peeling stage". These space and mechanical parameters jointly constitute the stage division basis to generate a surgical stage identifier with clinical semantics.

[0087] Determine whether the current surgical stage identifier belongs to the preset high-risk stage set.

[0088] Exemplarily, when judging the risk attribute of the stage, a preset high-risk stage set can be called for matching comparison, which is dynamically configured according to the type of surgery before surgery: for example, in the surgery of diabetic retinopathy, "epipapillary membrane peeling" and "macular pre-membrane hooking" are included in the high-risk set. The matching mechanism adopts a double-verification strategy, for example, first screening through the positional relationship between the instrument coordinates and the anatomical partition of the optic nerve, and then combining the current pressure fluctuation spectrum characteristics (such as the energy increase of the 0.5-2Hz frequency band) for secondary confirmation. When the spatial positioning condition and the mechanical characteristic condition are met at the same time, it is determined that there is a risk of mechanical damage to the optic nerve in the current stage.

[0089] If the current surgery stage identifier belongs to the preset high-risk stage set, the maximum allowable movement speed and the pressure safety threshold are calculated according to the visual function damage risk coefficient.

[0090] Specifically, based on the real-time visual function damage risk coefficient, the mechanical operation limit parameter is calculated through a non-linear mapping function, wherein the calculation formula of the maximum allowable movement speed is:

[0091] ;

[0092] Wherein, is the maximum allowable movement speed, is the current surgery stage reference speed, which is obtained from the preset initial speed of safe operation of the instrument determined by the preset surgical knowledge base, is the instrument attenuation coefficient, which represents the adjustment factor of the sensitivity of different instruments to risk, is the visual function damage risk coefficient;

[0093] The calculation formula of the pressure safety threshold is:

[0094] ;

[0095] Wherein, is the pressure safety threshold, is the critical pressure stress of the optic nerve (for example, glaucoma patient = 25mN), is the cumulative damage factor (positively correlated with the preoperative cup-to-disc ratio of the optic nerve), is the duration of the current stage.

[0096] If the postoperative visual recovery prediction value is less than the preset recovery threshold, a motion pause instruction is generated and an alarm signal is triggered.

[0097] Specifically, the postoperative visual acuity recovery prediction value is calculated based on the visual function impairment risk coefficient and the preoperative baseline visual acuity, which reflects the possibility of postoperative visual acuity recovery of the patient. The preset recovery threshold is preset according to clinical experience and surgical goals, and is used to judge whether the postoperative visual acuity recovery reaches the expected goal. If the postoperative visual acuity recovery prediction value is lower than the preset recovery threshold, it means that the current surgical operation may cause poor postoperative visual acuity recovery. In this case, a motion pause instruction is generated to immediately stop the motion of the robot end instrument to prevent further damage; at the same time, an alarm signal is triggered to remind the surgeon to pay attention to the risk of the current operation, so as to take timely measures for adjustment. Through this mechanism, the method can intervene in time when the postoperative visual acuity recovery is poor, and establishes an active protection paradigm for ophthalmic robot surgery, ensuring the safety and effectiveness of the surgery.

[0098] In one of the embodiments, the dynamic correction matrix is constructed according to the tissue displacement vector field, further comprising:

[0099] The affine transformation of the tissue displacement vector field is modeled to generate a spatial compensation parameter matrix, and the spatial compensation parameter matrix includes translation, rotation and scaling parameters;

[0100] The original coordinates of the visual evoked potential signal acquisition points are subjected to spatial transformation operation by using the spatial compensation parameter matrix to generate corrected spatial coordinates;

[0101] According to the Euclidean distance between the corrected spatial coordinates and the nodes in the optic nerve spatial model, an effective mapping node set is screened;

[0102] The spatial coordinate distribution of the fusion data body is updated based on the effective mapping node set.

[0103] Exemplarily, the embodiment constructs a dynamic correction matrix according to the in-situ acquired tissue displacement vector field, and realizes dynamic registration of the visual evoked potential signal and the anatomical structure. The method first models the affine transformation of the tissue displacement vector field, which is derived from the optical flow field analysis result of the three-dimensional OCT image sequence and reflects the real-time spatial deformation of the living tissue. Specifically, the singular value decomposition algorithm can be used to fit the optimal geometric transformation relationship to generate a spatial compensation parameter matrix containing a translation vector, a rotation matrix and a scaling factor. The matrix quantifies the spatial drift of the optic nerve caused by the compression of the surgical instrument or the breathing fluctuation, for example, in macular surgery, the rigid displacement component caused by the retina traction can be accurately compensated. Further, the spatial compensation parameter matrix is used to perform spatial transformation operation on the original coordinates of the visual evoked potential signal acquisition points. The original coordinates are provided by the optical positioning system of the scalp electrode, and the acquisition points are mapped to the OCT image coordinate system through homogeneous coordinate conversion, affine transformation calculation is performed to generate corrected spatial coordinates, and double-precision floating-point operation is used to ensure spatial accuracy to eliminate the electrode position deviation caused by in-situ medium disturbance, for example, to eliminate the 0.2 millimeter level coordinate offset caused by the ultrasonic emulsification probe vibration, and to improve the spatial positioning accuracy of the bioelectric signal. The method selects an effective mapping node set according to the Euclidean distance between the corrected spatial coordinates and the nodes in the optic nerve spatial model. The processing process introduces a weighted distance measurement mechanism to narrow the distance threshold for anatomical regions with significant curvature characteristics. The nearest neighbor search is accelerated by establishing a spatial topological index tree to screen out a node cluster with a distance error less than a preset threshold. This operation realizes high-precision spatial association between the signal and the anatomical structure, for example, effectively eliminates non-physiological mapping points in the optic disc edge area, and reduces the mismatch rate to one third of the traditional method. The spatial coordinate distribution of the fusion data body is updated based on the effective mapping node set, the node coordinate attributes of the reconstructed optic nerve spatial model are written into the heterogeneous data storage structure, and the spatial confidence evaluation index is established to add a dynamic weight coefficient to the nodes after displacement compensation. This process improves the spatial consistency of multi-source data, for example, maintains the real-time matching between the retinal nerve fiber layer and the evoked potential during retinal peeling, and ensures that the anatomical basis for functional damage assessment is always accurate and reliable.

[0104] In one embodiment, the training method of the convolutional recurrent neural network comprises:

[0105] A training data set is constructed, which contains preoperative three-dimensional OCT image features, intraoperative instrument operation parameter sequences and postoperative clinical visual function evaluation results;

[0106] An instrument operation parameter and visual function damage association simulation data is generated by an adversarial generative network, and the association simulation data is used to expand the training data set;

[0107] The long short-term memory module is used to extract the time sequence dependent characteristics of the instrument operation parameters, and the time sequence dependent characteristics and image characteristics are weighted and fused through the attention mechanism.

[0108] The network weight parameters are optimized through the back propagation algorithm, so that the KL divergence between the visual function damage risk coefficient output by the convolution recurrent neural network and the postoperative clinical evaluation result converges to a set range.

[0109] Specifically, preoperative three-dimensional OCT image features are obtained by high-resolution optical coherence tomography (OCT) equipment, which can provide detailed anatomical structure information of intraocular tissues; intraoperative instrument operation parameter sequence records real-time parameters such as position, angle, and force of the instrument during the operation, which directly affect the operation effect. Postoperative clinical visual function evaluation results are obtained through clinical examination, reflecting the actual situation of the patient's postoperative visual recovery. By integrating these multi-dimensional data, the training data set can provide comprehensive information support for network training. Further, the correlation simulation data of instrument operation parameters and visual function damage are generated by the generative adversarial network (GAN), which is a generative adversarial model that can learn the distribution of data and generate new data samples. In this embodiment, GAN is used to generate simulation data with similar distribution to real data, which can supplement the training data set and improve the generalization ability of the network to different situations. Specifically, the generator network in GAN generates simulation data according to the input noise, while the discriminator network tries to distinguish real data and simulation data. Through the adversarial process between the generator and the discriminator, the generator can generate more and more realistic simulation data, which can be used to expand the training data set and increase the diversity and quantity of data. Exemplarily, a long short-term memory module (LSTM) is used to extract the time-dependent features of the instrument operation parameters, and the time-dependent features and image features are weighted and fused through an attention mechanism. The long short-term memory module is a recurrent neural network structure that can effectively handle long-term dependencies in time series data. In this embodiment, LSTM is used to extract the time-dependent features of the intraoperative instrument operation parameter sequence, which reflect the variation law of the instrument operation parameters over time. At the same time, through the attention mechanism, the network can automatically learn the weight relationship between the image features and the time-dependent features, and weightedly fuse the two features, so as to more comprehensively capture the key information in the operation process. For example, the attention mechanism can dynamically adjust the weight of the features according to the importance of different time steps and the importance of different image regions, so that the network can more effectively utilize these features for prediction. Further, the network weight parameters are optimized by the back propagation algorithm to make the KL divergence of the visual function damage risk coefficient output by the convolutional recurrent neural network and the postoperative clinical evaluation results converge within a set range. The back propagation algorithm is a commonly used neural network training algorithm that calculates the gradient of the loss function with respect to the network weights to update the network weights to minimize the loss function. In this embodiment, the loss function uses KL divergence, i.e., Kullback-Leibler divergence, which measures the difference between two probability distributions. By optimizing the network weights, the KL divergence between the probability distribution of the visual function damage risk coefficient output by the network and the probability distribution of the postoperative clinical evaluation results converges within a set range, thereby improving the accuracy and reliability of the network prediction.For example, a threshold of KL divergence can be set as 0.1, when the KL divergence is less than the threshold, it is considered that the network training reaches convergence, and the network can accurately predict the visual function damage risk coefficient.

[0110] In one of the embodiments, the image-based ophthalmic robot control method further comprises:

[0111] In the postoperative verification stage, the visual function damage risk coefficient sequence recorded in the historical operation process and the postoperative visual recovery prediction value of the key operation point are obtained;

[0112] The visual function damage risk coefficient sequence and the postoperative visual recovery prediction value are subjected to Gaussian mixture model clustering analysis to generate a high-risk operation mode feature set;

[0113] Based on the high-risk operation mode feature set and the anatomical structure features of the preoperative three-dimensional OCT image, a prognosis optimization score report is generated through a random forest algorithm, and the prognosis optimization score report includes a surgical path planning suggestion and a risk avoidance strategy.

[0114] Exemplarily, the embodiment establishes a dynamic correlation between high-risk operation modes and anatomical features through multi-dimensional data analysis in the postoperative verification stage, and generates a quantitative evaluation report that can guide improvement. The method first obtains a sequence of visual function damage risk coefficients recorded during the historical surgery process and postoperative visual acuity recovery prediction values at key operation points. Specifically, the sequence of risk coefficients is derived from the convolutional recurrent neural network prediction results every 50 milliseconds during the surgery, covering instrument movement trajectory, pressure gradient, and biological electrical signal mutation events. The postoperative visual acuity recovery prediction values are generated by a coupling model of preoperative baseline visual acuity and real-time risk values during the surgery, for example, the correlation parameters in macular surgery include foveal thickness change value and visual acuity loss. Gaussian mixture model clustering analysis is performed on the above data. Exemplarily, the expectation maximization algorithm can be used to iteratively optimize the model parameters: first, determine the optimal number of clusters according to the Bayesian information criterion, then identify the spatiotemporal distribution characteristics of each cluster through covariance matrix decomposition, and also introduce an information entropy weight adjustment mechanism to give higher clustering weight to operation intervals that exceed the risk threshold for more than 500 milliseconds. The generated high-risk operation mode feature set includes three core dimensions: risk coefficient fluctuation spectrum characteristics, instrument path deviation mode, and predicted value decay curve morphology, for example, identifying the typical high-risk mode of "ultrasonic emulsification stage pressure oscillation accompanied by optic nerve conduction delay". Random forest is an ensemble learning algorithm that can handle a large number of features and provide accurate classification or regression results. In this embodiment, the random forest algorithm combines the high-risk operation mode feature set and the anatomical structure features of the preoperative three-dimensional OCT image to evaluate each surgical case. The prognosis optimization score report includes surgery path planning suggestions and risk avoidance strategies, which aim to optimize the surgery path and reduce the occurrence of high-risk operation modes, thereby improving the safety of the surgery and the possibility of postoperative visual acuity recovery. For example, the report may suggest using more cautious operation parameters near specific anatomical structures, or adjusting the surgery path to avoid high-risk areas.

[0115] In summary, the image-based ophthalmic robot control method provided in the present application constructs a dynamic optic nerve spatial model by fusing three-dimensional OCT images and visual evoked potential signals in real time during the surgery, and realizes millisecond-level precise registration of biological electrical signals and anatomical structures based on eye movement compensation algorithm; on the basis of this fusion data, the pre-trained neural network model is used to calculate the visual function damage risk coefficient and the postoperative visual acuity recovery prediction value in real time combined with the robot operation parameters, and the instrument movement constraint instructions and risk avoidance strategies are dynamically generated according to the prediction results. This technical solution innovatively establishes a real-time prediction correlation between intraoperative operation parameters and postoperative visual acuity recovery effect, actively optimizes the instrument path and operation force during the surgery, not only improves the precision of individualized surgical plan, but also realizes the paradigm shift from traditional anatomical positioning to optic nerve function protection, and fundamentally solves the core technical bottleneck of being unable to dynamically evaluate and protect the nerve function in ophthalmic robot surgery.

[0116] It should be understood that, although the steps in the flowcharts involved in the embodiments described above are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0117] Based on the same inventive concept, the embodiments of the present application also provide an image-based ophthalmic robot control system 10 for implementing the above-mentioned image-based ophthalmic robot control method. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme described in the above method, so the specific limitations in one or more image-based ophthalmic robot control system 10 embodiments provided below can refer to the limitations of the image-based ophthalmic robot control method described above, which will not be repeated here.

[0118] In one exemplary embodiment, as shown in Figure 2 An image-based ophthalmic robot control system 10 is provided, comprising:

[0119] An image segmentation module 11 is configured to acquire an intraoperative three-dimensional OCT image sequence and a visual evoked potential signal, segment a optic nerve region to generate an optic nerve spatial model;

[0120] A data fusion module 12 is configured to perform dynamic registration on the optic nerve spatial model and the visual evoked potential signal according to an eye movement compensation algorithm to generate a fusion data volume;

[0121] A visual function prediction analysis module 13 is configured to calculate a visual function damage risk coefficient and a postoperative visual acuity recovery prediction value based on the fusion data volume and real-time operation parameters of a robot end instrument through a pre-trained neural network prediction model;

[0122] A control instruction generation module 14 is configured to dynamically generate robot operation correction control instructions according to the damage risk coefficient and the postoperative visual acuity recovery prediction value.

[0123] In one embodiment, the data fusion module 12 comprises:

[0124] An optical flow analysis unit is configured to perform optical flow field analysis on the three-dimensional OCT image sequence of adjacent frames to generate a tissue displacement vector field;

[0125] an instruction execution unit configured to construct a dynamic correction matrix according to the tissue displacement vector field;

[0126] a coordinate mapping unit configured to map, by the dynamic correction matrix, a coordinate of a collection point of the visual evoked potential signal to a corresponding node of the optic nerve spatial model;

[0127] a data binding unit configured to bind, to the mapped node, an amplitude parameter and a latency offset parameter of the visual evoked potential signal, and generate a fusion data body.

[0128] In one of the embodiments, the visual function prediction analysis module 13 comprises:

[0129] a functional index calculation unit configured to calculate a dynamic optic nerve functional index based on the amplitude parameter, the latency offset parameter, and a real-time pressure change rate of the robot end instrument;

[0130] a risk prediction unit configured to input the dynamic optic nerve functional index and the real-time pressure into a pre-trained convolutional recurrent neural network, and output a visual function damage risk coefficient;

[0131] a visual function prediction unit configured to calculate a postoperative visual function recovery prediction value based on a functional relationship between the visual function damage risk coefficient and a preoperative baseline visual function.

[0132] In one of the embodiments, the control instruction generation module 14 comprises:

[0133] a stage identification unit configured to generate a current surgery stage identifier based on real-time position coordinates of the robot end instrument in the three-dimensional OCT image sequence, by a preset surgery stage division rule;

[0134] a risk judgment unit configured to judge whether the current surgery stage identifier belongs to a preset high-risk stage set;

[0135] a threshold calculation unit configured to calculate a maximum allowable movement speed and a pressure safety threshold value according to the visual function damage risk coefficient if the current surgery stage identifier belongs to the preset high-risk stage set;

[0136] an instruction execution unit configured to generate a motion pause instruction and trigger an alarm signal if the postoperative visual function recovery prediction value is less than a preset recovery threshold value.

[0137] In one of the embodiments, the instruction execution unit can be further configured to perform the following steps:

[0138] affine transformation modeling is performed on the tissue displacement vector field to generate a spatial compensation parameter matrix, the spatial compensation parameter matrix comprising translation, rotation, and scaling parameters;

[0139] The original coordinates of the visual evoked potential signal collection point are subjected to spatial transformation operation by using a spatial compensation parameter matrix to generate corrected spatial coordinates;

[0140] According to the Euclidean distance between the corrected spatial coordinates and the nodes in the optic nerve spatial model, an effective mapping node set is screened;

[0141] The spatial coordinate distribution of the fusion data body is updated based on the effective mapping node set.

[0142] In one of the embodiments, the training method of the convolutional recurrent neural network in the risk prediction unit comprises:

[0143] A training data set is constructed, which contains preoperative three-dimensional OCT image features, intraoperative instrument operation parameter sequences, and postoperative clinical visual function evaluation results;

[0144] The association simulation data of instrument operation parameters and visual function damage are generated by the generative adversarial network, which are used to expand the training data set;

[0145] The time sequence dependent features of the instrument operation parameters are extracted by using the long short-term memory module, and the time sequence dependent features and the image features are weighted and fused by using the attention mechanism;

[0146] The network weight parameters are optimized by the back propagation algorithm, so that the KL divergence between the visual function damage risk coefficient output by the convolutional recurrent neural network and the postoperative clinical evaluation result converges to a set range.

[0147] In one of the embodiments, the image-based ophthalmic robot control system 10 further comprises a postoperative optimization unit for performing the following steps:

[0148] In the postoperative verification stage, the visual function damage risk coefficient sequence recorded in the historical operation process and the postoperative visual recovery prediction value of the key operation point are obtained;

[0149] The visual function damage risk coefficient sequence and the postoperative visual recovery prediction value are subjected to Gaussian mixture model clustering analysis to generate a high-risk operation mode feature set;

[0150] Based on the high-risk operation mode feature set and the anatomical structure features of the preoperative three-dimensional OCT image, a prognosis optimization score report is generated by using the random forest algorithm, which includes operation path planning suggestions and risk avoidance strategies.

[0151] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of an image-based ophthalmic robot control method as previously described when executing the computer program.

[0152] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program, which when executed by a processor implements the steps of any of the above method embodiments.

[0153] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts are described in the part of the method embodiments. The above described device embodiments are only illustrative, wherein the components described as separate components can or can not be physically separated, the components displayed as units can or can not be physical units, i.e. can be located in one place or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the present disclosure according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0154] The above described embodiments only express several implementation manners of the present application, the description is more specific and detailed, but it should not be understood as a limitation to the patent scope of the application. It should be pointed out that, for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. An image-based ophthalmic robot control system, characterized in that, The system includes: The image segmentation module is used to acquire intraoperative 3D OCT image sequences and visual evoked potential signals, and to segment the optic nerve region to generate a spatial model of the optic nerve. The data fusion module is used to dynamically register the optic nerve spatial model and the visual evoked potential signal according to the eye movement compensation algorithm to generate a fused data volume. The visual function prediction and analysis module is used to calculate the visual function impairment risk coefficient and postoperative visual recovery prediction value based on the real-time operating parameters of the fused data body and the robot end effector through a pre-trained neural network prediction model. The control command generation module is used to dynamically generate robot operation correction control commands based on the injury risk coefficient and the postoperative visual recovery prediction value. The data fusion module includes: The optical flow analysis unit is used to perform optical flow field analysis on adjacent frames of three-dimensional OCT image sequences to generate tissue displacement vector fields; The instruction execution unit is used to construct a dynamic correction matrix based on the tissue displacement vector field; The coordinate mapping unit is used to map the coordinates of the acquisition point of the visual evoked potential signal to the corresponding node of the optic nerve spatial model through the dynamic correction matrix. The data binding unit is used to bind the amplitude parameter and latency offset parameter of the visual evoked potential signal to the mapped node to generate the fused data body; The visual function prediction and analysis module includes: The functional index calculation unit is used to calculate the dynamic visual nerve functional index based on the amplitude parameter, the latency offset parameter, and the real-time pressure change rate of the robot end effector. The risk prediction unit is used to input the dynamic visual nerve function index and real-time stress into a pre-trained convolutional recurrent neural network and output the visual function impairment risk coefficient. The vision prediction unit is used to calculate the predicted value of postoperative vision recovery based on the functional relationship between the visual function impairment risk coefficient and the preoperative baseline vision.

2. The system according to claim 1, characterized in that, The control command generation module includes: The stage identification unit is used to generate a current surgical stage identifier based on the real-time position coordinates of the robot end effector in the three-dimensional OCT image sequence and a preset surgical stage division rule. The risk assessment unit is used to determine whether the current surgical stage identifier belongs to a preset high-risk stage set. The threshold calculation unit is used to calculate the maximum allowable movement speed and pressure safety threshold based on the visual function impairment risk coefficient if the current surgical stage identifier belongs to the preset high-risk stage set. The instruction execution unit is used to generate a motion pause instruction and trigger an alarm signal if the postoperative visual recovery prediction value is less than a preset recovery threshold.

3. The system according to claim 1, characterized in that, The instruction execution unit is further configured to perform the following steps, including: Affine transformation modeling is performed on the tissue displacement vector field to generate a spatial compensation parameter matrix, which includes translation, rotation and scaling parameters. The original coordinates of the visual evoked potential signal acquisition points are spatially transformed using the spatial compensation parameter matrix to generate corrected spatial coordinates. Based on the Euclidean distance between the corrected spatial coordinates and the nodes in the optic nerve spatial model, a set of valid mapping nodes is selected; The spatial coordinate distribution of the fused data volume is updated based on the set of effective mapping nodes.

4. The system according to claim 1, characterized in that, The training method for the convolutional recurrent neural network in the risk prediction unit includes: A training dataset is constructed, which includes preoperative 3D OCT image features, intraoperative instrument operation parameter sequences, and postoperative clinical visual function assessment results. The correlation simulation data between instrument operating parameters and visual function impairment is generated by an adversarial generative network, and the correlation simulation data is used to expand the training dataset. The temporal dependency features of the instrument operation parameters are extracted using a long short-term memory module, and the temporal dependency features are weighted and fused with image features through an attention mechanism. The network weight parameters are optimized by backpropagation algorithm so that the visual function impairment risk coefficient output by the convolutional recurrent neural network and the KL divergence of the postoperative clinical assessment results converge to a set range.

5. The system according to claim 4, characterized in that, The system also includes a postoperative optimization unit for performing the following steps: During the postoperative validation phase, obtain the sequence of visual function impairment risk coefficients recorded during historical surgeries and the postoperative visual recovery prediction values ​​of key operation points; Gaussian mixture model cluster analysis was performed on the visual function impairment risk coefficient sequence and the postoperative visual recovery prediction value to generate a high-risk operation mode feature set; Based on the high-risk operation mode feature set and the anatomical structure features of the preoperative three-dimensional OCT images, a prognostic optimization scoring report is generated using the random forest algorithm. The prognostic optimization scoring report includes surgical path planning suggestions and risk avoidance strategies.

6. An image-based ophthalmic robot control system as described in any one of claims 1-5, characterized in that, The control method of the system includes: Acquire intraoperative 3D OCT image sequences and visual evoked potential signals, segment the optic nerve region to generate a spatial model of the optic nerve; The optic nerve spatial model and visual evoked potential signals are dynamically registered according to the eye movement compensation algorithm to generate a fused data volume. Based on the real-time operating parameters of the fused data body and the robotic end effector, the visual function impairment risk coefficient and postoperative visual recovery prediction value are calculated through a pre-trained neural network prediction model. Based on the injury risk coefficient and the postoperative visual recovery prediction value, the robot operation correction control instructions are dynamically generated.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of claim 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of claim 6.

Images