Data driving mapping method and device for accurate digital control of operating microscope

Abstract

The invention relates to the technical field of intelligent control of operating microscopes, and particularly provides a data-driven mapping method and device for precise digital control of an operating microscope, and the method comprises the steps: constructing a control model based on a multilayer neural network for any degree of freedom of the operating microscope; the input of the control model is the signal gating duration of the degree of freedom or an input vector containing the current state information and the signal gating duration, and the output of the control model is a corresponding physical output variable quantity; generating a training set by using the collected training data, and training a control model based on the training set to obtain a trained control model; and obtaining a target physical output variable quantity, and inputting the target physical output variable quantity into the trained control model to deduce a corresponding target gating duration. Accurate control can be achieved, and the internal structure of the microscope does not need to be changed.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of surgical microscopes, and in particular to a data-driven mapping method and device for precise digital control of surgical microscopes. Background Art

[0002] The precise control of surgical microscopes plays an important role in modern medical surgery. The current mainstream control schemes include traditional open-loop control, modified closed-loop control, external navigation control and high-end closed-loop control.

[0003] The traditional open-loop control solution mainly uses a foot pedal system to control the XY translation, Z lifting, and zooming of the microscope through coarse interval signals. Due to the lack of encoder feedback, this solution cannot achieve precise positioning, the motion response is uncertain, it is difficult to integrate with the navigation system, and it brings a large operational burden to doctors. The modified closed-loop control solution achieves closed-loop control by adding an encoder, but this solution requires disassembling the microscope, which affects the integrity of the equipment. The modification is not only costly and requires professional operation, but also may cause the original factory warranty to expire. In addition, since different models of microscopes require different modification solutions, the universality is poor.

[0004] The external navigation control solution integrates the microscope with the navigation system, but this solution requires a dedicated navigation system, which makes it costly. The complexity of the system makes the threshold for clinical use high. At the same time, the calibration process is cumbersome, and the control accuracy is still limited by the performance of the open-loop system.

[0005] High-end closed-loop control solutions use servo motor systems with encoders, but this solution is limited to a few high-end models, is costly, cannot be upgraded for existing equipment, makes it difficult for hospitals to fully update equipment, and has high maintenance requirements.

[0006] Therefore, a solution is needed that can achieve precise control without modifying the internal structure of the microscope. Summary of the invention

[0007] The present invention provides a data-driven mapping method and device for precise digital control of a surgical microscope, which is used to solve the defect of low control precision of the surgical microscope in the prior art and realize precise control of the surgical microscope. The technical solution proposed by the present invention is as follows: In a first aspect, the present invention provides a data-driven mapping method for precise digital control of a surgical microscope, comprising: For any degree of freedom of the surgical microscope, a control model based on a multi-layer neural network is constructed; the input of the control model is the signal gating duration of the degree of freedom or an input vector containing current state information and the signal gating duration, and the output is the corresponding physical output change; Generate a training set using the collected multiple training data, and train the control model based on the training set to obtain a trained control model; The target physical output variation is obtained, and the target physical output variation is input into the trained control model to infer the corresponding target gating duration.

[0008] Optionally, training the control model based on the training set to obtain a trained control model includes: Input each training data into the control model to obtain the corresponding predicted value of the physical output change; Determine the mean square error loss based on the predicted value of the physical output change amount and the actual value of the physical output change amount of each training data; Determine the monotonicity loss based on the difference between the predicted values ​​of the physical output change of adjacent training data in the training set; Determine the smoothness loss based on the predicted values ​​of the physical output variation of a plurality of adjacent training data in the training set; A composite loss is determined according to the mean square error loss, the monotonicity loss and the smoothness loss, and the control model is trained using the composite loss to obtain a trained control model.

[0009] Optionally, before collecting the training data, the method further includes: When the relative position between the optical positioning system and the surgical microscope changes, the surgical microscope is controlled to reciprocate along each degree of freedom, and a trajectory point cloud of a tracking marker pre-installed on the surgical microscope is obtained; Processing the trajectory point cloud using principal component analysis to extract the main motion direction of the surgical microscope and establish a basis vector of the mechanical coordinate system of the surgical microscope; The basis vectors of the optical positioning system coordinate system are obtained, and the rigid body transformation matrix between the mechanical coordinate system of the surgical microscope and the optical positioning system coordinate system is estimated by the least squares method according to the basis vectors of the optical positioning system coordinate system and the basis vectors of the mechanical coordinate system of the surgical microscope.

[0010] Optionally, before collecting the training data, the method further includes: For each degree of freedom, gradually increase the signal gating time until the mechanical limit position is reached, and record the corresponding maximum safe operating time to obtain the safe motion range; Before collecting each set of samples, a reverse motion signal is generated according to the maximum safe operating time and the margin time, and the reverse motion signal is applied to the surgical microscope to return the surgical microscope to an initial position.

[0011] Optionally, before collecting the training data, the method further includes: Setting three-dimensional grid sampling points in the surgical microscope workspace; For each three-dimensional grid sampling point, a signal gating duration range for each motion direction is determined based on the distance from the three-dimensional grid sampling point to each mechanical limit position; the signal gating duration of the training data is within the signal gating duration range, and different signal gating durations are used in different areas away from the mechanical limit position; An optical positioning system is used to record the displacement response under different signal gating time lengths, and the stability of the surgical microscope is evaluated based on the displacement response.

[0012] Optionally, the training data includes a change in microscope magnification; before collecting the microscope magnification, the method further includes: Identify the checkerboard corner points of a checkerboard calibration plate pre-placed in the center of the field of view of the surgical microscope; The area change of the region surrounded by the feature points is calculated according to the identified corner points of the chessboard, and the change in the magnification of the microscope is determined according to the area change.

[0013] In a second aspect, the present invention further provides a data-driven mapping device for precise digital control of a surgical microscope, comprising the following modules: A modeling module is used to construct a control model based on a multi-layer neural network for any degree of freedom of the surgical microscope; the input of the control model is the signal gating duration of the degree of freedom or an input vector containing current state information and the signal gating duration, and the output is the corresponding physical output change; A training module, used to generate a training set using the collected multiple training data, and train the control model based on the training set to obtain a trained control model; The inference module is used to obtain the target physical output change and input the target physical output change into the trained control model to infer the corresponding target gating duration.

[0014] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the data-driven mapping method for precise digital control of a surgical microscope as described in the first aspect above is implemented.

[0015] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data-driven mapping method for precise digital control of a surgical microscope as described in the first aspect above.

[0016] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the data-driven mapping method for precise digital control of a surgical microscope as described in the first aspect above.

[0017] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows: The data-driven mapping method and device for precise digital control of a surgical microscope provided by the present invention realizes precise control of the surgical microscope by constructing a control model based on a multi-layer neural network. This control model is independent of the internal structure of the microscope, and it only depends on the mapping relationship between the physical output of the microscope (such as displacement, speed, magnification, etc.) and the control signal (such as signal gating duration). Therefore, the present invention can realize precise control of the microscope without making any changes to the internal structure of the microscope. The present invention uses a multi-layer neural network to construct a control model, which has a powerful nonlinear mapping ability and can accurately capture the complex relationship between the control signal and the physical output. By collecting a large amount of training data and using these samples to train the control model, a high-precision control model can be obtained. This model can accurately infer the corresponding target signal gating duration based on the given target physical output change, and can realize precise digital control of the surgical microscope.

[0018] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a flow chart of a data-driven mapping method for precise digital control of a surgical microscope provided by the present invention.

[0022] Figure 2 It is a schematic diagram of the system composition of the surgical microscope provided by the present invention.

[0023] Figure 3The figure is a schematic diagram of the system hardware connection mode of the surgical microscope provided by the present invention.

[0024] Figure 4 It is a schematic diagram of the neural network structure of the control model provided by the present invention.

[0025] Figure 5 It is a data collection flow chart provided by the present invention.

[0026] Figure 6 It is a structural schematic diagram of a data-driven mapping device for precise digital control of a surgical microscope provided by the present invention.

[0027] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] Combine the following Figure 1-Figure 6 The present invention describes a data driven mapping method and apparatus for precise digital control of a surgical microscope.

[0030] The data-driven mapping method for precise digital control of a surgical microscope provided by the present invention replaces the traditional foot pedal with an external digital control module, and precise control can be achieved without changing the internal structure of the microscope. The present invention regards the internal motor system of the microscope as a black box, and establishes a precise mapping relationship between the signal gating duration and the physical output change through offline calibration. For any degree of freedom (XY translation, Z lifting, zooming resulting in image enlargement and reduction), the input variable is the signal gating duration, and the output variable is the corresponding physical output change (displacement δl or multiple change δm). This solution enables any model of microscope with an I / O foot control interface to obtain precise control capabilities, and has universal applicability. Reference Figure 1 As shown, the method includes the following: Step S110, for any degree of freedom of the surgical microscope, construct a control model based on a multi-layer neural network; the input of the control model is the signal gating duration of the degree of freedom or an input vector containing current state information and the signal gating duration, and the output is the corresponding physical output change.

[0031] Reference Figure 2As shown, the above surgical microscope is mainly composed of a surgical microscope body, an I / O control module, a position measurement module, an image acquisition module and a data processing module. Among them: I / O control module: Use a control card with multi-channel isolated PNP output, with a time resolution of not less than 1 ms, an output voltage matching the microscope control interface, and a signal bandwidth of not less than 1 kHz.

[0032] Position measurement module: It adopts an optical positioning system with a measurement accuracy better than 0.1 mm RMS, a spatial tracking range of not less than 0.5 m×0.5 m×0.5 m, a sampling frequency of not less than 30 Hz, and can operate in multiple working modes such as infrared and visible light.

[0033] Image acquisition module: Use a high-resolution medical or industrial camera with a pixel count of no less than 2 million and a sampling frequency of no less than 25 fps. It is also equipped with a checkerboard-style calibration plate for magnification measurement. The size and number of grid points of the calibration plate can be adjusted according to the working distance and field of view of the microscope. The image acquisition module and position measurement module are only used in the offline calibration stage of the system to reduce the actual application cost.

[0034] Figure 3 The specific system hardware connection method of the surgical microscope is shown. The following is a specific description of the surgical microscope: Passive markers: are objects that need to be identified and tracked, and are used as the starting point for tracking and positioning. The system also supports active / passive markers, providing the system with a flexible identification method.

[0035] Vision calibration board: This component is used in the initialization phase to determine the exact position and shape of the target object to ensure the measurement and positioning accuracy of the system.

[0036] Tracking markers: These markers are placed on the target object to enable the system to track and continuously monitor its position. They are used to provide real-time data on the object's movement.

[0037] Digital microscope: A high-resolution digital microscope is used to magnify and clearly observe objects. It provides detailed visual information, which is essential for subsequent processing and positioning. The digital microscope includes a microscope camera, a foot control interface, and a motor drive system. For example, the microscope camera is used to capture high-definition images and achieve real-time tracking and positioning of objects. The foot control interface provides a manual control option that allows the operator to input control instructions by foot, increasing the flexibility and ease of operation of the system. The motor drive system includes a motor and a corresponding drive circuit, which receives control instructions from a multi-channel isolated output control card and converts them into mechanical actions, such as moving the microscope, adjusting the focus, etc., to achieve precise digital control.

[0038] Control system: This system is responsible for coordinating the entire workflow. The control system includes a tracking and positioning receiver, a data processing host, and a multi-channel isolated output control card. The tracking and positioning receiver receives data from the tracking and positioning link and passes it to the next step of the data processing host. The data processing host is responsible for parsing the data from the tracking and positioning receiver and generating instructions for controlling the motor drive system. These instructions are then sent to the multi-channel isolated output control card. The multi-channel isolated output control card is a key component that connects the data processing host and the motor drive system. It receives control instructions from the data processing host and transmits these instructions to the motor drive system safely and reliably through multi-channel isolation technology. Multi-channel isolation ensures that the signal is not interfered with during transmission, improving the stability and reliability of the system.

[0039] The data-driven mapping system for precise digital control of surgical microscopes achieves real-time tracking, precise positioning and flexible control of objects under surgical microscopes through the collaborative work of a series of hardware components. This process involves multiple links such as initialization, real-time tracking, command transmission and execution, and manual control, which together constitute the complete control process of the system.

[0040] In the initialization phase, before the system starts working, the visual calibration board is first used to determine the exact position and shape of the target object. Through the calibration process, the system can establish an accurate coordinate system, providing a basis for subsequent positioning and measurement.

[0041] In the real-time tracking phase, tracking markers are placed on the objects that need to be observed and controlled. These markers can be passive or active, depending on the system configuration and requirements. The microscope camera in the digital microscope begins to capture high-definition images of the target object. The tracking and positioning receiving device receives the image data from the microscope camera and processes the data to determine the real-time position of the object. The data processing host receives the position data from the tracking and positioning receiving device. The data processing host interprets the data and generates instructions for controlling the motor drive system according to the preset control algorithm.

[0042] In the control execution phase, the generated control instructions are sent to the multi-channel isolated output control card. The multi-channel isolated output control card uses its internal multi-channel isolation technology to ensure that the instructions are not interfered during the transmission process, thereby improving the stability and reliability of the system. The motor drive system receives instructions from the multi-channel isolated output control card. The motor and drive circuit inside the motor drive system perform corresponding mechanical actions according to the instructions, such as moving the microscope, adjusting the focal length, etc.

[0043] Throughout the control process, the operator can input manual control instructions through the foot control interface. This provides additional flexibility for the operator, allowing them to directly intervene in the system's control process when necessary. The system continuously captures images of the target object and updates its position data in real time. The data processing host continuously generates new control instructions based on the new position data to maintain or adjust the precise position of the microscope. Through the cycle of the above steps, the system achieves precise digital control of objects under the surgical microscope. This is a closed-loop control process because the system continuously adjusts its control instructions based on real-time feedback data. For any degree of freedom of the microscope ( ), its control model can be expressed as a nonlinear mapping relationship , where the input is the signal gating duration of the degree of freedom or the input vector containing the current state information, and the output is the corresponding physical output change (such as the change in displacement or amplification factor). represents the signal gating duration of this degree of freedom, Indicates the corresponding physical output change, Indicates the mapping relationship between the signal gating duration and the physical output change. In particular, when When the above physical output changes Indicates the target displacement in millimeters; when When the above physical output changes It is dimensionless and represents the change in magnification. It should be noted that due to the mechanical characteristics of the microscope motor system, this mapping relationship usually exhibits nonlinearity and hysteresis, which is also the main reason for using neural networks for modeling. They correspond to three degrees of freedom of spatial movement, namely XY Translation and Z Rise and fall. F Represents the microscope zoom (Focus) freedom, and zooming causes the image to be enlarged or reduced.

[0044] In order to fit this nonlinear mapping relationship, the present invention designs an adaptive modeling structure based on a multi-layer neural network. Figure 4 As shown in Figure 1, the network contains 2-4 hidden layers, and the number of neurons in each layer is between 16 and 64. Figure 4 For example, the network includes input layer, hidden layer 1, hidden layer 2, hidden layer 3 and output layer, which can be used Nonlinear activation functions are used to enhance the model's expressiveness. The output layer uses a linear activation function to adapt to the output requirements of different ranges. Considering the acceleration and deceleration characteristics of the microscope during movement, as well as the possible differences in starting at different positions, the network input can be the signal gating time. , which can also be extended to include current state information Vector ,in,( ) represents the current three-dimensional position of the microscope, Characterizes the current magnification of the microscope. The mathematical expressions of each layer of the control model are: Input Layer: or .

[0045] Hidden Layer: .

[0046] Output layer: .

[0047] in, represents the output of the input layer, is the weight matrix, is the bias vector. Indicates The output of the hidden layer, Indicates The output of the hidden layer, represents the activation function, represents the number of hidden layers, represents the weight of the output layer. Represents the last hidden layer of the neural network ( The output of the neural network layer is used as the input of the output layer. According to the forward propagation process of the neural network, the final output layer will and the weights of the output layer and the bias vector Combined, the final output is the above physical output change . Represents the bias vector of the output layer. When state information is used, it is obtained through the position measurement module and the image acquisition module during the offline calibration phase, and is estimated by initializing the reference and accumulating the expected displacement during actual use. This design enables the network to naturally handle the differences in the motion characteristics of the microscope through nonlinear mapping. The above structure can be flexibly adjusted according to the characteristics of the microscope. For complex response characteristics, the network depth can be increased and state information can be introduced, while for simple cases, the basic structure can be used to improve efficiency.

[0048] Step S120: Generate a training set using the collected multiple training data, and train the control model based on the training set to obtain a trained control model; In the training phase, a training set is generated using the collected multiple training data, and a control model is trained based on the training set to obtain a trained control model.

[0049] Step S130, obtaining a target physical output variation, and inputting the target physical output variation into the trained control model to infer a corresponding target gating duration.

[0050] In the inference stage, the target physical output change is obtained and input into the trained control model. The model infers the corresponding target gating duration through nonlinear mapping relationship, thus realizing precise digital control of the surgical microscope.

[0051] The present invention achieves precise control of any degree of freedom of the microscope by fitting the nonlinear mapping relationship of the microscope through a multi-layer neural network, thereby improving the accuracy and stability of surgical operations. The control model can adapt to the mechanical characteristics of the microscope and the acceleration and deceleration characteristics during the movement, as well as the differences that may exist when starting at different positions, thereby improving the robustness and adaptability of the system. The image acquisition module and the position measurement module are only used in the offline calibration stage of the system, thereby reducing the actual application cost. At the same time, the design of the multi-layer neural network enables the system to be flexibly adjusted according to the characteristics of the microscope, thereby improving the flexibility and scalability of the system.

[0052] The present invention realizes precise control of the surgical microscope by constructing a control model based on a multi-layer neural network. This control model is independent of the internal structure of the microscope, and it only depends on the mapping relationship between the physical output of the microscope (such as displacement, speed, magnification, etc.) and the control signal (such as signal gating duration). Therefore, the present invention can realize precise control of the microscope without making any changes to the internal structure of the microscope. The present invention uses a multi-layer neural network to construct a control model, which has a powerful nonlinear mapping ability and can accurately capture the complex relationship between the control signal and the physical output. By collecting a large amount of training data and using these samples to train the control model, a high-precision control model can be obtained. This model can accurately infer the corresponding target signal gating duration based on the given target physical output change, and can realize precise digital control of the surgical microscope. The present invention is applicable to various types of open-loop control microscopes and is not limited by brand and model. The modular design is adopted, and the system is easy to upgrade and expand. Since the control model of the present invention is based on data drive, it can be customized and optimized according to different microscope models, different usage environments and different control requirements. When certain parameters of the microscope change, these changes can be adapted by re-collecting training data and training the control model, thereby maintaining the accuracy of control. The control strategy of the present invention can be easily extended to other auxiliary functions of the microscope, such as light source brightness adjustment, fluorescence mode switching, etc. The practical value of the entire system is improved, and a unified solution is provided for the full-degree-of-freedom precision control of surgical microscopes. For example, the present invention can be integrated with the surgical planning system to achieve automatic motion control of the microscope along a preset trajectory. This method can be extended to other medical devices with similar open-loop control characteristics, such as certain surgical robots, radiotherapy equipment, etc. Simple position feedback sensors (such as photoelectric switches, Hall sensors, etc.) can also be introduced to construct a semi-closed-loop system, and the control strategy of the present invention can be combined to achieve higher-precision control.

[0053] The present invention can achieve precise control only through an external control module and a temporarily used measuring device, without changing the internal structure of the microscope, and the overall modification cost is extremely low compared to high-end closed-loop control microscopes. At the same time, it avoids the risk of equipment damage and warranty invalidation caused by hardware modification, and can switch back to the pedal control mode at any time as needed or use a parallel mechanism to make the two control modes coexist.

[0054] In an optional embodiment, the step S120 described above of training the control model based on the training set to obtain the trained control model includes: S1201. Input each training data into the control model to obtain the corresponding predicted value of the physical output change.

[0055] Each training data is input into the control model. These training data contain various data and information required for model training. The control model calculates the corresponding predicted value of the physical output change based on the input training data. These predicted values ​​are the model's estimate of the actual physical output change.

[0056] S1202, determine the mean square error loss based on the predicted value of the physical output change of each training data and the true value of the physical output change; determine the monotonicity loss based on the difference between the predicted values ​​of the physical output change of adjacent training data in the training set; determine the smoothness loss based on the predicted values ​​of the physical output change of multiple adjacent training data in the training set.

[0057] Calculate the mean square error between the predicted value of the physical output change of each training data and the actual value of the physical output change. The mean square error loss function helps the model reduce large prediction errors during the training process and ensure prediction accuracy.

[0058]

[0059] In the formula, represents the mean square error loss, represents the number of training data, Degrees of freedom The corresponding predicted value of the physical output change, Degrees of freedom The corresponding actual value of the physical output change.

[0060] The monotonicity loss is determined based on the difference in predicted values ​​of the physical output changes of adjacent training data in the training set. The monotonicity loss is used to measure whether the trend of the model's predicted values ​​is consistent with the trend of the actual values. The physical output change should be monotonic, that is, as the input changes, the output should also change according to a certain fixed trend, such as ensuring that the output increases accordingly when the input increases. Monotonicity loss helps the model learn this trend, thereby improving the accuracy of the prediction.

[0061]

[0062] In the formula, represents the monotonicity loss, represents the number of training data, Degrees of freedom The corresponding The predicted value of the physical output change of training data, Degrees of freedom The corresponding The predicted value of the physical output change of training data, Represents the maximum value function.

[0063] The smoothness loss is determined based on the predicted values ​​of the physical output changes of multiple adjacent training data in the training set. The smoothness loss is used to measure the smoothness of the model's predicted values, that is, whether the predicted values ​​change smoothly under different inputs. The smoothness loss helps to avoid excessive fluctuations or mutations in the model, thereby improving the stability and reliability of the prediction.

[0064]

[0065] In the formula, represents the smoothness loss, represents the number of training data, Degrees of freedom The corresponding The predicted value of the physical output change of training data, Degrees of freedom The corresponding The predicted value of the physical output change of training data, Degrees of freedom The corresponding The predicted value of the physical output change of the training data.

[0066] S1203, determining a composite loss according to the mean square error loss, the monotonicity loss and the smoothness loss, and using the composite loss to train the control model to obtain a trained control model.

[0067] The mean square error loss, monotonicity loss and smoothness loss are combined according to certain weights to form a composite loss function. The composite loss function comprehensively considers losses in multiple aspects and can more comprehensively reflect the prediction performance of the model.

[0068]

[0069] In the formula, represents the compound loss, and is the weight coefficient, and the optional range is 0.1-10, which can be adjusted according to specific application requirements. This composite loss function design fully considers the characteristics of microscope control, which can not only ensure the prediction accuracy of the model, but also ensure the stability of the control process.

[0070] The control model is trained using a composite loss function, and the composite loss function is minimized by continuously adjusting the model's parameters. During the training process, the model will gradually learn the inherent laws and characteristics of the data, thereby improving the accuracy of the prediction.

[0071] The present invention introduces a neural network modeling with monotonicity and smoothness constraints and an error correction mechanism based on multi-level statistical compensation, so that the system has good generalization ability and anti-interference ability. The calibration process has a high degree of automation, simple operation and strong reliability. Smoothness loss helps to avoid excessive fluctuations or mutations in the model, which enables the model to maintain stable prediction performance when facing complex and changeable input data. Monotonicity loss helps the model learn the changing trend of the physical output change amount, further enhancing the robustness of the model. By comprehensively considering mean square error loss, monotonicity loss and smoothness loss, the composite loss function can more comprehensively reflect the prediction performance of the model. This helps the model to continuously learn and optimize during the training process, thereby improving the accuracy of the prediction. The composite loss function provides a clear goal and optimization direction for the training of the model. By continuously minimizing the composite loss function, the model can gradually approach the optimal solution, thereby improving the training efficiency and effect.

[0072] In an optional embodiment, before collecting training data for precise digital control of surgical microscope, system initialization and coordinate calibration are crucial steps. Since the optical positioning system and the surgical microscope follow their own independent coordinate systems, there may be a rotation and translation relationship between the two coordinate systems. In order to ensure the accuracy and consistency of data collection, the coordinate systems must be unified. Before collecting training data, the method also includes: S210, when the relative position between the optical positioning system and the surgical microscope changes, control the surgical microscope to reciprocate along each degree of freedom, and obtain the trajectory point cloud of the tracking marker pre-installed on the surgical microscope.

[0073] Tracking markers are pre-installed on the surgical microscope, and these markers will be used by the optical positioning system to track the movement trajectory of the microscope. Through the I / O control module, the surgical microscope is controlled to perform a large range of reciprocating motion along each degree of freedom. In order to ensure the comprehensiveness and accuracy of the data, the range of motion should not be less than 80% of the maximum stroke of the degree of freedom, and the number of reciprocating times should not be less than 5 times. The optical positioning system records the movement trajectory of the tracking markers and forms trajectory point cloud data. This data contains the movement information of the microscope in each degree of freedom.

[0074] S220, using principal component analysis to process the trajectory point cloud, extracting the main movement direction of the surgical microscope, and establishing a basis vector of the mechanical coordinate system of the surgical microscope.

[0075] The principal component analysis method is used to process the trajectory point cloud data and extract the main motion direction of the surgical microscope. Principal component analysis is a commonly used data dimensionality reduction technique that can identify the main change trends in the data. Based on the extracted main motion direction, the basis vector of the mechanical coordinate system of the surgical microscope is established. These basis vectors represent the principal directions of motion of the microscope in each degree of freedom.

[0076] S230, obtaining the basis vectors of the optical positioning system coordinate system, and estimating the rigid body transformation matrix between the mechanical coordinate system of the surgical microscope and the optical positioning system coordinate system by the least squares method according to the basis vectors of the optical positioning system coordinate system and the basis vectors of the mechanical coordinate system of the surgical microscope.

[0077] According to the measurement principle and data of the optical positioning system, the basis vectors of its coordinate system are obtained. These vectors define the measurement direction of the optical positioning system. The rigid body transformation matrix between the mechanical coordinate system of the surgical microscope and the coordinate system of the optical positioning system is estimated by the least squares method. The rigid body transformation matrix contains rotation and translation information. The present invention focuses on the displacement change, so the rotation relationship is mainly corrected. The least squares method is an optimization algorithm used to find parameter estimates that minimize the sum of squared errors. In this scenario, it is used to estimate the optimal rotation matrix between the two coordinate systems. Whenever the relative position of the optical positioning system and the surgical microscope changes (such as equipment movement, collision, etc.), the above coordinate system calibration process needs to be re-executed to ensure the accuracy and consistency of the coordinate system.

[0078] The present invention realizes the unification between the optical positioning system and the mechanical coordinate system of the surgical microscope through coordinate calibration, eliminating the measurement error caused by inconsistent coordinate systems. Coordinate calibration ensures the consistency and accuracy of data acquisition, thereby enhancing the stability and reliability of the entire system. The unified coordinate system simplifies the subsequent data processing process, making the calculation of displacement changes more direct and accurate. Whenever the relative position of the optical positioning system and the microscope changes, the consistency of the coordinate system can be restored by re-executing the calibration process, ensuring the flexibility and adaptability of the system.

[0079] In an optional embodiment, before collecting training samples for training the control model, in order to ensure the accuracy and safety of data collection, the method also includes a series of system initialization and preparation steps. These steps are intended to determine the safe range of motion of each degree of freedom of the surgical microscope and reliably return the microscope to its initial position before each set of data collection.

[0080] Before collecting the training data, the method further includes: S310, gradually increase the signal gating time for each degree of freedom until the mechanical limit position is reached, and record the corresponding maximum safe operating time to obtain a safe motion range.

[0081] Since the open-loop system lacks direct limit feedback signals and cannot sense the position of the mechanical limit in real time, it is necessary to conduct preliminary experiments to determine the safe range of motion for each degree of freedom. , , The signal gating time is gradually increased until the microscope reaches the mechanical limit position. During this process, the maximum safe operating time corresponding to each degree of freedom is recorded. , which represents the maximum time the microscope can move safely without triggering the mechanical limit. For each degree of freedom, a specific maximum safe operating time value is recorded, which will be used in the subsequent data acquisition process to ensure that the microscope moves within a safe range.

[0082] S320. Before collecting each group of samples, generate a reverse motion signal according to the maximum safe operating time and the margin time, and apply the reverse motion signal to the surgical microscope to return the surgical microscope to an initial position.

[0083] Before each data acquisition, the system state is initialized to ensure that the microscope is in a known and stable state. The reverse motion signal is sent to return the microscope to the mechanical limit position. This process ensures that the microscope is in a certain initial state before data acquisition. is the margin time.

[0084] When the microscope returns to the mechanical limit position, the optical tracking coordinates of each degree of freedom at that position are recorded as the zero point. These zero points will be used for the subsequent calculation of relative displacement to ensure the accuracy of displacement measurement. The establishment of the initial reference position is the basis of data acquisition. It provides a stable starting point, making the subsequent collected data comparable and consistent.

[0085] Before collecting each set of data, a reverse motion signal is generated based on the previously recorded maximum safe operating time and a preset margin time (eg, 10%-20% of the maximum safe operating time).

[0086] Applying this reverse motion signal to the surgical microscope allows it to reliably return from its current position to its initial position (i.e., near the mechanical limit position). The margin time is added to ensure that the microscope can safely and reliably return to its initial position to avoid insufficient motion due to system errors or external interference.

[0087] When the microscope returns to its initial position and the system is initialized, formal data collection can begin. At this point, the microscope is in a known and stable state, and the accuracy and reliability of data collection are guaranteed.

[0088] The present invention ensures the accuracy and consistency of data collection by determining the safe range of motion and returning the microscope to the initial position before data collection. During the data collection process, mechanical damage or safety risks caused by movement beyond the safe range are avoided. The establishment of the initial reference position makes the calculation of subsequent displacement simple and direct, simplifying the data processing process. The addition of system initialization and preparation steps makes the entire data collection process more stable and reliable, providing high-quality data support for subsequent control model training.

[0089] In an optional embodiment, before collecting training data for training precise digital control of a surgical microscope, in order to comprehensively and accurately evaluate the motion characteristics and stability of the microscope under different conditions, the method further includes a series of three-dimensional grid sampling and stability evaluation steps. These steps are intended to systematically collect displacement response data of the microscope in the workspace through a dense grid sampling strategy, and evaluate the stability of the microscope based on these data.

[0090] Before collecting the training data, the method further includes: S410, setting three-dimensional grid sampling points in the surgical microscope workspace.

[0091] First, determine the boundaries of the 3D grid sampling according to the working space range of the surgical microscope. Set the 3D grid sampling points in the working space. The grid density can be adjusted according to the required accuracy requirements. For example, for each direction ( X axis, Y axis, Z Axis) to set 200-500 sampling points to ensure that the entire working space is covered and the motion characteristics of the microscope at different positions are captured.

[0092] S420. For each three-dimensional grid sampling point, determine the signal gating duration range for each motion direction based on the distance from the three-dimensional grid sampling point to each mechanical limit position; the signal gating duration of the training data is within the signal gating duration range, and different signal gating durations are used in different areas away from the mechanical limit position.

[0093] For each 3D grid sampling point, calculate its distance to each mechanical limit position. Based on the distance from the sampling point to each mechanical limit, adaptively determine six movement directions (e.g. X+, X-, Y+, Y-, Z+, Z- ) signal gating time range. This range is from 50 ms to 2000 ms to ensure that the movement response of the microscope at different speeds can be captured.

[0094] S430, using an optical positioning system to record displacement responses at different signal gating durations, and evaluating the stability of the surgical microscope based on the displacement responses.

[0095] In the area less than 20% of the range from the mechanical limit, a smaller sampling interval (such as 25-50 ms) is used. This is because the motion characteristics of the microscope in these areas may be more complex and sensitive, and more refined sampling is required to capture the details. In the area 20%-80% of the range from the mechanical limit, a medium sampling interval (such as 50-100 ms) is used. These areas are the main areas of the microscope's motion characteristics and require sufficient sampling density to ensure data accuracy. In the area greater than 80% of the range from the mechanical limit, a larger sampling interval (such as 100-200 ms) is used. These areas are usually relatively stable and do not require overly dense sampling to improve sampling efficiency.

[0096] Use an optical positioning system to record the displacement response under different signal gating durations. Repeat the measurement of each set of parameters (i.e., each sampling point and the corresponding signal gating duration) 3-5 times to reduce random errors and improve the reliability of the data. Based on the recorded displacement response data, evaluate the stability of the surgical microscope under different conditions. This can be achieved by comparing the displacement differences between different measurement times. If the displacement difference is small, it means that the microscope has good stability under this condition; if the displacement difference is large, it may be necessary to further adjust the control parameters or optimize the system structure. The size of the displacement difference can be determined by comparing the displacement difference with the preset displacement difference threshold.

[0097] The present invention systematically covers the entire working space of the surgical microscope through a three-dimensional grid sampling strategy and captures the motion characteristics at different positions. This improves the coverage and accuracy of the data and provides more comprehensive and reliable data support for subsequent control model training. The use of an adaptive sampling interval strategy not only ensures fine sampling in key areas (such as areas near limit positions), but also improves the sampling efficiency in stable areas. This reduces the time and cost of data acquisition while maintaining the accuracy and reliability of the data. By recording the displacement response and evaluating the stability, the unstable factors of the microscope under different conditions can be discovered in a timely manner, and a basis can be provided for subsequent optimization work. This helps to further improve the accuracy and reliability of the surgical microscope and meet the needs of clinical applications.

[0098] In an optional embodiment, when collecting training data for training precise digital control of surgical microscopes, in addition to considering the spatial displacement characteristics of the microscope, it is also necessary to pay attention to the collection of its magnification change. This process is crucial for a comprehensive understanding of the motion characteristics of the microscope and the optimization of the control algorithm. Figure 5As shown in FIG. 1 , the complete process of data acquisition is shown in detail, including the placement of the calibration plate, image preprocessing, corner point detection, calculation of the area of ​​the feature point enclosed area, and determination of the magnification change. First, the system is initialized and the coordinate system calibration is realized through the above steps S210-S230. By placing the calibration plate and performing a series of calibration operations, the accurate calibration of the coordinate system is realized, providing an accurate reference for subsequent positioning and measurement. Pre-measurement is performed through the above steps S310-S320 to determine the safe motion range of each degree of freedom of the surgical microscope, thereby ensuring that the safety limit will not be exceeded during the data acquisition process. Then, in order to systematically collect the displacement response data of the microscope in the workspace, a dense sampling network is set up. This step provides a clear sampling strategy for subsequent data acquisition. Through a dense grid sampling strategy, the displacement response data of the microscope in the workspace is systematically collected, and the stability of the microscope is evaluated based on these data. If the stability of the microscope meets the preset requirements, displacement measurement or zoom measurement is selected according to the needs, and data acquisition is performed according to the selection results. This process ensures the accuracy and reliability of the data and provides a solid foundation for subsequent control model training. If the measurement type is determined to be displacement measurement, sampling is performed in the X / Y / Z directions, that is, in three-dimensional space, the system samples along the X, Y and Z directions to obtain comprehensive displacement data. Position tracking measurement is performed based on the sampled data, and the system continuously monitors the position changes of the microscope or target object to ensure the accuracy and reliability of the data. In the data processing stage, the system detects and identifies possible outliers to ensure the accuracy of the data. If the measurement type is zoom measurement, magnification sampling is performed to obtain more detailed microscope response data. Image processing area calculation is also performed, that is, the collected image data is processed to calculate parameters such as the area of ​​the area surrounded by the feature points. A comprehensive validity check is performed on the collected data to ensure the reliability and integrity of the data.

[0099] The verified and processed data is stored for subsequent control model training or other analysis. If the sampling is completed, the entire data collection process ends. If not completed, return to the sampling stage to continue collecting data. The training data includes the change in microscope magnification; before collecting the microscope magnification, the method also includes: S510, identifying the checkerboard corner points of a checkerboard calibration plate pre-placed at the center of the field of view of the surgical microscope.

[0100] Use a pre-designed checkerboard calibration plate with a specification of 10×10 or larger to ensure a sufficient number of feature points to improve the accuracy of the calculation. The checkerboard calibration plate adopts a black and white format, and the size of the single grid can be appropriately adjusted according to the working distance of the microscope to adapt to different magnification ranges.

[0101] Place the checkerboard calibration plate in the center of the surgical microscope's field of view to ensure that the calibration plate is clearly visible within the microscope's imaging range. Perform preprocessing on the acquired checkerboard image, including denoising and contrast enhancement. Denoising can remove noise points in the image and improve the accuracy of corner detection; contrast enhancement helps to better identify the black and white edges of the checkerboard. Use the Harris corner detection algorithm or a similar corner detection algorithm to identify the corners of the checkerboard. The Harris corner detection algorithm is a corner detection method based on image gradient information, which can accurately detect the corner positions of the checkerboard.

[0102] S520, calculating the area change of the region surrounded by the feature points according to the identified checkerboard corner points, and determining the change in microscope magnification according to the area change.

[0103] Based on the identified checkerboard corner points, calculate the area of ​​the region enclosed by the feature points (i.e., corner points). This can be achieved by connecting adjacent corner points to form a polygon and calculating the area of ​​the polygon. As the magnification of the microscope changes, the area of ​​the region enclosed by the feature points will also change accordingly. By comparing the changes in the area of ​​the region enclosed by the feature points at different time points or under different control signals, the change in the magnification of the microscope can be calculated. This can be achieved by calculating the percentage or relative value of the area change.

[0104] During the acquisition process, the quality of corner detection is monitored in real time. If the corner detection is unstable or has large errors, it may cause errors in area calculation, which in turn affects the accuracy of the magnification change. Therefore, it is necessary to set certain quality evaluation indicators (such as the stability and repeatability of corner detection) and monitor them in real time during the acquisition process to ensure the reliability of the data.

[0105] The present invention can accurately identify and calculate the change in microscope magnification by combining the checkerboard calibration plate and the corner detection algorithm. This helps to fully understand the motion characteristics of the microscope and provide accurate data support for the optimization of the control algorithm. The quality of corner detection is monitored in real time during the acquisition process to ensure the reliability and stability of the data. This reduces the risk of errors in the data acquisition process and improves the accuracy and availability of the data. By collecting data such as the spatial displacement characteristics and magnification change of the microscope, comprehensive data support is provided for the training of the control model. This helps to train a more accurate and reliable control model and improve the accuracy and stability of the surgical microscope.

[0106] In the model training stage, the present invention first pre-processes the collected raw data. Criteria or density-based clustering methods, data normalization uses maximum and minimum value normalization or The standardization method is used, and data enhancement is performed by adding Gaussian noise (standard deviation is 0.01-0.05). The Adam optimizer is used in the training process, the initial learning rate is 0.001-0.01, and the learning rate adjustment strategy such as cosine annealing is adopted. The above specific parameters are only empirical values ​​and can be adaptively adjusted according to the actual training situation. In order to improve the robustness of the model, the Fold cross validation ( The training rounds can be adjusted dynamically according to the actual convergence situation, and an early stopping mechanism can be set to prevent overfitting.

[0107] The actual verification shows that the surgical microscope has good control performance. After the digital transformation, the microscope, which could only be roughly controlled by foot pedals, has The average positioning accuracy of the direction is better than 0.1 mm, the magnification control accuracy is better than 0.05 times, and the response time does not exceed 50 ms. These indicators meet the basic requirements of clinical precision navigation. The main error sources of the system include the control error of the signal gating time (controlled within ±0.1 ms), the inherent error of the position measurement system (less than 0.1 mm RMS), the recognition error of the image processing algorithm (less than 1 pixel), and the fitting error of the neural network model (root mean square error less than 1% full scale).

[0108] In order to compensate for these errors and improve the accuracy of microscope control, the present invention designs a multi-level compensation mechanism based on historical data. First, at the system error compensation level, the system error distribution characteristics of each control quantity are obtained through calibration data analysis. Building an Error Compensation Lookup Table The lookup table converts the physical output change Mapped to the corresponding statistical error :

[0109] in, The number of repeated measurements can be set according to actual needs, such as ; For the The actual displacement obtained by the measurement is is the target displacement, Represents the maximum value function. is the error compensation parameter, which is used to construct the system error compensation in the multi-level compensation mechanism based on historical data. Specifically, it is the system error distribution characteristics of each control quantity obtained through calibration data analysis, for each degree of freedom The error compensation lookup table is established. In actual control, for the physical output change , the system applies times the control amount to offset the system error, thereby improving control accuracy.

[0110] Specifically, first collect the calibration data of the surgical microscope under different working conditions, including the target displacement and actual displacement of each degree of freedom. These data should cover the entire working range of the microscope to ensure the comprehensiveness and accuracy of the analysis. For each degree of freedom, calculate the difference between the target displacement and the actual displacement, that is, the systematic error. The error calculation should be based on a large number of data samples to reduce the impact of random errors. Next, use statistical methods to analyze the distribution characteristics of the systematic error, including the mean, standard deviation, maximum and minimum values ​​of the error. These statistics help to understand the laws and trends of errors and provide a basis for the subsequent establishment of an error compensation lookup table. Then, based on the control system of the surgical microscope, determine the degrees of freedom that need to be compensated, such as X axis, Y axis, Z Axis, etc. The target displacement of each degree of freedom is divided into several intervals, each of which corresponds to a specific error range. The division of intervals should be based on the results of the error distribution characteristic analysis to ensure that the errors in each interval have similar statistical characteristics. In each interval, calculate the statistical quantities such as the mean or median of the system error as the statistical error of the interval. The calculation of statistical errors should be based on a large number of data samples to improve its accuracy and reliability. The target displacement and the corresponding statistical error of each interval are recorded to form an error compensation lookup table. The lookup table should be easy to query and update so that error compensation can be performed in real time during the control process.

[0111] During the control process, when a certain target displacement needs to be executed, the error compensation lookup table is first queried to find the statistical error (i.e., compensation value) corresponding to the target displacement. The queried compensation value is added to the original target displacement to obtain the corrected target displacement. The corrected target displacement will be used to control the movement of the surgical microscope to reduce or eliminate the influence of the system error. As the surgical microscope is used and the working environment changes, the system error may change. Therefore, it is necessary to recalibrate and update the error compensation lookup table regularly to ensure its accuracy and effectiveness.

[0112] The multi-level compensation mechanism based on historical data of the present invention realizes effective compensation for the system error of the surgical microscope by analyzing the calibration data, establishing the error compensation lookup table and applying the lookup table to correct the target displacement during the control process. This mechanism improves the accuracy of microscope control and provides more reliable and accurate support for surgical operations.

[0113] In terms of random error compensation, the present invention establishes the confidence interval of each control quantity by analyzing the repeatability of the calibration data, and adopts a step-by-step approximation strategy for position control with high precision requirements. Specifically, the system first executes 90% of the target control quantity, and then supplements the remaining control quantity according to the residual deviation after the system stabilizes (such as waiting time 50-100 ms). This progressive control strategy can effectively reduce the impact of random errors and significantly improve the positioning accuracy of the system.

[0114] Specifically, first, collect the calibration data of the surgical microscope under different conditions, which should cover various possible working conditions and influencing factors. Perform repeatability analysis on the calibration data, and calculate the mean value, standard deviation and other statistics of each control quantity to evaluate its stability and consistency. According to the repeatability analysis results of the calibration data, use statistical methods to calculate the confidence interval of each control quantity. The calculation of the confidence interval can be based on the normal distribution assumption and is obtained by determining the confidence level and sample size. The confidence interval can reflect the possible value range of the control quantity and provide a basis for subsequent error compensation. Next, execute the preliminary control quantity, that is, in the position control with high precision requirements, the system first executes 90% of the target control quantity. The purpose of this step is to move the microscope to an area close to the target position while reducing the deviation caused by system inertia and random errors. After executing the preliminary control quantity, the system needs to wait for a period of time (such as 50-100 ms) to ensure that the system is stable and reduce the influence of dynamic errors. The choice of waiting time should be determined according to the response characteristics and control requirements of the system. After the system is stable, the remaining control quantity is calculated and supplemented according to the remaining deviation between the current position and the target position. The purpose of this step is to eliminate the remaining deviation through fine-tuning, thereby achieving high-precision position control. After executing the remaining control amount, the system should detect the current position again and compare it with the target position. If there is still a deviation, it can be adjusted according to the feedback results until the required accuracy is achieved.

[0115] The step-by-step approximation strategy of the present invention effectively reduces the influence of random errors on the positioning accuracy of the system by combining preliminary control and fine-tuning. This strategy can ensure that the microscope can still accurately move to the target position in a complex working environment, thereby improving the positioning accuracy and stability of the system. By analyzing the repeatability of calibration data and establishing confidence intervals, the system can better adapt to different working environments and conditions, improving its adaptability and reliability.

[0116] In terms of load compensation, the present invention takes into account the differences in the dynamic characteristics of the microscope under different load conditions. During the calibration phase, light load (only the necessary lens weight) ), medium load (with some accessories, the total weight is about ) and Heavy Duty (with all accessories, total weight approx. ) Mapping data under three typical working conditions to establish a load-related compensation model:

[0117] in, For load compensation, represents the compensation function. Corresponding to light, medium and heavy load states. , the system can select appropriate model parameters through spline interpolation method. The system can select the current load mode through simple configuration operation, or automatically identify the load state through motion characteristics during use, so as to select the most suitable control parameters and effectively compensate for the system error caused by load changes.

[0118] The multi-level compensation mechanism provided by the present invention is completely based on offline calibration data, does not require real-time feedback signals, and is particularly suitable for the characteristics of open-loop control systems; by processing different types of errors in layers, a more accurate compensation effect can be obtained; at the same time, due to the use of lookup tables and simple mathematical models, the system has a small computational load and timely response, which is very suitable for real-time control needs; in addition, the modular design of the mechanism makes it easy to adjust the compensation strategies of each layer according to actual application needs, and has good adaptability and scalability. The data-driven mapping method for precise digital control of surgical microscopes adopted by the present invention enables the surgical microscope system to be more precise and accurate. XYZ The average positioning accuracy of the direction is better than 0.1 mm, the magnification control accuracy is better than 0.05 times, and the response time does not exceed 50 ms.

[0119] According to the characteristics of different degrees of freedom of microscopes, the present invention designs corresponding offline precise measurement solutions. XYZ For the translation in the direction, infrared optical positioning systems or electromagnetic navigation positioning systems are used for displacement measurement. These systems can achieve sub-millimeter measurement accuracy on the one hand, and take into account the convenience of non-invasiveness on the other hand. It is only necessary to bond the marker to the surface of the packaged microscope shell and place it in the nominal detection space. For the degree of freedom of image zooming, an area change tracking algorithm based on ROI recognition of the calibration plate is designed, and the magnification change is indirectly calculated through image processing to ensure the accuracy of magnification measurement. It should be noted that the above measurement system is only used for the offline calibration stage and does not affect the actual online use of the microscope.

[0120] At the hardware level, the present invention uses an I / O control card with multi-channel isolated outputs connected to the host to replace the traditional foot pedal to achieve precise control of the signal gating duration. At the data acquisition level, an adaptive sampling strategy is designed to improve data acquisition efficiency. At the modeling level, a neural network structure with monotonicity and smoothness constraints is developed, and a stable and reliable signal mapping model is established through offline training, and it is ensured that the model can accurately predict the control signal gating duration required to achieve a specific change when it is used online without the measuring device. At the same time, the acceleration and deceleration characteristics of the microscope during movement are naturally processed through the nonlinear mapping characteristics of the neural network, and the current state information is optionally introduced as feedback input in the network input to adapt to the possible differences in motion characteristics of the microscope at different positions and magnifications.

[0121] The prior art uses a machine learning method based on binocular vision to achieve surgical target recognition and tracking, which requires a special visual system. The present invention uses a data-driven method to directly establish a mapping relationship between the control signal and the physical output without adding any hardware, and at the same time achieves sub-millimeter high-precision control through neural network fitting and multi-dimensional error compensation. The prior art also uses visual gesture recognition and wearable sensors as input interfaces respectively. The present invention uses a data-driven control model to directly process the original control signal of the microscope. While keeping the operating habits unchanged, the control accuracy is significantly improved through a systematic error compensation mechanism. The prior art uses a joystick to control the focal length and magnification. The present invention realizes the microscope XY Pan, Z The precise control of all parameters such as lifting, zooming, etc. has greatly expanded the application scope of automatic control and provided all-round technical support for precision surgery. The existing technology is based on a digital microscope system with 3D imaging and 4K display, but it emphasizes the digital connection of each component and does not solve the problem of fine control. The present invention solves the key problem of precise control of microscopes through innovative data-driven methods and error compensation mechanisms, improves the control accuracy to the sub-millimeter level, and provides more reliable technical support for precision surgery.

[0122] In summary, the present invention adopts a data-driven method combined with a multi-dimensional error compensation mechanism to overcome the problems of reliance on specific hardware and inaccurate manual pedal operation in the prior art, and has significant innovation and practical value. Compared with the traditional open-loop control scheme, the position accuracy and magnification control accuracy are improved by an order of magnitude, approaching the performance level of high-end microscopes. The present invention can achieve precise control only through an external control module and a temporarily used measuring device, without changing the internal structure of the microscope, and the overall modification cost is extremely low compared to high-end closed-loop control microscopes. At the same time, it avoids the risk of equipment damage and warranty invalidation caused by hardware modification, and can switch back to the pedal control mode as needed at any time or allow the two control modes to coexist through a parallel mechanism.

[0123] The data driven mapping device for precise digital control of a surgical microscope provided by the present invention is described below. The data driven mapping device for precise digital control of a surgical microscope described below and the data driven mapping method for precise digital control of a surgical microscope described above can be referred to each other.

[0124] The data driven mapping device for precise digital control of a surgical microscope provided by the present invention is referred to Figure 6 As shown, including: Modeling module 610, for constructing a control model based on a multi-layer neural network for any degree of freedom of the surgical microscope; the input of the control model is the signal gating duration of the degree of freedom or an input vector containing current state information and the signal gating duration, and the output is the corresponding physical output change; A training module 620 is used to generate a training set using the collected multiple training data, and train the control model based on the training set to obtain a trained control model; The inference module 630 is used to obtain the target physical output change and input the target physical output change into the trained control model to infer the corresponding target gating duration.

[0125] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communications interface 720 and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the data-driven mapping method for precise digital control of the surgical microscope.

[0126] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0127] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data-driven mapping method for precise digital control of a surgical microscope provided by the above-mentioned methods.

[0128] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the data-driven mapping method for precise digital control of a surgical microscope provided by the above methods.

[0129] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0130] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data-driven mapping method for precise digital control of a surgical microscope, characterized in that: include: For any degree of freedom of the surgical microscope, a control model based on a multi-layer neural network is constructed; the input of the control model is the signal gating duration of the degree of freedom or an input vector containing current state information and the signal gating duration, and the output is the corresponding physical output change; Generate a training set using the collected multiple training data, and train the control model based on the training set to obtain a trained control model; The target physical output variation is obtained, and the target physical output variation is input into the trained control model to infer the corresponding target gating duration.

2. The data-driven mapping method for precise digital control of a surgical microscope according to claim 1, characterized in that: The step of training the control model based on the training set to obtain a trained control model includes: Input each training data into the control model to obtain the corresponding predicted value of the physical output change; Determine the mean square error loss based on the predicted value of the physical output change amount and the actual value of the physical output change amount of each training data; Determine the monotonicity loss based on the difference between the predicted values ​​of the physical output change of adjacent training data in the training set; Determine the smoothness loss based on the predicted values ​​of the physical output variation of a plurality of adjacent training data in the training set; A composite loss is determined according to the mean square error loss, the monotonicity loss and the smoothness loss, and the control model is trained using the composite loss to obtain a trained control model.

3. The data-driven mapping method for precise digital control of a surgical microscope according to claim 1, characterized in that: Before collecting the training data, the method further includes: When the relative position between the optical positioning system and the surgical microscope changes, the surgical microscope is controlled to reciprocate along each degree of freedom, and a trajectory point cloud of a tracking marker pre-installed on the surgical microscope is obtained; The trajectory point cloud is processed by principal component analysis to extract the main motion direction of the surgical microscope and establish the basis vector of the mechanical coordinate system of the surgical microscope; The basis vectors of the optical positioning system coordinate system are obtained, and the rigid body transformation matrix between the mechanical coordinate system of the surgical microscope and the optical positioning system coordinate system is estimated by the least squares method according to the basis vectors of the optical positioning system coordinate system and the basis vectors of the mechanical coordinate system of the surgical microscope.

4. The data-driven mapping method for precise digital control of a surgical microscope according to claim 1, characterized in that: Before collecting the training data, the method further includes: For each degree of freedom, gradually increase the signal gating time until the mechanical limit position is reached, and record the corresponding maximum safe operating time to obtain the safe motion range; Before collecting each set of samples, a reverse motion signal is generated according to the maximum safe operating time and the margin time, and the reverse motion signal is applied to the surgical microscope to return the surgical microscope to an initial position.

5. The data driven mapping method for precise digital control of a surgical microscope according to claim 1, characterized in that: Before collecting the training data, the method further includes: Setting three-dimensional grid sampling points in the surgical microscope workspace; For each three-dimensional grid sampling point, a signal gating duration range for each motion direction is determined based on the distance from the three-dimensional grid sampling point to each mechanical limit position; the signal gating duration of the training data is within the signal gating duration range, and different signal gating durations are used in different areas away from the mechanical limit position; An optical positioning system is used to record the displacement response under different signal gating time lengths, and the stability of the surgical microscope is evaluated based on the displacement response.

6. The data-driven mapping method for precise digital control of a surgical microscope according to claim 1, characterized in that: The training data includes the change in microscope magnification; Before collecting the microscope magnification, the method further includes: Identify the checkerboard corner points of a checkerboard calibration plate pre-placed in the center of the field of view of the surgical microscope; The area change of the region surrounded by the feature points is calculated according to the identified corner points of the chessboard, and the change in the magnification of the microscope is determined according to the area change.

7. A data-driven mapping device for precise digital control of a surgical microscope, characterized in that: include: A modeling module is used to construct a control model based on a multi-layer neural network for any degree of freedom of the surgical microscope; the input of the control model is the signal gating duration of the degree of freedom or an input vector containing current state information and the signal gating duration, and the output is the corresponding physical output change; A training module, used to generate a training set using the collected multiple training data, and train the control model based on the training set to obtain a trained control model; The inference module is used to obtain the target physical output change and input the target physical output change into the trained control model to infer the corresponding target gating duration.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the data-driven mapping method for precise digital control of a surgical microscope as claimed in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data-driven mapping method for precise digital control of a surgical microscope as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data-driven mapping method for precise digital control of a surgical microscope as claimed in any one of claims 1 to 6 is implemented.

Images