Active intervention predictive control method for ophthalmic surgical robot and related device
By using FBG sensors and RNN neural networks to predict scleral force values on ophthalmic surgical robots and implementing safety control strategies, the lack of safety monitoring in the prior art is solved, and the safety and accuracy of ophthalmic surgical robots work is improved.
Patent Information
- Application Number
- CN202410073853.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-18
- Publication Date
- 2024-12-31
AI Technical Summary
Existing ophthalmic surgical robots lack effective safety monitoring and prediction mechanisms, and cannot accurately control the movement trajectory of end tools, resulting in unpredictable risks.
Using the terminal tool loaded with FBG sensors, by obtaining force feedback values and depth data, inputting them to the RNN neural network for prediction, to determine whether the scleral force value exceeds the safety threshold in the future. If it is dangerous, implement safety control strategies to update joint speed to ensure movement speed and trajectory. security.
Accurate and safe control of the movement speed and trajectory of the end tools of the ophthalmic surgical robot is achieved, which improves work safety and reduces the risk of unsafe operation.
Smart Images
Figure CN119214861A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical robots, and in particular to an active intervention prediction control method for an ophthalmic surgical robot and related equipment. Background Art
[0002] With the development of medical robot technology, various types of ophthalmic surgical robot systems are now available to assist doctors in completing difficult retinal microsurgery. However, existing ophthalmic surgical robots lack effective safety monitoring, especially a certain prediction mechanism, and are unable to predict the force applied by the end tool to the target object, so as to accurately control their own motion trajectory to keep it within a safe range of motion, which is prone to unpredictable risks. Summary of the invention
[0003] The purpose of the present invention is to provide an active intervention predictive control method and related equipment for an ophthalmic surgical robot. The method and related equipment can predict the output force of the end tool, thereby controlling the movement speed and trajectory of the end tool, thereby improving the safety of the ophthalmic surgical robot.
[0004] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: an active intervention predictive control method for an ophthalmic surgical robot, for an ophthalmic surgical robot equipped with an end tool having an FBG sensor, active intervention predictive control is performed according to the following steps after the robot is started:
[0005] 1) Obtain the force feedback values Fx and Fy of the end tool and the depth value depth of the end tool entering the target object, where Fx and Fy are the components of force along the x-axis and y-axis tangential direction of the end tool axis respectively; obtain the current speed u of each joint of the robot;
[0006] 2) inputting the data obtained in step 1) into the RNN neural network for learning, and outputting the future scleral force value predicted by the network;
[0007] 3) Determine whether the future scleral force value predicted by the network exceeds the safety threshold. If so, it is judged as dangerous and the safety control strategy is executed to force the update of the speed u' of each joint of the robot in the next step. Otherwise, it is judged as safe and the current speed of each joint of the robot is used as the speed of each joint of the robot in the next step.
[0008] 4) Control the next action of the robot according to the speed of each joint of the robot obtained in step 3);
[0009] 5) During the entire working cycle of the robot, steps 1) to 4) are executed repeatedly until the robot stops working.
[0010] Furthermore, the current speed u of each joint of the robot is obtained through an encoder or speed sensor on the motor of each joint of the robot.
[0011] Furthermore, the force feedback values Fx and Fy of the end tool and the depth value depth of the end tool entering the target object are obtained as follows:
[0012] The force feedback values Fx and Fy are used as the elements of the scleral force value Fs, that is, Fs = [Fx, Fy], and Fs is calculated by formula (1):
[0013]
[0014] The FBG sensor is a surface sensing structure, which is distributed and covered on the end tool; therefore, Mo = [Mox, Moy] is the force feedback value matrix at the end of the end tool obtained by the FBG sensor, where Mox and Moy are the force feedback values of the FBG sensor at the end of the end tool along the x-axis and y-axis, respectively, Mp = [Mpx, Mpy] is the force feedback value matrix at the top of the end tool obtained by the FBG sensor, where Mpx and Mpy are the force feedback values of the FBG sensor at the top of the end tool along the x-axis and y-axis, respectively, and Δl is the overall length of the FBG sensor distributed on the end tool;
[0015] The depth value of the end tool entering the target object is calculated by formula (2):
[0016]
[0017] Among them, |||| represents the 2-norm of the matrix; Mmax is the maximum value of all force feedback values obtained by the FBG sensor at different positions.
[0018] Furthermore, the implementation method of the RNN neural network is as follows:
[0019] The RNN neural network is based on the LSTM neural network and consists of an input layer, an LSTM network, an FC layer and an output layer; the input layer reads the force feedback values Fx, Fy, the depth value depth and the speed u of each joint of the robot, and synthesizes them into an n×1 matrix input to the LSTM network, where n is the number of elements of the input data;
[0020] The output of the LSTM network is connected to the FC layer with a Softmax activation function; the Softmax activation function normalizes the output of the LSTM network according to formula (3):
[0021]
[0022] Among them, x jAny output element value of LSTM, y i is the final output after the Softmax activation function is processed, that is, the predicted future scleral force value, and m is the total number of output elements, that is, the number of predicted force value states.
[0023] Furthermore, the implementation method of the security control strategy is:
[0024] When the future scleral force value predicted by the judgment network is in a dangerous state, the safety control strategy is executed and the speed u' of each joint of the robot in the next step is updated according to formula (4):
[0025] u′=u-WV T (4)
[0026] Where u′ is the updated command motion speed of each joint of the robot, u is the original command motion speed of each joint of the robot, W is the k×k unit matrix, k is the number of controlled joints of the robot; V is a 1×k compensation vector, where any element V i for:
[0027]
[0028] Among them, u i is the original command motion speed of the i-th joint; ||u|| is the 2-norm of the original command motion speed u vector of the robot joint; C is an adjustable constant.
[0029] The present invention also provides an active intervention predictive control system for an ophthalmic surgical robot, comprising a memory, a processor, and executable instructions stored in the memory and capable of being executed by the processor. When the processor executes the executable instructions, the above-mentioned active intervention predictive control method is implemented.
[0030] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned active intervention predictive control method is implemented.
[0031] Compared with the prior art, the present invention has the following beneficial effects: the present invention obtains the scleral force value, tool entry depth and key movement speed, and uses the RNN neural network to predict the possible force applied by the robot's end tool to the target object in the future, and then adjusts the robot's next movement speed through the corresponding control strategy, thereby achieving accurate and safe control of the movement speed and trajectory of the robot's end tool, effectively improving the safety of the ophthalmic surgical robot and reducing the risk of unsafe operations in robot-assisted surgery. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1is a flow chart of an implementation method of active intervention prediction control according to an embodiment of the present invention;
[0033] Figure 2 It is an architecture diagram of the RNN neural network in an embodiment of the present invention. DETAILED DESCRIPTION
[0034] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0035] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.
[0036] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0037] like Figure 1 As shown, this embodiment provides an active intervention predictive control method for an ophthalmic surgical robot. For an ophthalmic surgical robot equipped with an end tool having an FBG (Fiber Bragg Gratings) force feedback sensor, active intervention predictive control is performed according to the following steps after the robot is started:
[0038] 1) Obtain the force feedback values Fx and Fy of the end tool and the depth value depth of the end tool entering the target object, where Fx and Fy are the components of force in the x-axis and y-axis directions tangential to the axis of the end tool respectively; obtain the current speed u of each joint of the robot.
[0039] 2) The data obtained in step 1) is input into the RNN neural network for learning, and the future scleral force value predicted by the network is output.
[0040] 3) Determine whether the future scleral force value predicted by the network exceeds the safety threshold. If so, it is judged as dangerous and the safety control strategy is executed to force the update of the speed u' of each joint of the robot in the next step. Otherwise, it is judged as safe and the current speed of each joint of the robot is used as the speed of each joint of the robot in the next step.
[0041] 4) The speed of each joint of the robot in the next step obtained in step 3) controls the next action of the robot.
[0042] 5) During the entire working cycle of the robot, steps 1) to 4) are executed repeatedly until the robot stops working.
[0043] In this embodiment, the current speed u of each joint of the robot can be obtained through an encoder or speed sensor on the motor of each joint of the robot.
[0044] The method for obtaining the force feedback values Fx and Fy of the end tool and the depth value depth of the end tool entering the target object is as follows:
[0045] The force feedback values Fx and Fy are used as the elements of the scleral force value Fs, that is, Fs = [Fx, Fy], and Fs is calculated by formula (1):
[0046]
[0047] The FBG sensor is a surface sensing structure, which is distributed and covered on the end tool, so that the force feedback values at the end and top of the end tool can be detected. Therefore, Mo = [Mox, Moy] is the force feedback value matrix at the end of the end tool obtained by the FBG sensor, where Mox and Moy are the force feedback values of the FBG sensor at the end of the end tool along the x-axis and y-axis, respectively, Mp = [Mpx, Mpy] is the force feedback value matrix at the top of the end tool obtained by the FBG sensor, where Mpx and Mpy are the force feedback values of the FBG sensor at the top of the end tool along the x-axis and y-axis, respectively, and Δl is the overall length of the FBG sensor distributed on the end tool.
[0048] The depth value of the end tool entering the target object is calculated by formula (2):
[0049]
[0050] Among them, |||| represents taking the 2-norm of the matrix; Mmax is the maximum value of all force feedback values obtained by the current FBG sensor at different positions.
[0051] The architecture of the RNN neural network is as follows Figure 2 As shown, the implementation method is as follows:
[0052] The RNN neural network is based on the LSTM (Long Short-Term Memory) neural network, and consists of an input layer, an LSTM network, an FC layer, and an output layer. The input layer reads the force feedback values Fx, Fy, the depth value depth, and the speed u of each joint of the robot, and synthesizes them into an n×1 matrix input to the LSTM network, where n is the number of elements of the input data.
[0053] To make predictions, the output of the LSTM network is connected to an FC layer with a Softmax activation function. The Softmax activation function normalizes the output of the LSTM network according to formula (3):
[0054]
[0055] Among them, x j Any output element value of LSTM, y i is the final output after the Softmax activation function is processed, that is, the predicted future scleral force value, and m is the total number of output elements, that is, the number of predicted force value states, which is 2 in this embodiment, that is, F x (t+n) and F y (t+n) (represents the predicted contact force between the end tool and the target object along the x-axis and y-axis directions at the next moment); thus, in the final output layer, the final judgment on whether the predicted scleral force state is safe can be obtained. For any force value at the next t+n moment, the present invention takes the safety threshold F gate If the safety threshold is exceeded in any direction, it will be considered dangerous.
[0056] In this embodiment, the implementation method of the security control strategy is:
[0057] When the future scleral force value predicted by the judgment network is in a dangerous state, the safety control strategy is executed and the speed u' of each joint of the robot in the next step is updated according to formula (4):
[0058] u′=u-WV T (4)
[0059] Where u′ is the updated command motion speed of each joint of the robot, u is the original command motion speed of each joint of the robot, W is the k×k unit matrix, k is the number of controlled joints of the robot; V is a 1×k compensation vector, where any element V i for:
[0060]
[0061] Among them, u i is the original command motion speed of the i-th joint; ||u|| is the 2-norm of the original command motion speed u vector of the robot joint; C is an adjustable constant. In this embodiment, C is taken as 6mm / s according to the actual robot operation speed.
[0062] Therefore, using formula (4), when the predicted scleral force enters a dangerous state, the robot can be decelerated, stopped or even reversed in advance according to the predicted force value. When the predicted force returns to a safe state, the safety control strategy is not executed and the robot action is directly controlled according to the original control mode.
[0063] This embodiment also provides an active intervention predictive control system for an ophthalmic surgical robot, including a memory, a processor, and executable instructions stored in the memory and capable of being executed by the processor. When the processor executes the executable instructions, the above-mentioned active intervention predictive control method is implemented.
[0064] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned active intervention prediction control method is implemented.
[0065] The active intervention prediction control method and related equipment for an ophthalmic surgical robot provided by the present invention achieve accurate monitoring and control of the scleral force and insertion depth during the robot's operation by applying FBG sensors, robot motion mode switching, and prediction of future scleral force, thereby improving the safety and accuracy of the ophthalmic surgical robot's operation.
[0066] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0067] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0068] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0070] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.
Claims
1. An active intervention predictive control method for an ophthalmic surgical robot, characterized in that: For an ophthalmic surgical robot equipped with an end tool having an FBG sensor, active intervention predictive control is performed according to the following steps after the robot is started: 1) Obtain the force feedback values Fx and Fy of the end tool and the depth value depth of the end tool entering the target object, where Fx and Fy are the components of force along the x-axis and y-axis tangential direction of the end tool axis respectively; obtain the current speed u of each joint of the robot; 2) inputting the data obtained in step 1) into the RNN neural network for learning, and outputting the future scleral force value predicted by the network; 3) Determine whether the future scleral force value predicted by the network exceeds the safety threshold. If so, it is judged as dangerous and the safety control strategy is executed to force the update of the speed u' of each joint of the robot in the next step. Otherwise, it is judged as safe and the current speed of each joint of the robot is used as the speed of each joint of the robot in the next step. 4) Control the next action of the robot according to the speed of each joint of the robot obtained in step 3); 5) During the entire working cycle of the robot, steps 1) to 4) are executed repeatedly until the robot stops working.
2. The active intervention prediction control method for an ophthalmic surgical robot according to claim 1, characterized in that: The current speed u of each joint of the robot is obtained through the encoder or speed sensor on the motor of each joint of the robot.
3. The active intervention prediction control method for an ophthalmic surgical robot according to claim 1, characterized in that: The method for obtaining the force feedback values Fx and Fy of the end tool and the depth value depth of the end tool entering the target object is as follows: The force feedback values Fx and Fy are used as the elements of the scleral force value Fs, that is, Fs = [Fx, Fy], and Fs is calculated by formula (1): The FBG sensor is a surface sensing structure, which is distributed and covered on the end tool; therefore, Mo = [Mox, Moy] is the force feedback value matrix at the end of the end tool obtained by the FBG sensor, where Mox and Moy are the force feedback values of the FBG sensor at the end of the end tool along the x-axis and y-axis, respectively, Mp = [Mpx, Mpy] is the force feedback value matrix at the top of the end tool obtained by the FBG sensor, where Mpx and Mpy are the force feedback values of the FBG sensor at the top of the end tool along the x-axis and y-axis, respectively, and Δl is the overall length of the FBG sensor distributed on the end tool; The depth value of the end tool entering the target object is calculated by formula (2): Wherein, || || represents the 2-norm of the matrix; Mmax is the maximum value of all force feedback values obtained by the FBG sensor at different positions.
4. The active intervention prediction control method for an ophthalmic surgical robot according to claim 1, characterized in that: The implementation method of the RNN neural network is as follows: The RNN neural network is based on the LSTM neural network and consists of an input layer, an LSTM network, an FC layer and an output layer; the input layer reads the force feedback values Fx, Fy, the depth value depth and the speed u of each joint of the robot, and synthesizes them into an n×1 matrix input to the LSTM network, where n is the number of elements of the input data; The output of the LSTM network is connected to the FC layer with a Softmax activation function; the Softmax activation function normalizes the output of the LSTM network according to formula (3): Among them, x j Any output element value of LSTM, y i is the final output after the Softmax activation function is processed, that is, the predicted future scleral force value, and m is the total number of output elements, that is, the number of predicted force value states.
5. The active intervention prediction control method for an ophthalmic surgical robot according to claim 1, characterized in that: The implementation method of the security control strategy is: When the future scleral force value predicted by the judgment network is in a dangerous state, the safety control strategy is executed and the speed u' of each joint of the robot in the next step is updated according to formula (4): u′=u-WV T (4) Where u′ is the updated command motion speed of each joint of the robot, u is the original command motion speed of each joint of the robot, W is the k×k unit matrix, k is the number of controlled joints of the robot; V is a 1×k compensation vector, where any element V i for: Among them, u i is the original command motion speed of the i-th joint; ||u|| is the 2-norm of the original command motion speed u vector of the robot joint; C is an adjustable constant.
6. An active intervention prediction control system for an ophthalmic surgical robot, characterized in that: It includes a memory, a processor, and executable instructions stored in the memory and capable of being executed by the processor. When the processor executes the executable instructions, the active intervention prediction control method as described in any one of claims 1 to 5 is implemented.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the active intervention prediction control method according to any one of claims 1 to 5 is implemented.