Surgical robot system

Abstract

The present invention provides a surgical robot system, comprising: a master end, a slave end and a controller; the master end includes a robotic arm; the controller includes a manipulation enhancement module, and the manipulation enhancement module is configured to perform the following steps: obtaining at least one of manipulation state information, motion state information and interaction prompt information; obtaining a manipulation force based on at least one of the manipulation state information, the motion state information and the interaction prompt information; and feeding back the manipulation force into the driving force of the robotic arm. With such a configuration, through the setting of the manipulation enhancement module, a manipulation force can be obtained according to at least one of the manipulation state information, the motion state information and the interaction prompt information. By feeding back the manipulation force into the driving force of the robotic arm, the operation experience can be effectively optimized, the discomfort caused by the lack of haptic feedback can be helped to be alleviated, and the operator can better control the surgical robot system to perform complex surgical tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and particularly to a surgical robot system. Background Art

[0002] The current teleoperation surgical robot system mainly includes a master control device, a slave robot device, a communication module, and an image module. The operator issues commands by manipulating the robotic arm of the master control device, and these commands are transmitted to the slave robot device through the communication module. After receiving the commands, the slave robot device replicates the operations of the master control device. At the same time, the slave robot device transmits the surgical images to the master control device and the image module through the communication module.

[0003] With the continuous development of technology, the application scope and functions of surgical robot systems are also constantly expanding. Currently, surgical robot systems can not only be applied to a variety of surgical fields, but also perform surgical operations at a long distance. However, with the increase in the control distance, latency has become a prominent and inevitable problem. Latency usually causes the operator to feel out of sync with the actual operation when manipulating the master control device, thus increasing the surgical risk. The existing technology lacks an effective tactile feedback mechanism, resulting in the operator being unable to accurately perceive the state and position of the robotic arm of the slave robot device, especially when performing remote operations. This lack of feedback may make the operator feel uncomfortable during fine operations, thus affecting the surgical quality. Summary of the Invention

[0004] The purpose of the present invention is to provide a surgical robot system to solve the problem that the existing surgical robot system lacks an effective tactile feedback mechanism.

[0005] To solve the above technical problems, the present invention provides a surgical robot system, which includes: a master end, a slave end, and a controller; the master end includes a robotic arm.

[0006] The controller includes a manipulation enhancement module, and the manipulation enhancement module is configured to perform the following steps:

[0007] Obtain at least one of manipulation state information, motion state information, and interaction prompt information;

[0008] Based on at least one of the manipulation state information, the motion state information, and the interaction prompt information, obtain a manipulation force;

[0009] Feed back the manipulation force into the driving force of the robotic arm.

[0010] Optionally, the manipulation state information includes communication state information, and the communication state information includes at least one of the delay, bandwidth, throughput, jitter, bit error rate, and packet loss rate of the communication between the master end and the slave end.

[0011] Optionally, the manipulation state information includes configuration state information, and the configuration state information is obtained based on at least one of the communication state, image frame difference, and motion state.

[0012] Optionally, the configuration state information is stored in the surgical robot system, or the configuration state information is obtained based on the interactive hardware or interactive software.

[0013] Optionally, the motion state information includes at least one of the master end motion information, the slave end motion information, and the motion deviation information between the master end and the slave end; the master end motion information and the slave end motion information respectively include at least one of the position information, speed information, acceleration information, and joint angle information; the motion deviation information includes at least one of the position deviation information, speed deviation information, acceleration deviation information, and joint angle deviation information.

[0014] Optionally, the interactive prompt information includes at least one of voice prompt information, text prompt information, graphic prompt information, image prompt information, and video prompt information.

[0015] Optionally, the steps of calculating the manipulation force based on the manipulation state information include:

[0016] Obtaining a delay parameter associated with the delay between the master end and the slave end according to the manipulation state information;

[0017] Obtaining the manipulation force at the moment before the delay based on the delay parameter;

[0018] Obtaining the manipulation force at the current moment according to the manipulation force at the moment before the delay.

[0019] Optionally, the steps of calculating the manipulation force based on the motion state information include:

[0020] Obtaining the manipulation force at the current moment according to the motion state information and the corresponding preset coefficient.

[0021] Optionally, the steps of calculating the manipulation force based on the interactive prompt information include:

[0022] Parsing to obtain command information according to the interactive prompt information, and determining a gain adjustment factor;

[0023] Adjusting to obtain the manipulation force at the current moment based on the manipulation force of the robotic arm at the previous moment, according to the command information and the gain adjustment factor.

[0024] Optionally, the step of obtaining a manipulation force based on the manipulation state information and the motion state information includes:

[0025] Based on the manipulation state information, set a resistance coefficient corresponding to the motion state information;

[0026] According to the motion state information, and in combination with the resistance coefficient, obtain the manipulation force.

[0027] Optionally, the step of setting a resistance coefficient corresponding to the motion state information based on the manipulation state information includes:

[0028] Based on the manipulation state information, predict the resistance coefficient through a trained learning model;

[0029] After the manipulation force is fed back to the driving force of the robotic arm, perform feedback monitoring on the driving effect of the robotic arm, and adjust the parameters of the learning model based on the monitoring results.

[0030] Optionally, the manipulation force is obtained by calculation in Cartesian space or joint space; the resistance coefficient is set corresponding to the obtaining method of the manipulation force.

[0031] Optionally, the manipulation force includes at least one of a simulated spring force, a simulated inertial force, Cartesian damping, joint damping, a gravity compensation force, and a friction compensation force.

[0032] Optionally, the surgical robot system further includes a communication device, and the master end and the slave end are configured to perform data exchange through the communication device to achieve remote operation; the manipulation state information includes network communication state information of the communication device, and the manipulation enhancement module is configured to obtain the manipulation force based on the network communication state information.

[0033] Optionally, the number of master ends is one or more than two.

[0034] In summary, the surgical robot system provided by the present invention includes: a master end, a slave end, and a controller; the master end includes a robotic arm; the controller includes a manipulation enhancement module, and the manipulation enhancement module is configured to perform the following steps: obtain at least one of manipulation state information, motion state information, and interaction prompt information; based on at least one of the manipulation state information, the motion state information, and the interaction prompt information, obtain a manipulation force; and feed back the manipulation force to the driving force of the robotic arm.

[0035] With such a configuration, by manipulating the settings of the enhancement module, the manipulation force can be obtained according to at least one of the manipulation state information, the motion state information, and the interaction prompt information. By feeding back this manipulation force into the driving force of the robotic arm, it is equivalent to providing a certain resistance or compensatory force to the operator. This resistance or compensatory force can effectively optimize the operation experience, help reduce the discomfort caused by the lack of tactile feedback, and enable the operator to better control the surgical robot system to perform complex surgical tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Those of ordinary skill in the art will understand that the provided drawings are used to better understand the present invention and do not constitute any limitation to the scope of the present invention.

[0037] Figure 1 is a schematic diagram of the surgical robot system according to an embodiment of the present invention.

[0038] Figure 2 is a schematic diagram of the master end according to an embodiment of the present invention.

[0039] Figure 3 is a schematic diagram of the master control arm according to an embodiment of the present invention.

[0040] Figure 4 is a schematic diagram of the delay of local control surgery according to an embodiment of the present invention.

[0041] Figure 5 is a schematic diagram of the delay of remote control surgery according to an embodiment of the present invention.

[0042] Figure 6 is a schematic diagram of the interactive software interface according to an embodiment of the present invention.

[0043] Figure 7 is a schematic diagram of the physical model for obtaining the master end resistance according to the master end speed according to an embodiment of the present invention.

[0044] Figure 8 is a control block diagram of the surgical robot system based on the master end resistance according to an embodiment of the present invention.

[0045] Figure 9 is a schematic diagram of the physical model for obtaining the slave end resistance according to the slave end speed according to an embodiment of the present invention.

[0046] Figure 10 is a control block diagram of the surgical robot system based on the slave end resistance according to an embodiment of the present invention.

[0047] Figure 11 is a schematic diagram of the physical model for obtaining the master-slave deviation resistance according to the master-slave speed deviation according to an embodiment of the present invention.

[0048] Figure 12It is a control block diagram of the surgical robot system according to an embodiment of the present invention based on the master-slave deviation resistance.

[0049] Figure 13 It is a control block diagram of the surgical robot system according to an embodiment of the present invention based on the master-end joint resistance.

[0050] In the accompanying drawings: 10 - master end; 11 - robotic arm; 12 - joint; 13 - motor; 14 - force interaction center; 15 - display device; 16 - desired operation trajectory; 20 - slave end; 21 - operating arm; 26 - following trajectory; 30 - image trolley; 40 - communication device. Detailed implementation manners

[0051] To make the objectives, advantages and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are all in very simplified forms and are not drawn to scale, only for the purpose of facilitating and clearly assisting in explaining the objectives of the embodiments of the present invention. In addition, the structures shown in the accompanying drawings are often part of the actual structures. In particular, the emphasis that each accompanying drawing needs to show is different, and sometimes different scales are used.

[0052] As used in the present invention, the singular forms "a", "an", "one" and "the" include plural referents, the term "or" is generally used in the sense of including "and / or", the term "several" is generally used in the sense of including "at least one", the term "at least two" is generally used in the sense of including "two or more", in addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", "third" may explicitly or implicitly include one or at least two of such features. "One end" and "the other end", as well as "proximal end" and "distal end" generally refer to two corresponding parts, which not only include the endpoints. In addition, as used in the present invention, "mounted", "connected", "coupled", an element "disposed" on another element should be understood in a broad sense, generally only indicating that there is a connection, coupling, cooperation or transmission relationship between the two elements, and the two elements may be directly or indirectly connected, coupled, cooperated or transmitted through an intermediate element, and cannot be construed as indicating or implying the spatial position relationship between the two elements, that is, an element may be in any position such as inside, outside, above, below or on one side of another element, unless the content clearly indicates otherwise. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, directional terms such as above, below, up, down, upward, downward, left, right, etc. are used relative to the exemplary embodiments as shown in the figures, and the upward or upper direction is towards the top of the corresponding figure, and the downward or lower direction is towards the bottom of the corresponding figure.

[0053] The object of the present invention is to provide a method for operating a surgical robot and a surgical robot system to solve the problem that the existing surgical robot system lacks an effective tactile feedback mechanism. The following will be described with reference to the accompanying drawings.

[0054] Please refer to Figures 1 to 3 , which exemplarily shows a surgical robot system, which includes a master control device (hereinafter simply referred to as "master 10" for convenience of description), a slave robot device (hereinafter simply referred to as "slave 20" for convenience of description), a controller (not shown), an image cart 30, etc.

[0055] The master 10 includes a robotic arm 11 (also referred to as a master control arm or master hand), which includes several joints 12 and motors 13. An operator (such as a doctor) can directly operate the robotic arm 11, and the end of the robotic arm 11 is in direct contact with the operator's hand, and this contact point is the force interaction center 14. The interaction force is generated by the motors 13 of the respective joints 12 of the robotic arm 11.

[0056] The slave end 20 includes a number of operating arms 21, which can be used to mount surgical instruments and image acquisition devices (such as endoscopes). The master end 10 and the slave end 20 are configured in a master-slave control relationship. The operations of the master end 10 can be mapped to the slave end 20 through a controller. The actions of the robotic arm 11 driven by the operator are mapped to the actions of the surgical instruments and image acquisition devices of the operating arms 21 through the controller, so as to achieve master-slave mapping operations and perform surgeries.

[0057] The image screen captured by the image acquisition device is fed back and transmitted to the image trolley 30, and after being processed by the image trolley 30, it is transmitted to the display device 15 of the master end 10 for display, for the operator to observe. The operator can operate based on the observed image screen.

[0058] In applications, in addition to mapping the operations of the master end 10 and the slave end 20, the controller also provides compensatory control forces for the robotic arm 11 of the master end 10 and the operating arms 21 of the slave end 20 to compensate for the effects of factors such as gravity and friction on the robotic arm 11 and the operating arms 21, so as to minimize the resistance felt by the operator when operating the master end 10 due to the gravity or friction of the robotic arm 11. However, in the prior art, the operation of the robotic arm 11 of the master end 10 by the operator often lacks an effective tactile feedback mechanism. In some scenarios, for example, when there is a certain delay between the master and slave ends, it causes the operator to feel like operating cotton in suspended air, making the operator feel uncomfortable when performing fine operations, thus affecting the surgical quality.

[0059] To solve this problem, an embodiment of the present invention provides a surgical robot system, which includes: a master end, a slave end, and a controller; the master end includes a robotic arm; the controller includes a manipulation enhancement module, and the manipulation enhancement module is configured to perform the following steps:

[0060] Step S1: Obtain at least one of manipulation state information, motion state information, and interaction prompt information;

[0061] Step S2: Obtain a manipulation force based on at least one of the manipulation state information, the motion state information, and the interaction prompt information;

[0062] Step S3: Feed back the manipulation force into the driving force of the robotic arm 11.

[0063] Optionally, the manipulation force here includes at least one of a simulated spring force, a simulated inertia force, a Cartesian damping, a joint damping, a gravity compensation force, and a friction compensation force. Among them, the simulated spring force, the simulated inertia force, the Cartesian damping, etc. refer to the operating resistance of the robotic arm 11 in the Cartesian space, and the joint damping refers to the resistance (or resistance moment) of the joint 12 in the joint space. The gravity compensation force and the friction compensation force are compensation control forces for gravity, friction, etc. The manipulation force obtained in step S2 can be any one or a combination of several of the foregoing.

[0064] In the embodiment of the present invention, through the setting of the manipulation enhancement module, the manipulation force can be obtained according to at least one of the manipulation state information, the motion state information, and the interaction prompt information. By feeding back the manipulation force to the driving force of the robotic arm 11, it is equivalent to providing a certain resistance or compensation force to the operator. This resistance or compensation force can effectively optimize the operation experience, help reduce the discomfort caused by the lack of tactile feedback, and enable the operator to better control the surgical robot system to perform complex surgical tasks.

[0065] The manipulation state information, the motion state information, and the interaction prompt information referred to in this embodiment will be described separately below.

[0066] In some embodiments, the manipulation state information includes communication state information. The communication between the master end 10 and the slave end 20 can be realized, for example, through network communication, satellite communication, digital communication, multimedia communication, etc. Different communication methods will have different state information. The communication state information includes at least one of the delay, bandwidth, throughput, jitter, bit error rate, and packet loss rate of the communication between the master end 10 and the slave end 20. The delay may include information such as the average delay or the maximum delay. The communication state information reflects the communication situation between the master end 10 and the slave end 20, and further reflects the delay situation between the master end 10 and the slave end 20.

[0067] It can be understood that a certain master-slave delay will inevitably be introduced in the surgical robot system with master-slave mapping operation. Please refer to Figure 4 and Figure 5 , which respectively show the local surgical delay and the remote surgical delay.

[0068] As Figure 4 shown, in local control surgery, the delay includes: the delay t1 from the master end 10 outputting a control signal to the slave end 20 receiving the signal; the delay t2 from the slave end 20 performing an action to the image acquisition device capturing the current image frame; and the delay t3 from the image trolley 30 outputting the image frame to the display device 15 of the master end 10 for imaging.

[0069] As Figure 5As shown, in remote-controlled surgery, the surgical robot system further includes a communication device 40 (such as a server). The master end 10 and the slave end 20 are configured to exchange data through the communication device 40 to achieve remote operation. The manipulation state information includes the network communication state information of the communication device 40, and the manipulation enhancement module is configured to obtain the manipulation force based on the network communication state information. In remote operation, due to the addition of the communication device 40, the communication state information includes the network communication state information of the communication device 40. The delays at this time include: the delay t4 from when the remote master end 10 outputs a control signal to when the communication device 40 receives the signal; the delay t5 from when the communication device 40 outputs a control signal to when the local slave end 20 receives the signal; the delay t6 from when the local slave end 20 performs an action to when the local image acquisition device captures the current image frame; the delay t7 from when the local image trolley 30 outputs an image frame to when the communication device 40 receives the signal (which may involve an encoding / decoding process); and the delay t8 from when the communication device 40 outputs a signal to when the display device 15 of the remote master end 10 forms an image.

[0070] It can be understood that the above delays can be mainly divided into two categories: Category A delays are the delays from when the instruction of the master end 10 reaches the slave end 20 for response. For local-controlled surgery, it is t1, and for remote-controlled surgery, it is t4 + t5. Category B delays are from when the slave end 20 responds to when the display device 15 forms an image. For local-controlled surgery, it is t2 + t3, and for remote-controlled surgery, it is t6 + t7 + t8.

[0071] It can be seen that whether it is local-controlled surgery or remote-controlled surgery, there will be a certain master-slave delay. To solve the problem that the operator feels out of sync with the actual operation when manipulating the master end 10, the inventor's research found that, considering the delay situation, applying an appropriate manipulation force (resistance) on the robotic arm 11 of the master end 10 can help reduce the discomfort caused by the delay, enabling the operator to better control the robotic arm 11 of the master end 10 to perform complex surgical tasks.

[0072] Optionally, the number of the master ends 10 is one or more than two. Different master ends 10 can communicate with the same slave end 20. Moreover, different master ends 10 can be respectively provided with corresponding different manipulation enhancement modules and respectively set different manipulation forces.

[0073] The previous embodiment described the method of obtaining the manipulation force based on the communication state information. It can be understood that the communication state information may vary depending on the different configurations of the master end 10 and the slave end 20, as well as their different communication methods. The acquisition of the communication state information can be achieved through some devices or sensors, etc. For simplicity, in some embodiments, the manipulation state information includes configuration state information, and the configuration state information is obtained based on at least one of the communication state, the image frame difference, and the motion state. The configuration state information here can be considered as the information obtained based on the automatic or manual configuration of the master end 10 and the slave end 20.

[0074] In one example, when the master end 10 and the slave end 20 are set, some parameters between them are actually fixed. For example, when the master end 10 and the slave end 20 are configured for remote-controlled surgery, the latency situation often depends on the communication state. For example, when the master end 10 and the slave end 20 communicate through a dedicated fiber optic network, it can be considered that their communication state is good, and the latency can be not detected, and a relatively low configuration state information can be directly configured to obtain a relatively low manipulation force (resistance). Another example is that if the communication state between the master end 10 and the slave end 20 is poor, a relatively high configuration state information can be configured, and thus a relatively high manipulation force (resistance) can be obtained.

[0075] In one example, the configuration state information can be obtained based on the image frame difference. An image frame refers to the frame of the image screen captured by an image acquisition device, and the image frame difference refers to the inter-frame latency and / or the change speed of the image frame. By regularly calculating the time difference between image frames, the inter-frame latency can be obtained. To a certain extent, it also reflects the latency situation between the master end 10 and the slave end 20. In one embodiment, the latency of each frame can be calculated through a timestamp or a frame difference algorithm. For example, assuming that the timestamp of the first image frame is T1 and the timestamp of the second image frame is T2, then the inter-frame latency ΔT = T2 - T1. If the inter-frame latency ΔT is large, that is, the time difference between consecutive image frames is large, it indicates a high latency and a slow system response, and a relatively high configuration state information can be configured, and thus a relatively high manipulation force (resistance) can be obtained. In some other embodiments, the change speed of the image frame can also be obtained by calculating the change situation of the image frame (such as how many frames change per second, or the pixel change amount between consecutive image frames). If the change speed of the image frame is relatively slow or discontinuous, it may indicate a serious latency, and a relatively high configuration state information and manipulation force (resistance) can be configured.

[0076] In one example, the configuration status information can be obtained based on the motion state, where the motion state refers to the position, speed, acceleration, etc. of the motion of the master end 10 and / or the slave end 20. Based on the motion state, appropriate configuration status information can be directly configured to meet the requirements. For example, smaller configuration status information is suitable for fast-moving operations, medium configuration status information is suitable for operations that require a certain degree of precision, and higher configuration status information is suitable for high-precision and low-speed operations.

[0077] Of course, the configuration status information can include the combination of any two or the combination of all three of the above network status, image frame difference, and motion state. In some embodiments, the configuration status information can be stored in the surgical robot system, such as stored in the controller or the master end 10. After the master end 10 and the slave end 20 are set fixed, the configuration status information can be directly stored in the controller based on the master end 10 and the slave end 20. In this way, when subsequent operations are performed, the configured status information stored in the controller is automatically read, and the adapted manipulation force can be obtained and fed back to the driving force of the robotic arm 11.

[0078] In other embodiments, the configuration status information can also be obtained based on the interactive hardware or interactive software. Please refer to Figure 6 , which shows a demonstration example of obtaining the configuration status information based on the interactive software. The specific medium of the interactive software can be, for example, a touch screen, etc., and it can be optionally set on the master end 10 for easy operator interaction. Through this touch screen, the operator can directly input the configuration status information, such as Figure 6 the drag manipulation resistance setting bar shown. The interactive hardware includes, for example, a DIP switch, physical buttons, etc., which can set several fixed gears or can be set with stepless variation. This embodiment is not limited thereto. The operator can also achieve interactive input of the configuration status information.

[0079] In step S2, the steps of calculating the manipulation force based on the manipulation status information include:

[0080] Step S2A1: Obtain a delay parameter associated with the delay of the master end 10 and the slave end 20 according to the manipulation status information;

[0081] Step S2A2: Based on the delay parameter, obtain the manipulation force at the moment before the delay;

[0082] Step S2A3: Obtain the manipulation force at the current moment according to the manipulation force at the moment before the delay.

[0083] As described above, during the operation of the surgical robot system, compensation control forces such as gravity compensation force and friction compensation force are provided. Whether the manipulation state information includes communication state information or configuration state information, it essentially manifests as a delay parameter d associated with delay. The manipulation force at the current moment can be obtained through the relationship between the manipulation force at the moment before delay and the delay parameter d:

[0084] F(t)=a*F control (t - d)

[0085] Where: F(t) is the manipulation force at the current moment; F control (t - d) is the manipulation force at the moment before delay. a is an adjustable parameter. Taking the delay parameter d representing a 1 - second delay as an example, F control (t - d) is equivalent to the manipulation force 1 second ago, and this manipulation force is, for example, a compensation control force. Then the current manipulation force F(t) is obtained by multiplying the compensation control force 1 second ago by the adjustable parameter a. Either the communication state information or the configuration state information in the manipulation state information can establish a mapping relationship with the delay parameter d for conversion.

[0086] In step S2, the manipulation force can also be obtained based on the motion state information. The motion state information includes at least one of the master - end motion information, the slave - end motion information, and the motion deviation information between the master - end 10 and the slave - end 20; the master - end motion information and the slave - end motion information respectively include at least one of the position information x, the velocity information , the acceleration information and the joint - angle information θ; the motion deviation information includes at least one of the position deviation information, the velocity deviation information, the acceleration deviation information, and the joint - angle deviation information.

[0087] The steps of calculating the manipulation force based on the motion state information include:

[0088] Step S2B1: Obtain the manipulation force at the current moment according to the motion state information and the corresponding preset coefficients.

[0089] For the manipulation force in the Cartesian space, its model can be set as a second - order system, which is expressed as a second - order ordinary differential equation, and its form is as follows:

[0090]

[0091] Where: x is the position information (or position deviation information), is the velocity information (or velocity deviation information), is the acceleration information (or acceleration deviation information), m is the preset virtual mass (or preset virtual inertia), b is the preset damping coefficient, k is the preset stiffness coefficient (spring constant), and F is the manipulation force.

[0092] For the manipulation force in the joint space, according to the mapping matrix J from Cartesian space to joint space T , the conversion between the joint torque τ and the force F in the Cartesian space can be achieved through the formula τ = J T ·F. Thus, the manipulation force can be directly generated at the joint end.

[0093] In step S2B1, by means of preset coefficients, as long as the current motion state information is known, the manipulation force at the current moment can be dynamically obtained. Further, when it is fed back to the driving force of the robotic arm 11, the motion conditions of the current master end 10 and / or slave end 20 can be matched in real time.

[0094] In step S2, the manipulation force can also be obtained based on the interaction prompt information. The interaction prompt information includes at least one of voice prompt information, text prompt information, graphic prompt information, image prompt information, and video prompt information. These interaction prompt information can be obtained based on the voice, text, graphics, images, etc. issued by the operator.

[0095] The steps of calculating the manipulation force based on the interaction prompt information include:

[0096] Step S2C1: According to the interaction prompt information, parse to obtain the command information and determine the gain adjustment factor;

[0097] Step S2C2: Based on the manipulation force of the robotic arm 11 at the previous moment, adjust to obtain the manipulation force at the current moment according to the command information and the gain adjustment factor.

[0098] Taking the interaction prompt information including voice prompt information as an example, in step S2C1, through voice recognition, the voice prompt information (such as voice commands) of the operator can be converted into command information, and the form of the command information can be, for example, digital signals or text. Further, a natural language processing model, etc. can be used to analyze the command information to determine the nature of the command (such as "increase the force" or "decrease the force").

[0099] Further, determine the gain adjustment factor according to the parsed command information to reflect the intensity of the manipulation force that the operator hopes to adjust.

[0100] In step S2C2, the gain adjustment factor can participate in the adjustment of the manipulation force in different forms.

[0101] Form 1: The gain adjustment factor can set different gain levels, such as "increase by 10%" and "increase by 50%". Use the adjusted gain adjustment factor K command to calculate the manipulation force at the current moment:

[0102] F = Kcommand ·F control

[0103] wherein, F control is the manipulation force at the previous moment on the robotic arm 11, for example, the compensated control force before adjustment, and F is the manipulation force at the current moment, that is, the new compensated control force after adjustment.

[0104] Form 2: The gain adjustment factor can be directly set as the increment or decrement of the manipulation force ΔF:

[0105] F = F control + ΔF

[0106] According to the new manipulation force F after adjustment, it is fed back to the driving force of the robotic arm 11, and the operator's requirements can be responded to.

[0107] In some embodiments, in step S2, the manipulation force can also be obtained based on the combination of the manipulation state information and the motion state information. The steps include:

[0108] Step S2D1: Based on the manipulation state information, set the resistance coefficient corresponding to the motion state information;

[0109] Step S2D2: According to the motion state information, combine the resistance coefficient to obtain the manipulation force.

[0110] Based on the second-order ordinary differential equation model of the manipulation force described above, any one of the preset coefficients in step S2D1 can be replaced with the resistance coefficient corresponding to the motion state information, so that the manipulation force obtained in step S2D2 is more in line with the current motion state and the requirements of the current actual operation.

[0111] Optionally, the manipulation force is calculated and obtained through the Cartesian space or the joint space; the resistance coefficient is set corresponding to the obtaining method of the manipulation force. The resistance coefficient in step S2D1 corresponds to the motion state information. In some embodiments, the manipulation force is calculated and obtained through the Cartesian space. At this time, the motion state information includes at least one of position information, velocity information, acceleration information, position deviation information, velocity deviation information, and acceleration deviation information. The corresponding resistance coefficient is also set corresponding to the Cartesian space. Corresponding to the position information or the position deviation information, the resistance coefficient is expressed as a virtual spring constant; corresponding to the velocity information or the velocity deviation information, the resistance coefficient is expressed as a damping coefficient; corresponding to the acceleration information or the acceleration deviation information, the resistance coefficient is expressed as a virtual mass. In other embodiments, the manipulation force is calculated and obtained through the joint space. At this time, the motion state information includes joint angle information, and the resistance coefficient is also set correspondingly.

[0112] In step S2D1, the drag coefficient is set according to the control state information. As described above, whether the control state information includes communication state information or configuration state information, it can actually be converted into a delay parameter d associated with the delay. In one embodiment, a drag coefficient Ct based on the delay parameter d and an adjustable parameter kt can be defined, and the drag coefficient Ct can be obtained through the formula Ct = kt * d.

[0113] For example, in one embodiment, the control state information includes communication state information, specifically including the delay in communication between the master end 10 and the slave end 20. Based on this delay, the delay parameter d can be obtained, and then the drag coefficient Ct can be obtained through the formula Ct = kt * d. Another example is that in another embodiment, the control state information includes configuration state information, specifically including the information input according to the interaction software. Based on this configuration state information, the delay parameter d can also be quantified to obtain the drag coefficient Ct.

[0114] In another embodiment, the drag coefficient can also be predicted through a software algorithm. Optionally, step S2D1 includes:

[0115] Step S2D11: Based on the control state information, predict the drag coefficient through a trained learning model;

[0116] Step S2D12: After feeding the manipulation force back to the driving force of the robotic arm 11, perform feedback monitoring on the driving effect of the robotic arm 11, and adjust the parameters of the learning model based on the monitoring results.

[0117] The drag coefficient can also be adjusted in real time through a software algorithm (such as machine learning method) according to the delay parameter d. In a demonstration example, a learning model can be trained first. The learning model can be selected from regression models (such as linear regression, random forest regression) or neural network models, etc., and selected according to the complexity and characteristics of the data. The training process can use historical data for training, and the training goal is to predict the optimal drag coefficient to cope with different delay situations. The loss function can be set as the difference between the predicted drag coefficient and the actual required drag coefficient.

[0118] The training process may optionally include steps such as data collection, feature selection, and data preprocessing. Data collection step: Obtain the delay parameter d between the master and slave ends, the speeds of the master end 10 and the slave end 20, and the current drag coefficient, etc. Feature selection step: Extract key features from the collected data, such as: master end speed, slave end speed, delay time, current drag coefficient, etc. Data preprocessing step: Clean and standardize the data to ensure the effectiveness and accuracy during the training of the learning model.

[0119] After the learning model is trained, the resistance coefficient can be predicted in real time according to the real-time monitored delay parameter d and combined with the motion state of the current surgical robot system.

[0120] Furthermore, a certain feedback mechanism can also be set for the method of predicting the resistance coefficient by the learning model. After the predicted resistance coefficient is applied in practice in step S2D12, the actual driving effect can be monitored through the feedback mechanism. If the actual driving effect is not ideal, the parameters in the learning model can be adjusted by the method of online learning to further improve the prediction accuracy.

[0121] When the manipulation force is obtained by Cartesian space calculation, it can be added to the driving force of the robotic arm 11 in the form of Cartesian space force. When the manipulation force is obtained by joint space calculation, it can be added to the joint 12 of the robotic arm 11 in the form of joint torque. The following takes the form of Cartesian space force as an example for illustration.

[0122] In an exemplary embodiment, taking the motion state information including velocity information as an example, in the surgical robot system, the velocity information can actually be divided into the master velocity V master , the slave velocity V salve and the master-slave velocity deviation (V master - V salve ).

[0123] Please refer to Figure 7 and Figure 8 , taking the master velocity V master as an example for illustration. The master velocity V master refers to the velocity of the robotic arm 11 in the Cartesian space. The resistance coefficient corresponding to the master velocity V master is the master damping coefficient C vm . According to the master velocity V master and the corresponding master damping coefficient C vm , the master resistance F mastervelocity can be obtained. Its physical model is as shown in Figure 7 . Figure 7 As shown in, the robotic arm 11 moves along the desired operation trajectory 16, and its Cartesian velocity is the master velocity V master . At this time, a master resistance F master in the direction opposite to the master velocity V mastervelocity is applied to the robotic arm 11, so as to provide resistance feedback for the operator, reduce the discomfort caused by the delay, and improve the operation stability. The master resistance F mastervelocity is obtained according to the master velocity V master and the master damping coefficient C vm . Specifically, the following formula:

[0124] F mastervelocity = Cvm ·V master

[0125] Drag coefficient C vm It can be obtained according to the aforementioned step S2D1, which will not be elaborated here.

[0126] In one embodiment, the manipulation force may include the main-end resistance F mastervelocity , and this main-end resistance F mastervelocity Obtain the joint torque τ through the transpose of the main hand Jacobian, and combine with the output of the main hand dynamics to obtain the joint torque τ of the main end 10 f , so that the motor 13 of the robotic arm 11 drives according to the joint torque τ f .

[0127] At this time, the control block diagram of the entire surgical robot system is as Figure 8 shown. Where θ m is the angle of the joint 12 of the main end 10, X m is the Cartesian position of the main end 10, V m (that is, V master ) is the Cartesian velocity of the main end 10, X’ m , V’ m are the main-end commands converted to the control system, θ s is the joint angle of the slave end 20, θ’ s is the joint angle after position and velocity limitation, τ pd is the joint torque of the slave end 20.

[0128] Please refer to Figure 9 and Figure 10 , in another embodiment, since the force received by the slave end 20 will also be fed back to the main end 10 based on the master-slave mapping, the corresponding slave-end resistance can also be generated according to the velocity of the slave end 20, so as to obtain the manipulation force. Specifically, the velocity state includes the slave-end velocity V slave , and this slave-end velocity V slave refers to the velocity of the end of the manipulator 21 of the slave end 20 (which may also include surgical instruments or image acquisition devices, etc.) in the Cartesian space. The drag coefficient corresponding to the slave-end velocity V slave is the slave-end damping coefficient C vs , and the slave-end resistance can be obtained according to the slave-end velocity V slave and the corresponding slave-end damping coefficient C vs , and its physical model is as Figure 10 shown. Figure 10 In, the robotic arm 11 moves along the desired operation trajectory 16, and the end of the manipulator 21 of the slave end 20 moves along the following trajectory 26, and its Cartesian velocity is the slave-end velocity V slave , and at this time, a force is applied to the end of the manipulator 21 that is opposite to the slave-end velocity V slaveThe slave-end resistance F in the opposite direction slavevelocity can also provide resistance feedback to the operator. The slave-end resistance F slavevelocity is obtained according to the slave-end velocity V slave and the slave-end damping coefficient C vs as follows:

[0129] F slavevelocity = C vs ·V slave

[0130] In one embodiment, the manipulation force may include the slave-end resistance F slavevelocity , and the slave-end resistance F slavevelocity obtains the joint torque τ through the transpose of the master hand Jacobian, and combines the output of the master hand dynamics to obtain the joint torque τ of the master end 10 f , so that the motor 13 of the robotic arm 11 is driven according to the joint torque τ f . At this time, the control block diagram of the entire surgical robot system is as shown in Figure 10 X s is the Cartesian position of the slave end 20, and V s (i.e., V slave ) is the Cartesian velocity of the slave end 20. The meanings of the other symbols can be referred to Figure 10 .

[0131] Please refer to Figure 11 and Figure 12 . In another embodiment, the master-slave deviation resistance can also be generated according to the deviation between the master-end velocity V master and the slave-end velocity V slave , that is, the master-slave velocity deviation (V master - V salve ), so as to obtain the manipulation force

[0132] The resistance coefficient corresponding to the master-slave velocity deviation (V master - V salve ) is the master-slave velocity deviation damping coefficient C d . According to the master-slave velocity deviation (V master - V salve ) and the corresponding master-slave velocity deviation damping coefficient C d , the master-slave deviation resistance can be obtained, and its physical model is as shown in Figure 11 . Figure 11 In, the robotic arm 11 moves along the desired operation trajectory 16, and its Cartesian velocity is V master . The end of the operating arm 21 of the slave end 20 moves along the following trajectory 26, and its Cartesian velocity is the slave-end velocity V salve . At this time, a master-slave deviation resistance F deviation, and can also provide resistance feedback to the operator. The master-slave deviation resistance F deviation can be obtained according to the master-slave speed deviation (V master - V salve ) and the master-slave speed deviation damping coefficient C d . Specifically, the formula is as follows:

[0133] F deviation = C d · (V master - V salve )

[0134] In one embodiment, the manipulation force may include the master-slave deviation resistance F deviation , and this master-slave deviation resistance F deviation is compensated into the driving force of the master end 10. At this time, the control block diagram of the entire surgical robot system is as Figure 12 shown.

[0135] It should be noted that the manipulation force may include at least one of the master end resistance F mastervelocity , the slave end resistance F slavevelocity , and the master-slave deviation resistance F deviation . Preferably, the manipulation force may include any two or three of the master end resistance F mastervelocity , the slave end resistance F slavevelocity , and the master-slave deviation resistance F deviation .

[0136] In step S3, the manipulation force is fed back into the driving force of the robotic arm 11, that is, according to the manipulation force obtained in step S2, it is synthesized with the control force F control of the robotic arm 11 (including joint position control, dynamic feedforward, etc.) to form the final operating force F total . Specifically, the formula is as follows:

[0137] F total = F control + F mastervelocity + F slavevelocity + F deviation

[0138] Of course, it can be understood that since the manipulation force can also include one or any combination of two of the master end resistance F mastervelocity , the slave end resistance F slavevelocity , and the master-slave deviation resistance F deviation , the synthesis of the final operating force F total can also be changed accordingly.

[0139] In the above embodiments, the calculation of the manipulation force is described by taking the motion state information including velocity information as an example. In some other embodiments, the motion state information may also include position information or acceleration information. The position information includes the master end position information, the slave end position information, and the position deviation information between the master end 10 and the slave end 20. The acceleration information includes the master end acceleration information, the slave end acceleration information, and the acceleration deviation information between the master end 10 and the slave end 20. Any one or any combination of these information can also be used to calculate the manipulation force.

[0140] Taking the master end position information x m as an example, the resistance coefficient corresponding to the master end position information x m is the master end spring constant k m . According to the master end position information x m and the corresponding master end spring constant k m , the master end simulated spring force F spring can be obtained, as shown in the following formula:

[0141] F spring = k m ·x m

[0142] The slave end simulated spring force and the master-slave deviation simulated spring force can be understood correspondingly and will not be elaborated here. The manipulation force may include at least one of the master end simulated spring force F spring , the slave end simulated spring force, and the master-slave deviation simulated spring force, and then feedback to the driving force of the robotic arm 11.

[0143] Taking the master end acceleration information a m as an example, the resistance coefficient corresponding to the master end acceleration information a m is the master end virtual mass m m . According to the master end acceleration information a m and the corresponding master end virtual mass m m , the master end simulated inertial force F inertia can be obtained, as shown in the following formula:

[0144] F inertia = m m ·a m

[0145] The slave end simulated inertial force and the master-slave deviation simulated inertial force can be understood correspondingly and will not be elaborated here. The manipulation force may include at least one of the master end simulated inertial force F inertia , the slave end simulated inertial force, and the master-slave deviation simulated inertial force, and then feedback to the driving force of the robotic arm 11.

[0146] Continuing to take the master end 10 as an example, the manipulation force may simultaneously include the master end resistance F mastervelocity, the main - end simulated spring force F spring and the main - end simulated inertial force F inertia , or any one or any combination of the two. The manipulation force based on the slave - end 20 and based on the master - slave deviation can be understood accordingly, which will not be elaborated here.

[0147] The manipulation forces in the foregoing several embodiments are all obtained through Cartesian - space calculation. In another embodiment, an example is given in the way that the manipulation force is obtained through joint - space calculation. At this time, the motion - state information includes joint - angle information, and the joint - angle information further includes at least one of the main - end joint angle, the slave - end joint angle, and the master - slave joint - angle deviation. The corresponding resistance coefficient is set corresponding to the joint rotation. Taking the main - end joint angle θ m as an example, at this time, the resistance coefficient is set corresponding to the joint rotation. Through the main - end joint angle θ m and the corresponding resistance coefficient, the joint torque τ corresponding to the main - end joint resistance can be directly obtained. Further, by combining the joint torque τ with the master - hand dynamics output, the joint torque τ f of the main - end 10 can be obtained, so that the motor 13 of the robotic arm 11 is driven according to the joint torque τ f . At this time, the control block diagram of the entire surgical - robot system is as shown in Figure 13 . The calculation based on the slave - end joint angle and the master - slave joint - angle deviation can be understood accordingly, which will not be elaborated here.

[0148] In summary, the surgical - robot system provided by the present invention includes: a main - end, a slave - end, and a controller; the main - end includes a robotic arm; the controller includes a manipulation - enhancement module, and the manipulation - enhancement module is configured to perform the following steps: obtain at least one of manipulation - state information, motion - state information, and interaction - prompt information; obtain a manipulation force based on at least one of the manipulation - state information, the motion - state information, and the interaction - prompt information; and feedback the manipulation force to the driving force of the robotic arm. With such a configuration, through the setting of the manipulation - enhancement module, the manipulation force can be obtained according to at least one of the manipulation - state information, the motion - state information, and the interaction - prompt information. By feeding back the manipulation force to the driving force of the robotic arm, it is equivalent to providing a certain resistance or compensation force to the operator. This resistance or compensation force can effectively optimize the operation experience, help reduce the discomfort caused by the lack of haptic feedback, and enable the operator to better control the surgical - robot system to perform complex surgical tasks.

[0149] It should be noted that the foregoing several embodiments can be combined with each other. The above description is only a description of the preferred embodiments of the present invention, and does not limit the scope of the present invention in any way. Any changes and modifications made by those of ordinary skill in the art of the present invention according to the above disclosure belong to the protection scope of the present invention.

Claims

1. A surgical robot system, characterized in that: include: A master end, a slave end and a controller; the master end includes a robotic arm; The controller includes a control enhancement module, and the control enhancement module is configured to perform the following steps: Acquiring control state information; and obtaining control force based on the control state information; Feeding back the manipulation force into the driving force of the robotic arm; Based on the control state information, the step of obtaining the control force includes: Obtaining a delay parameter associated with a delay of the master end and the slave end according to the control state information; Based on the delay parameter, obtaining the control force at the moment before the delay; The control force at the current moment is obtained according to the control force at the moment before the delay.

2. The surgical robot system according to claim 1, characterized in that: The control state information includes communication state information, and the communication state information includes at least one of a delay, a bandwidth, a throughput, a jitter, a bit error rate, and a packet loss rate of communication between the master end and the slave end.

3. The surgical robot system according to claim 1, characterized in that: The manipulation state information includes configuration state information, and the configuration state information is obtained based on at least one of a communication state, an image frame difference, and a motion state.

4. The surgical robot system according to claim 3, characterized in that: The configuration status information is stored in the surgical robot system, or the configuration status information is obtained based on interactive hardware or interactive software.

5. The surgical robot system according to claim 1, characterized in that: The control force is also obtained based on motion state information; the motion state information includes master-end motion information, slave-end motion information and at least one of motion deviation information of the master and slave ends; the master-end motion information and the slave-end motion information respectively include at least one of position information, speed information, acceleration information and joint angle information; the motion deviation information includes at least one of position deviation information, speed deviation information, acceleration deviation information and joint angle deviation information.

6. The surgical robot system according to claim 1, characterized in that: The control force is also obtained based on the interactive prompt information; the interactive prompt information includes at least one of voice prompt information, text prompt information, graphic prompt information, image prompt information and video prompt information.

7. The surgical robot system according to claim 1, characterized in that: The control force is also obtained based on the motion state information, and the step of calculating the control force further includes: The control force at the current moment is obtained according to the motion state information and the corresponding preset coefficient.

8. The surgical robot system according to claim 1, characterized in that: The control force is also obtained based on the interactive prompt information, and the step of calculating the control force further includes: According to the interactive prompt information, the command information is parsed and the gain adjustment factor is determined; Based on the control force of the robot arm at a previous moment, the control force at a current moment is adjusted according to the command information and the gain adjustment factor.

9. The surgical robot system according to claim 5, characterized in that: Based on the control state information and the motion state information, the step of obtaining the control force includes: Based on the control state information, setting a drag coefficient corresponding to the motion state information; The control force is obtained according to the motion state information and in combination with the drag coefficient.

10. The surgical robot system according to claim 9, characterized in that: Based on the control state information, the step of setting a drag coefficient corresponding to the motion state information comprises: Based on the control state information, the drag coefficient is predicted by a trained learning model; After the manipulation force is fed back into the driving force of the robotic arm, feedback monitoring is performed on the driving effect of the robotic arm, and the parameters of the learning model are adjusted based on the monitoring results.

11. The surgical robot system according to claim 9, characterized in that: The control force is obtained by calculation in Cartesian space or joint space; and the resistance coefficient is set corresponding to the way of obtaining the control force.

12. The surgical robot system according to claim 1, characterized in that: The control force includes at least one of a simulated spring force, a simulated inertial force, a Cartesian damping, a joint damping, a gravity compensation force, and a friction compensation force.

13. The surgical robot system according to claim 1, characterized in that: The surgical robot system also includes a communication device, and the master end and the slave end are configured to exchange data through the communication device to achieve remote operation; the control status information includes network communication status information of the communication device, and the control enhancement module is configured to obtain the control force based on the network communication status information.

14. The surgical robot system according to claim 1, characterized in that: The number of the master terminals is one or more than two.

15. The surgical robot system according to claim 1, characterized in that: The control force is obtained based on the control state information, the motion state information and the interaction prompt information.

Images