Surgical robot force control parameter updating method, device, equipment, medium and product
By using a pre-trained algorithm to automatically adjust the force control model parameters in the force control parameter update mode of the surgical robot, the problems of high maintenance costs and low efficiency caused by changes in the mechanical structure of the surgical robot are solved, and automated updates and improved accuracy are achieved.
Patent Information
- Application Number
- CN202410839442.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-26
- Publication Date
- 2025-12-05
- Estimated Expiration
- 2044-06-26
AI Technical Summary
During the use of surgical robots, the force control parameters need to be frequently adjusted due to changes in the mechanical structure, resulting in high maintenance costs and low efficiency. Existing technologies rely on manual adjustments and experience, making it difficult to update them efficiently.
By driving the surgical arm joint to move under the control of a preset load torque and the force control model to be updated in the force control parameter update mode, motion state parameters are collected, and an update strategy is generated using a pre-trained parameter update algorithm model to automatically adjust the force control model parameters. The update effect is then verified.
It enables automated updating of the force control parameters of the surgical robot, reduces maintenance costs, improves the accuracy and convenience of operation, and reduces human intervention.
Smart Images

Figure CN118752478B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of mechanical control technology, and in particular to a method, apparatus, device, medium and product for updating force control parameters of a surgical robot. Background Technology
[0002] Surgical robots typically consist of one or more robotic arms. To adapt to different surgical procedures, surgical robots usually support drag-and-teach functionality for the robotic arms. The control system of a surgical robot needs to ensure that the robotic arms can follow the surgeon's movements to change positions, while maintaining sufficient stability to avoid accidental operations.
[0003] In the process of realizing the drag teaching function of a robotic arm through zero-force control using torque control, it is usually necessary to collect the motion data of the robotic arm, then design and build a dynamic model for parameter identification, and finally, engineers make adjustments based on experience on actual machines, which requires a certain amount of human development costs. Moreover, surgical robots are affected by the decay of structural components and equipment transportation, resulting in deformation of the mechanical structure or changes in the clearance of the transmission structure, which leads to changes in friction and makes the original force control parameters unsuitable. At this time, engineers need to intervene and recalibrate the force control parameters according to the current working conditions, resulting in high development and maintenance costs, and the efficiency of force control parameter updates also needs to be improved. Summary of the Invention
[0004] This invention provides a method, apparatus, device, medium, and product for updating force control parameters of a surgical robot. It can automatically update the parameters of the force control model in the drag-and-drop teaching function, reducing the cost of function maintenance and improving the accuracy and convenience of surgical robot operation.
[0005] In a first aspect, embodiments of the present invention provide a method for updating force control parameters of a surgical robot, the method comprising:
[0006] In the force control parameter update mode, the target surgical arm joint is driven to move under the control of a preset load torque and the force control model to be updated, and motion state parameters are collected during the movement process.
[0007] The motion state parameters are input into a pre-trained parameter update algorithm model to obtain the force control parameter update strategy;
[0008] The parameters of the force control model to be updated are updated according to the force control parameter update strategy to obtain the updated force control model;
[0009] The update effect of the updated force control model is verified to complete the force control parameter update process.
[0010] In a second aspect, embodiments of the present invention provide a force control parameter update device for a surgical robot, the device comprising:
[0011] The motion state parameter acquisition module is used to drive the target surgical arm joint to move under the control of a preset load torque and the force control model to be updated in the force control parameter update mode, and to acquire motion state parameters during the movement process.
[0012] The parameter update strategy determination module is used to input motion state parameters into a pre-trained parameter update algorithm model to obtain the force control parameter update strategy.
[0013] The force control parameter update module is used to update the parameters of the force control model to be updated according to the force control parameter update strategy, so as to obtain the updated force control model.
[0014] The force control parameter update determination module is used to verify the update effect of the updated force control model in order to complete the force control parameter update process.
[0015] Thirdly, embodiments of the present invention also provide a computer device, the computer device comprising:
[0016] One or more processors;
[0017] Memory, used to store one or more programs;
[0018] When the above one or more programs are executed by the above one or more processors, the above one or more processors implement the force control parameter update method for surgical robots as provided in any embodiment of the present invention.
[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the force control parameter update method for a surgical robot as provided in any embodiment of the present invention.
[0020] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the force control parameter update method for a surgical robot as provided in any embodiment of the present invention.
[0021] The embodiments of the above invention have the following advantages or beneficial effects:
[0022] In this embodiment of the invention, the target surgical arm joint is driven to move under the control of a preset load torque and the force control model to be updated in a force control parameter update mode, and motion state parameters are collected during the movement. These motion state parameters are then input into a pre-trained parameter update algorithm model to obtain a force control parameter update strategy. The parameters of the force control model to be updated are updated according to the force control parameter update strategy to obtain the updated force control model. The update effect of the updated force control model is verified to complete the force control parameter update process. This invention solves the problem of high maintenance costs and low efficiency of surgical robot functions. It can automatically update the parameters of the force control model in the drag-and-drop teaching function through an algorithm, reducing the cost of function maintenance and improving the accuracy and convenience of surgical robot operation. Attached Figure Description
[0023] Figure 1 This is a flowchart of a force control parameter update method for a surgical robot provided in an embodiment of the present invention;
[0024] Figure 2 This is a flowchart of a force control parameter update method for a surgical robot provided in an embodiment of the present invention;
[0025] Figure 3 This is a flowchart illustrating an application example of a force control parameter update method for a surgical robot provided in this embodiment of the invention;
[0026] Figure 4 This is a schematic diagram of the structure of a force control parameter update device for a surgical robot provided in an embodiment of the present invention;
[0027] Figure 5 This is a schematic diagram of the offline training process of a reinforcement learning agent for parameter updating provided in an embodiment of the present invention;
[0028] Figure 6 This is a schematic diagram of the training process of a reinforcement learning agent for parameter updating provided in an embodiment of the present invention;
[0029] Figure 7 This is a schematic diagram of force control parameter updating based on a reinforcement learning agent, provided by an embodiment of the present invention;
[0030] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation
[0031] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.
[0032] Figure 1 This is a flowchart illustrating a method for updating force control parameters of a surgical robot according to an embodiment of the present invention. This embodiment is applicable to scenarios involving updating the control parameters of a surgical robot, particularly in cases of adaptive updates to the control parameters. The method can be executed by a force control parameter updating device for the surgical robot, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.
[0033] like Figure 1 As shown, the force control parameter update method for the surgical robot in this embodiment includes the following steps:
[0034] S110. In the force control parameter update mode, the target surgical arm joint is driven to move under the control of the preset load torque and the force control model to be updated, and the motion state parameters during the movement are collected.
[0035] The force control parameter update mode can be a working mode of the surgical robot. In this mode, the surgical robot can automatically perform force control parameter correction operations based on its own operating conditions to optimize the parameters of the force control model in the drag-and-teach mode. For example, after transportation, parameter correction can be performed before formal use; or, after the surgical robot has experienced extreme external environmental influences and has been used for a long time, the joint components of the surgical robot may be worn and require parameter correction. In this case, the force control parameter update mode can be activated to perform an adaptive update of the force control parameters.
[0036] For the force control parameter update function, corresponding functional controls can be set to trigger this function, causing the surgical robot to enter the force control parameter update mode. These functional controls can be physical controls set on the surgical robot or virtual controls set in the surgical robot's functional interaction interface. The force control parameter update mode can be initiated through either physical controls or virtual controls.
[0037] The preset load torque is an additional force designed to mimic the external force applied to the surgical arm during the drag-and-teach function. Under the action of the preset load torque, the surgical arm joints of the surgical robot can move along a pre-defined trajectory. The value of the preset load torque is set based on the load torque values used during the algorithm model training process, updated according to the following parameters.
[0038] The force control model is the dynamic model driving the movement of the surgical arm joints, which incorporates a friction compensation model under specific working conditions. The force control model to be updated is the object being updated in a single force control parameter optimization. The target surgical arm joint can be the control target object of the force control model to be updated during the current force control parameter update process. The target surgical arm joint can include one or more joints of the main arm and / or slave arm of the surgical robot.
[0039] In the force control parameter update mode, the target surgical arm joint is driven to move under the control of a preset load torque and the force control model to be updated, and motion state parameters are collected during the movement. Among them, the motion state parameters can be data of the target surgical arm joint during the movement process, including relevant variables that can characterize its motion state, such as torque value, joint angle, velocity, and acceleration.
[0040] S120. Input the motion state parameters into the pre-trained parameter update algorithm model to obtain the force control parameter update strategy.
[0041] The pre-trained parameter update algorithm model can output a force control parameter update strategy based on the input motion state parameters. The force control parameter update strategy can update the adjustable parameters in the force control model to be updated. Specifically, the strategy can correspond to the adjustment amount of each adjustable parameter, such as increasing or decreasing the value. For example, the force control parameter model can be represented as follows: Where τ is the joint torque and θ is the joint angle. For speed, Let M(θ) be the acceleration, M(θ) be the mass matrix of the surgical arm, and G(θ) be the gravity vector. τ represents the vector of centrifugal force and Coriolis force. fric The Coulomb viscous friction model can be used to calculate the frictional torque.
[0042] Motion state parameters can be θ, where θ is the joint angle. For speed, For acceleration. Adjustable parameters can be the viscosity coefficient, Coulomb friction, and... (the parameters mentioned in the force control parameter model above). The coefficients of related variables.
[0043] S130. Update the parameters of the force control model to be updated according to the force control parameter update strategy to obtain the updated force control model.
[0044] According to the force control parameter update strategy, the updated force control model can be obtained by updating the adjustable parameters based on their current values in the force control model to be updated.
[0045] S140. Verify the update effect of the updated force control model to complete the force control parameter update process.
[0046] The updated force control model should ensure that the target surgical arm joint moves under the control of a preset load torque and the updated force control model, making the surgical arm's motion state close to the set target motion state. Updating the force control model parameters should make the pushing force of the surgical arm in drag-and-teach mode similar to or consistent with the initial force adjusted by the engineer at the factory. This indicates that the force control parameter update is effective, and the result of this update can be used as the final update result, completing the force control parameter update process. If the update effect fails to pass verification, the above parameter update steps can be repeated based on the currently updated force control model to obtain another updated force control model, until the update effect of one of the updated force control models passes verification, ending the force control parameter update process.
[0047] The above process can be fully automated, requiring no manual adjustments from relevant experts. This greatly reduces the cost of updating force control parameter models, improves the efficiency of parameter updates, and makes the maintenance of surgical robots more convenient.
[0048] The technical solution of this embodiment involves driving the target surgical arm joint to move under the control of a preset load torque and the force control model to be updated in a force control parameter update mode, and collecting motion state parameters during the movement. These motion state parameters are then input into a pre-trained parameter update algorithm model to obtain a force control parameter update strategy. The parameters of the force control model to be updated are updated according to the force control parameter update strategy to obtain the updated force control model. The update effect of the updated force control model is then verified to complete the force control parameter update process. This embodiment of the invention solves the problem of high maintenance costs and low efficiency of surgical robot functions. It can automatically update the parameters of the force control model in the drag-and-drop teaching function through an algorithm, reducing the cost of function maintenance and improving the accuracy and convenience of surgical robot operation.
[0049] Figure 2 This is a flowchart illustrating a method for updating force control parameters of a surgical robot according to an embodiment of the present invention. This embodiment belongs to the same inventive concept as the force control parameter updating method for surgical robots in the above embodiments, and further describes the process of verifying the update effect of the updated force control model. This method can be executed by a force control parameter updating device for the surgical robot, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.
[0050] like Figure 2 As shown, the force control parameter update method for the surgical robot in this embodiment includes the following steps:
[0051] S210, In response to the trigger operation of the force control parameter update mode, enter the force control parameter update mode.
[0052] Once the force control parameter update function is triggered, the surgical robot can respond to the trigger operation of the force control parameter update mode and enter the force control parameter update mode.
[0053] S220. In the force control parameter update mode, the target surgical arm joint is driven to move under the control of the preset load torque and the force control model to be updated, and the motion state parameters during the movement are collected.
[0054] S230. Input the motion state parameters into the pre-trained parameter update algorithm model to obtain the force control parameter update strategy.
[0055] S240. Update the parameters of the force control model to be updated according to the force control parameter update strategy to obtain the updated force control model.
[0056] S250 drives the target surgical arm joint to perform reciprocating motion under the control of a preset load torque and an updated force control model, and collects reciprocating motion data.
[0057] The reciprocating motion can be a repeated motion of the target surgical arm joint according to a preset motion trajectory and a preset motion cycle. During the reciprocating motion, when the target surgical arm joint reaches its joint limit, the preset load torque is reversed so that the target surgical arm joint can complete the full reciprocating motion cycle.
[0058] Reciprocating motion data can be the position, velocity, acceleration, and other data of the target surgical arm joint during the movement under a preset load torque.
[0059] S260. Based on the reciprocating motion data and the corresponding preset reference motion data, the updated force control model is updated to verify the effect and the verification result is obtained.
[0060] Among them, the preset reference motion data can be the motion data under the ideal state under the preset load torque. It can be understood as the target of force control parameter update, so that the motion data of the target surgical arm joint under the control of the preset load torque and the updated force control model is as close as possible to the preset reference motion data.
[0061] In one optional implementation, the similarity between the reciprocating motion data and the preset reference motion data can be calculated to obtain a similarity value; and a verification result can be obtained based on the numerical relationship between the similarity value and the preset lower similarity threshold.
[0062] S270. When the verification result meets the preset conditions, the updated force control model is determined as the target force control model, and the force control parameter update process is completed.
[0063] The verification result meets the preset condition if the similarity between the reciprocating motion data and the preset reference motion data reaches the lower limit of the similarity threshold.
[0064] When the verification result does not meet the preset conditions, that is, the similarity between the reciprocating motion data and the preset reference motion data does not reach the lower limit of the similarity threshold, the force control parameter update method of the surgical robot in the above steps can be repeated until the updated force control model passes the corresponding update effect verification and the force control parameter update process is completed.
[0065] The technical solution of this embodiment enters the force control parameter update mode in response to the trigger operation of the force control parameter update mode. In the force control parameter update mode, the target surgical arm joint is driven to move under the control of a preset load torque and the force control model to be updated, and motion state parameters are collected during the movement. The motion state parameters are input into a pre-trained parameter update algorithm model to obtain a force control parameter update strategy. The parameters of the force control model to be updated are updated according to the force control parameter update strategy to obtain the updated force control model. The target surgical arm joint is driven to reciprocate under the control of a preset load torque and the updated force control model, and reciprocating motion data is collected. The update effect of the updated force control model is verified based on the reciprocating motion data and the corresponding preset reference motion data to obtain the verification result. When the verification result meets the preset conditions, the updated force control model is determined as the target force control model, and the force control parameter update process is completed. When the verification result does not meet the preset conditions, the force control parameter update method of the surgical robot in the above steps can be repeated until the updated force control model passes the corresponding update effect verification, and the force control parameter update process is completed. The technical solution of this invention solves the problems of high cost and low efficiency in the functional maintenance of surgical robots. It can automatically update the parameters of the force control model in the drag teaching function through an algorithm, and verify the update effect after each update to ensure that the parameter update is an effective update process, thereby reducing the cost of functional maintenance and improving the accuracy and convenience of surgical robot operation.
[0066] Figure 3 This is a flowchart illustrating a force control parameter update method for a surgical robot, provided as an embodiment of the present invention. This embodiment belongs to the same inventive concept as the force control parameter update method for surgical robots described in the previous embodiments, and further describes the training process of the parameter update algorithm model. This method can be executed by a force control parameter update device for the surgical robot, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.
[0067] like Figure 3 As shown, the force control parameter update method for the surgical robot in this embodiment includes the following steps:
[0068] S3010. Based on the dynamic model of the target surgical arm joint and the friction compensation model that matches the motion scenario of the target surgical arm joint, a reference force control model of the target surgical arm joint is constructed.
[0069] The dynamic model is derived from the physical structure of the surgical robot's arm, while the friction compensation model is designed based on the actual usage scenarios of the surgical robot. To ensure that the force control model matches the actual operation of the surgical robot as closely as possible, the dynamic model and the friction compensation model can be superimposed to obtain a reference force control model for the target surgical arm joint.
[0070] In one alternative implementation, the reference force control model can be represented as: Where τ is the joint torque and θ is the joint angle. For speed, Let M(θ) be the acceleration, M(θ) be the mass matrix of the surgical arm, and G(θ) be the gravity vector. τ represents the vector of centrifugal force and Coriolis force. fric The Coulomb viscous friction model can be used to calculate the frictional torque. This part corresponds to the dynamic model.
[0071] The initial values of each adjustable force control parameter in the reference force control model can be determined based on the motion data of the surgical arm, or they can be manually adjusted based on the experience of domain experts to obtain the optimal parameters to suit the current working conditions.
[0072] S3020: Obtain the first motion test data of the target surgical arm joint based on the reference force control model and the preset test load torque, and determine the motion optimization target based on the first motion test data.
[0073] The first motion test data refers to the motion test data collected by the target surgical arm joint under the control of a reference force control model with optimal parameters that conform to the current working conditions and a preset test load torque. The first motion test data may include relevant variables that can characterize its motion state, such as torque value, joint angle, and velocity.
[0074] In one optional implementation, the velocity change curve during reciprocating motion can be fitted based on the first motion test data to obtain the functional relationship between the position and velocity of the target surgical arm joint under a given preset test load torque, expressed as V. goal (θ) is used to characterize the optimization target state for subsequent force control parameter updates.
[0075] It's important to note that the preset test load torque can be a constant value or a dynamically changing value based on a preset linear or nonlinear function, used to collect more expressive motion state data. That is, the motion test data obtained under different preset test load torques will be different, and the position-velocity functional relationship fitted based on this motion test data will also be different. Different preset test load torques correspond to different target optimization states.
[0076] S3030: Drive the target surgical arm joint to move under the control of the preset test load torque and the force control model to be updated, and acquire the first motion state data during the motion process.
[0077] The first motion state data may include relevant variable data that can characterize its motion state, such as torque value, joint angle, and velocity.
[0078] In the first motion state data of the target surgical arm joint, the state at time t can be represented as S. t ={x1,x2,x3,…,x n}, where x i This represents the value of the i-th parameter at time t in the dynamic model, where n is the number of parameters.
[0079] Based on the preset motion state data acquisition time and sampling rate, motion state data at multiple moments can be obtained, which are then combined to obtain the first motion state data. The specific first motion state data can be displayed in the form of a matrix array.
[0080] S3040. Input the first motion state data into the parameter update algorithm model to be trained to obtain the force control parameter update strategy, and update the force control model to be updated according to the force control parameter update strategy to obtain the updated force control model.
[0081] S3050: Based on the updated force control model and the preset test load torque, the second motion test data of the target surgical arm joint is obtained, and the motion optimization result is determined based on the second motion test data.
[0082] The second motion test data refers to the motion test data collected on the target surgical arm joint under the control of a preset test load torque and an updated force control model. The second motion test data may include relevant variables that can characterize its motion state, such as torque values, joint angles, and velocities.
[0083] Based on the second motion test data, the velocity change curve during reciprocating motion can be fitted to obtain the functional relationship between the position and velocity of the target surgical arm joint under a given preset test load torque, expressed as V. action (θ) is used to characterize the motion state after the force control parameters are updated, i.e., the motion optimization result.
[0084] S3060. Compare the motion optimization results with the motion optimization target to obtain the comparison results, and update the parameters of the parameter update algorithm model to be trained based on the comparison results to obtain the updated parameter update algorithm model.
[0085] It can be a comparison of V action (θ) and V goal The similarity (θ) and the corresponding difference are used as comparison results, and then the parameters of the parameter update algorithm model to be trained are updated in reverse to obtain the updated parameter update algorithm model.
[0086] In one alternative implementation, the parameter update algorithm model to be trained can be a pre-built reinforcement learning system, such as a proximal policy optimization algorithm model. This proximal policy optimization algorithm model includes three elements: state, action, and reward value; where the state corresponds to the motion state data of the target surgical arm joint, denoted as S. t ={x1,x2,x3,…,x n}; The action corresponds to the force control parameter update strategy, defined as A = {a1, a2, a3, ..., a n}, a i This represents the adjustment amount for the i-th dynamic parameter. To avoid an excessively large action space that makes the agent difficult to converge, the adjustment range of the action can be defined such that |a| = 0. i |≤σ i , σ i is the maximum adjustment amount for the i-th parameter in a single operation; the reward value corresponds to the reward value determined based on the similarity between the motion optimization result and the motion optimization objective.
[0087] S3070. Verify the model training effect of the updated parameter update algorithm model until a parameter update algorithm model that meets the preset model training objective is obtained.
[0088] Verify the training effect of the updated parameter update algorithm model by calculating the reward value in the above steps.
[0089] In one alternative implementation, the step of determining the reward value can be achieved by calculating the Kullback-Leibler divergence between the motion optimization result and the motion optimization objective. This measure indicates the similarity between the current motion and the desired target motion state; a smaller Kullback-Leibler divergence indicates greater similarity between the two motion states. The calculation formula can be expressed as:
[0090] Then, the KL divergence values are mapped to the interval 0-1 to obtain the divergence mapping values. This step can be achieved using the sigmoid function. This function is expressed as sigmoid(x) = (1-e^(-1 / 2))^(-1 / 2 ... x ) -1 .
[0091] Furthermore, the difference between 1 and the divergence mapping value is calculated, and this difference is used as the reward value. The reward value can be expressed as: reward = 1 - sigmoid(KL(V) action (θ),V goal (θ)). The closer the reward value is to 1, the better the parameter update effect of the force control model.
[0092] In this step, the parameters of the parameter update algorithm model to be trained can be updated in reverse based on the magnitude of the reward value, such as using gradient update. This continues until a parameter update algorithm model is obtained where the reward value reaches the preset requirement after updating. Accordingly, the final parameter update algorithm model can be applied to the force control parameter update mode of surgical robots to efficiently achieve adaptive updates of the force control model parameters.
[0093] S3080. In the force control parameter update mode, the target surgical arm joint is driven to move under the control of the preset load torque and the force control model to be updated, and the motion state parameters during the movement are collected.
[0094] S3090. Input the motion state parameters into the parameter update algorithm model that has undergone the above pre-training process to obtain the force control parameter update strategy.
[0095] S3100. Update the parameters of the force control model to be updated according to the force control parameter update strategy to obtain the updated force control model.
[0096] S3110. Verify the update effect of the updated force control model to complete the force control parameter update process.
[0097] The technical solution of this embodiment constructs a reference force control model for the target surgical arm joint based on a dynamic model of the target surgical arm joint and a friction compensation model matching the motion scenario of the target surgical arm joint; obtains first motion test data of the target surgical arm joint based on the reference force control model and a preset test load torque, and determines the motion optimization target based on the first motion test data; drives the target surgical arm joint to move under the control of the preset test load torque and the force control model to be updated, and obtains first motion state data during the motion process; inputs the first motion state data into the parameter update algorithm model to be trained to obtain a force control parameter update strategy, and updates the force control model to be updated according to the force control parameter update strategy to obtain the updated force control model; obtains second motion test data of the target surgical arm joint based on the updated force control model and the preset test load torque, and determines the motion optimization target based on the second motion test data. The motion optimization results are determined by dynamic test data; the motion optimization results are compared with the motion optimization target to obtain the comparison results, and the parameters of the parameter update algorithm model to be trained are updated according to the comparison results to obtain the updated parameter update algorithm model; the model training effect of the updated parameter update algorithm model is verified until a parameter update algorithm model that meets the preset model training target is obtained; in the force control parameter update mode, the target surgical arm joint is driven to move under the control of the preset load torque and the force control model to be updated, and the motion state parameters during the movement are collected; the motion state parameters are input into the parameter update algorithm model that has undergone the above pre-training process to obtain the force control parameter update strategy; the parameters of the force control model to be updated are updated according to the force control parameter update strategy to obtain the updated force control model; the update effect of the updated force control model is verified to complete the force control parameter update process. The technical solution of this invention solves the problems of high cost and low efficiency in the functional maintenance of surgical robots. It can train an algorithm model for users to update force control parameters, and use the algorithm to automatically update the parameters of the force control model in the drag teaching function, and verify the update effect after each update to ensure that the parameter update is an effective update process, thereby reducing the cost of functional maintenance and improving the accuracy and convenience of surgical robot operation.
[0098] Figure 4 This is a schematic diagram of a force control parameter updating device for a surgical robot according to an embodiment of the present invention. This embodiment is applicable to scenarios involving updating the control parameters of a surgical robot, particularly in cases of adaptive updating of the control parameters. This force control parameter updating device for the surgical robot can be implemented in software and / or hardware and integrated into a computer terminal device with application development capabilities.
[0099] like Figure 4As shown, the force control parameter update device for the surgical robot includes: a motion state parameter acquisition module 410, a parameter update strategy determination module 420, a force control parameter update module 430, and a force control parameter update determination module 440.
[0100] The system includes a motion state parameter acquisition module 410, which drives the target surgical arm joint to move under the control of a preset load torque and the force control model to be updated in the force control parameter update mode, and collects motion state parameters during the movement; a parameter update strategy determination module 420, which inputs the motion state parameters into a pre-trained parameter update algorithm model to obtain a force control parameter update strategy; a force control parameter update module 430, which updates the parameters of the force control model to be updated according to the force control parameter update strategy to obtain an updated force control model; and a force control parameter update determination module 440, which verifies the update effect of the updated force control model to complete the force control parameter update process.
[0101] The technical solution of this embodiment involves driving the target surgical arm joint to move under the control of a preset load torque and the force control model to be updated in a force control parameter update mode, and collecting motion state parameters during the movement. These motion state parameters are then input into a pre-trained parameter update algorithm model to obtain a force control parameter update strategy. The parameters of the force control model to be updated are updated according to the force control parameter update strategy to obtain the updated force control model. The update effect of the updated force control model is then verified to complete the force control parameter update process. This embodiment of the invention solves the problem of high maintenance costs and low efficiency of surgical robot functions. It can automatically update the parameters of the force control model in the drag-and-drop teaching function through an algorithm, reducing the cost of function maintenance and improving the accuracy and convenience of surgical robot operation.
[0102] In one optional implementation, the force control parameter update determination module 440 is specifically used for:
[0103] The target surgical arm joint is driven to perform reciprocating motion under the control of a preset load torque and an updated force control model, and reciprocating motion data is collected.
[0104] The update effect of the updated force control model is verified based on the reciprocating motion data and the corresponding preset reference motion data, and the verification results are obtained.
[0105] When the verification result meets the preset conditions, the updated force control model is determined as the target force control model, and the force control parameter update process is completed.
[0106] When the verification result does not meet the preset conditions, the force control parameter update method of the surgical robot in the above embodiment is repeated until the updated force control model passes the corresponding update effect verification and the force control parameter update process is completed.
[0107] In an alternative implementation, the force control parameter update determination module 440 may be further specifically used for:
[0108] Calculate the similarity between the reciprocating motion data and the preset reference motion data to obtain a similarity value;
[0109] The verification result is obtained based on the numerical relationship between the similarity value and the preset lower similarity threshold.
[0110] In one optional implementation, the force control parameter update device of the surgical robot further includes a model training module for training a parameter update algorithm model. The specific training process includes:
[0111] Based on the dynamic model of the target surgical arm joint and the friction compensation model that matches the motion scenario of the target surgical arm joint, a reference force control model of the target surgical arm joint is constructed.
[0112] The first motion test data of the target surgical arm joint is obtained based on the reference force control model and the preset test load torque, and the motion optimization target is determined based on the first motion test data;
[0113] The target surgical arm joint is driven to move under the control of a preset test load torque and a force control model to be updated, and the first motion state data during the motion process is acquired.
[0114] The first motion state data is input into the parameter update algorithm model to be trained to obtain the force control parameter update strategy, and the force control model to be updated is updated according to the force control parameter update strategy to obtain the updated force control model.
[0115] The second motion test data of the target surgical arm joint is obtained based on the updated force control model and the preset test load torque, and the motion optimization result is determined based on the second motion test data;
[0116] The motion optimization results are compared with the motion optimization target to obtain the comparison results. Based on the comparison results, the parameters of the algorithm model to be trained are updated to obtain the updated parameter update algorithm model.
[0117] Verify the training effect of the updated parameter update algorithm model until a parameter update algorithm model that meets the preset model training objective is obtained.
[0118] In one optional implementation, the parameter update algorithm model to be trained includes a proximal policy optimization algorithm model, which includes three elements: state, action, and reward value.
[0119] Among them, the state corresponds to the motion state data of the target surgical arm joint, the action corresponds to the force control parameter update strategy, and the reward value corresponds to the reward value determined based on the similarity between the motion optimization result and the motion optimization target.
[0120] In one alternative implementation, the model training module may specifically be used for:
[0121] Calculate the KL divergence between the motion optimization results and the motion optimization objective;
[0122] The KL divergence values are mapped to the interval of 0-1 to obtain the divergence mapping values;
[0123] The difference between 1 and the divergence mapping value is calculated, and the difference is used as the reward value.
[0124] In one alternative implementation, the preset test load torque is a preset constant value, or a value that changes dynamically according to a preset linear or nonlinear function.
[0125] The force control parameter update device for surgical robots provided in this embodiment of the invention can execute the force control parameter update method for surgical robots provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0126] In one specific example, a reinforcement learning agent was trained to optimize the force control parameters for drag-and-drop teaching of a surgical robot. This example, by introducing a reinforcement learning algorithm, achieves automatic parameter optimization of the surgical robot in drag-and-drop teaching mode. It can adaptively adjust parameters according to the device's own operating conditions, effectively counteracting the time-varying effects of friction caused by the external environment. This results in good consistency of the surgical robot under different operating conditions, while effectively reducing development and maintenance costs. Drag-and-drop teaching of a surgical robot is an operational method. In this teaching mode, the operator can directly and manually drag the robot's robotic arm to the desired position, thereby recording and transmitting the motion trajectory. This method helps improve the accuracy and convenience of surgical robot operation.
[0127] Specifically, the adaptive force control optimization method consists of two stages: offline training and online parameter tuning. The flowchart for the offline training stage can be found in [reference needed]. Figure 5 A schematic diagram of an offline training process for a reinforcement learning agent used for parameter updates is provided.
[0128] The offline phase is mainly used for training reinforcement learning agents. The detailed implementation steps are as follows:
[0129] First, based on the physical structure of the surgical arm, the corresponding dynamic model is derived, and a friction compensation model is designed based on the actual scenario to establish a force control model.
[0130]
[0131] Where τ is the joint torque and θ is the joint angle. For speed, Let M(θ) be the acceleration, M(θ) be the mass matrix of the surgical arm, and G(θ) be the gravity vector. τ represents the vector of centrifugal force and Coriolis force. fric The Coulomb viscous friction model can be used to calculate the frictional torque.
[0132] After establishing the force-controlled model, it is necessary to adjust some of the dynamic parameters, such as the viscosity coefficient, Coulomb friction, and... The coefficients of relevant variables. Engineers can choose to collect motion data of the surgical arm to identify these parameters or manually adjust them based on the experience of domain experts to obtain the optimal parameters suitable for the current working conditions.
[0133] Switching the surgical arm to torque control mode adds an extra load torque to the force control model described above. Once the surgical arm reaches the joint limit, this torque is reversed to allow the arm to complete a full reciprocating cycle. Data collected during this process includes torque values, joint angles, velocity, and other relevant variables characterizing the motion state. Engineers can also extend this to use dynamically changing linear or non-linear load torques to collect more expressive motion state data, depending on the specific circumstances.
[0134] Fit the velocity variation curve of the data to obtain the functional relationship V between position and velocity under a given load. goal (θ) is used to characterize the target state of this motion.
[0135] The reinforcement learning agent can be constructed using the proximal policy optimization algorithm. The agent is initialized and trained offline. The three elements of reinforcement learning (state, action, and reward) can be defined as follows:
[0136] State: The state of the current surgical arm at time t can be represented as S. t ={x1,x2,x3,…,x n}, where x i This represents the current value of the i-th parameter in the dynamic model, where n is the number of parameters. Similarly, we can also add a variable to describe the system state update after the agent's action is executed, as a state representation.
[0137] Action: The action of a reinforcement learning agent is defined as A = {a1, a2, a3, ..., a...} n}, a i This represents the adjustment amount for the i-th dynamic parameter. To avoid an excessively large action space that makes the agent difficult to converge, the adjustment range of the action can be defined such that |a| = 0. i |≤σ i, σ i This represents the maximum adjustment amount of the i-th parameter in a single operation.
[0138] Reward: Update the dynamic model parameters of the surgical arm based on the parameter adjustments output by the agent, and superimpose a given load torque (the same as the load torque above) on top of the dynamic model to make it move spontaneously, collect motion data, and calculate the functional relationship V between position and velocity during this movement. action (θ), as shown in Formula 2, is obtained by calculating V. action (θ) and V goal The KL divergence of (θ) is used to measure the similarity between the current motion and the expected target motion state. The smaller the KL divergence, the more similar the two motion states are. Therefore, the state reward calculation can be constructed as shown in Formula 3. At the same time, the sigmoid function is used to shrink it to the range of (0,1) as shown in Formula 3. Finally, its form is transformed into a positive correlation state score that is closer to 1, as shown in Formula 4.
[0139]
[0140] sigmoid(x)=(1-e x ) -1 (3)
[0141] reward = 1 - sigmoid(KL(V) action (θ),V goal (θ))) (4)
[0142] For further details on the initialization and training process of reinforcement learning agents, please refer to [link / reference]. Figure 6 The diagram illustrates the training process of a reinforcement learning agent used for parameter updates. The specific process is as follows:
[0143] The agent reads the current state information of the surgical arm, such as the current values of the dynamic model parameters, and then calculates the parameter tuning strategy that maximizes the expected return.
[0144] The parameter tuning strategy is applied to update the current dynamic model parameters of the surgical arm.
[0145] In torque mode, an additional load torque is applied to make it reciprocate, and motion data is collected.
[0146] The reward value is calculated according to the definition above.
[0147] The reinforcement learning agent learns by being trained based on the reward value of the feedback, and then proceeds to the next round of decision-making, iterating multiple times until convergence.
[0148] When using the surgical arm drag teaching function, users can actively enable the online optimization function for dynamic parameters. Figure 7The diagram shown illustrates a force control parameter update based on a reinforcement learning agent. It is a flowchart of the algorithm during the online optimization phase, with detailed execution steps as follows:
[0149] First, a load torque is superimposed to drive the movement of the surgical arm, and the agent reads the surgical arm's state data to calculate the parameter tuning strategy.
[0150] Then, the parameters of the surgical arm dynamics model (force control model) are updated according to the parameter tuning strategy.
[0151] Then, the same load torque as in the offline phase is applied to induce reciprocating motion and collect data. A reward value is calculated based on this reciprocating motion data. It is then determined whether the current reward has reached a given threshold, which must be obtained in the offline phase and can be given by engineers based on experience. This threshold defines the minimum performance metric required to meet usage requirements.
[0152] If the reward of the current optimization strategy does not meet the threshold, the system returns to the first step based on the current state parameters to perform the next round of parameter tuning. This continues until the reward value meets the requirements or the number of optimization attempts exceeds a certain limit, at which point online optimization exits. The upper limit for the number of optimization attempts can be determined based on the training effect during the offline training phase; setting it to the highest number of optimization attempts in history is sufficient. The method in this example allows the device to maintain consistency in its drag-and-drop teaching function under different operating conditions, and enables the device to automatically adjust force control parameters after experiencing transportation or extreme external environmental influences, effectively reducing additional maintenance work for engineers.
[0153] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 8 A block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present invention is shown. Figure 8 The computer device 12 shown is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as intelligent controllers and servers, mobile phones, and other terminal devices.
[0154] like Figure 8 As shown, the computer device 12 is represented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).
[0155] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0156] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.
[0157] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 8 Not shown; usually referred to as a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.
[0158] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.
[0159] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although... Figure 8 As not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, R AID systems, tape drives, and data backup storage systems.
[0160] Processing unit 16 executes various functional applications and data processing by running programs stored in system memory 28, such as implementing the force control parameter update method for the surgical robot provided in this embodiment, the method including:
[0161] In the force control parameter update mode, the target surgical arm joint is driven to move under the control of a preset load torque and the force control model to be updated, and motion state parameters are collected during the movement process.
[0162] The motion state parameters are input into a pre-trained parameter update algorithm model to obtain the force control parameter update strategy;
[0163] The parameters of the force control model to be updated are updated according to the force control parameter update strategy to obtain the updated force control model;
[0164] The update effect of the updated force control model is verified to complete the force control parameter update process.
[0165] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a force control parameter update method for a surgical robot as provided in any embodiment of this invention. The method includes:
[0166] In the force control parameter update mode, the target surgical arm joint is driven to move under the control of a preset load torque and the force control model to be updated, and motion state parameters are collected during the movement process.
[0167] The motion state parameters are input into a pre-trained parameter update algorithm model to obtain the force control parameter update strategy;
[0168] The parameters of the force control model to be updated are updated according to the force control parameter update strategy to obtain the updated force control model;
[0169] The update effect of the updated force control model is verified to complete the force control parameter update process.
[0170] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0171] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.
[0172] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0173] Computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages such as Java, Smalltalk, Python, and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0174] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the force control parameter update method for a surgical robot as provided in any embodiment of this application.
[0175] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages such as Java, Smalltalk, Python, and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0176] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.
[0177] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.
Claims
1. A method for updating force control parameters of a surgical robot, characterized in that, include: In the force control parameter update mode, the target surgical arm joint is driven to move under the control of a preset load torque and the force control model to be updated, and motion state parameters are collected during the movement process. The motion state parameters are input into a pre-trained parameter update algorithm model to obtain a force control parameter update strategy. The parameters of the force control model to be updated are updated according to the force control parameter update strategy to obtain the updated force control model; The update effect of the updated force control model is verified to complete the force control parameter update process; The training process of the parameter update algorithm model includes: Based on the dynamic model of the target surgical arm joint and the friction compensation model that matches the motion scenario of the target surgical arm joint, a reference force control model of the target surgical arm joint is constructed. The first motion test data of the target surgical arm joint is obtained based on the reference force control model and the preset test load torque, and the motion optimization target is determined based on the first motion test data; The target surgical arm joint is driven to move under the control of the preset test load torque and the force control model to be updated, and the first motion state data during the motion process is acquired. The first motion state data is input into the parameter update algorithm model to be trained to obtain the force control parameter update strategy, and the force control model to be updated is updated according to the force control parameter update strategy to obtain the updated force control model. The second motion test data of the target surgical arm joint is obtained based on the updated force control model and the preset test load torque, and the motion optimization result is determined based on the second motion test data; The motion optimization result is compared with the motion optimization target to obtain the comparison result, and the parameters of the parameter update algorithm model to be trained are updated according to the comparison result to obtain the updated parameter update algorithm model. Verify the model training effect of the updated parameter update algorithm model until a parameter update algorithm model that meets the preset model training objective is obtained.
2. The method according to claim 1, characterized in that, The verification of the updated force control model's effect to complete the force control parameter update process includes: The target surgical arm joint is driven to reciprocate under the control of the preset load torque and the updated force control model, and reciprocating motion data is collected. The updated force control model is updated based on the reciprocating motion data and the corresponding preset reference motion data to verify the update effect and obtain the verification result. When the verification result meets the preset conditions, the updated force control model is determined to be the target force control model, and the force control parameter update process is completed. When the verification result does not meet the preset conditions, the force control parameter update method of the surgical robot as described in claim 1 is repeated until the updated force control model passes the corresponding update effect verification, thus completing the force control parameter update process.
3. The method according to claim 2, characterized in that, The step of verifying the update effect of the updated force control model based on the reciprocating motion data and the corresponding preset reference motion data, and obtaining the verification result, includes: Calculate the similarity between the reciprocating motion data and the preset reference motion data to obtain a similarity value; The verification result is obtained based on the numerical relationship between the similarity value and the preset similarity lower limit threshold.
4. The method according to claim 1, characterized in that, The parameter update algorithm model to be trained includes a proximal policy optimization algorithm model, which includes three elements: state, action, and reward value. Wherein, the state corresponds to the motion state data of the target surgical arm joint, the action corresponds to the force control parameter update strategy, and the reward value corresponds to the reward value determined based on the similarity between the motion optimization result and the motion optimization target.
5. The method according to claim 4, characterized in that, The step of determining the reward value includes: calculating the KL divergence between the motion optimization result and the motion optimization objective; The KL divergence values are mapped to the range of 0-1 to obtain the divergence mapping values. The difference between 1 and the divergence mapping value is calculated, and the difference is used as the reward value.
6. The method according to claim 1, characterized in that, The preset test load torque is a preset constant value, or a value that changes dynamically according to a preset linear or nonlinear function.
7. A force control parameter update device for a surgical robot, characterized in that, include: The motion state parameter acquisition module is used to drive the target surgical arm joint to move under the control of a preset load torque and the force control model to be updated in the force control parameter update mode, and to acquire motion state parameters during the movement process. The parameter update strategy determination module is used to input the motion state parameters into a pre-trained parameter update algorithm model to obtain the force control parameter update strategy; The force control parameter update module is used to update the parameters of the force control model to be updated according to the force control parameter update strategy, so as to obtain the updated force control model. The force control parameter update determination module is used to verify the update effect of the updated force control model in order to complete the force control parameter update process. The model training module is used to train the parameter update algorithm model. The specific training process includes: Based on the dynamic model of the target surgical arm joint and the friction compensation model that matches the motion scenario of the target surgical arm joint, a reference force control model of the target surgical arm joint is constructed. The first motion test data of the target surgical arm joint is obtained based on the reference force control model and the preset test load torque, and the motion optimization target is determined based on the first motion test data; The target surgical arm joint is driven to move under the control of a preset test load torque and a force control model to be updated, and the first motion state data during the motion process is acquired. The first motion state data is input into the parameter update algorithm model to be trained to obtain the force control parameter update strategy, and the force control model to be updated is updated according to the force control parameter update strategy to obtain the updated force control model. The second motion test data of the target surgical arm joint is obtained based on the updated force control model and the preset test load torque, and the motion optimization result is determined based on the second motion test data; The motion optimization results are compared with the motion optimization target to obtain the comparison results. Based on the comparison results, the parameters of the algorithm model to be trained are updated to obtain the updated parameter update algorithm model. Verify the training effect of the updated parameter update algorithm model until a parameter update algorithm model that meets the preset model training objective is obtained.
8. A computer device, characterized in that, The computer device includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the force control parameter update method for the surgical robot as described in any one of claims 1-6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the force control parameter update method for the surgical robot as described in any one of claims 1-6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the force control parameter update method for the surgical robot as described in any one of claims 1-6.
Citation Information
Patent Citations
Mechanical arm control method and system based on multi-source information deep reinforcement learning
CN114789443A
Graph debugging-free method and device for torque ring of servo system
CN116184809A