Method and device for motion planning of a robot arm and computer equipment

Abstract

Embodiments of the present application are applicable to the field of orthopedic surgery robots and computer-aided medical technology, and provide a motion planning method and device for a mechanical arm and a computer device. The method comprises: constructing a three-dimensional simulation environment of a surgery scene, the three-dimensional simulation environment comprising objects for simulating a mechanical arm and multiple human objects; determining a current surgery indication and configuring a state space for the surgery indication, the state space comprising multiple parameters matched with the surgery indication; training the three-dimensional simulation environment based on the state space to obtain a motion planning model matched with the surgery indication; and planning the motion of an intraoperative mechanical arm using the motion planning model. The above method can automatically plan the arm angle of a seven-axis mechanical arm, ensure that the arm body of the mechanical arm is always in a reasonable position during the surgery process, and reduce the interference caused by the adjustment of the pose of the mechanical arm on the operation of the doctor.

Description

Technical Field

[0001] The embodiments of the present application belong to the field of orthopedic surgical robots and computer-assisted medical technology, and in particular, relate to a motion planning method, device, and computer equipment for a robotic arm. Background Art

[0002] In the field of orthopedic surgical robotics, robotic arms are widely used for surgical assistance, precise positioning, and manipulation of surgical tools. Unlike traditional six-axis industrial robotic arms, medical orthopedic surgical robots typically use redundant robotic arms with seven or more axes of freedom to achieve greater flexibility and operational precision. However, this redundancy brings the following problems:

[0003] 1. The robotic arm interferes with the surgeon's operation: During orthopedic surgery, the robotic arm must precisely position and operate within a confined space. Because the seven-axis robotic arm has redundant degrees of freedom, multiple joint solutions may exist for the same end-position. This can cause the arm to be within the surgeon's field of view or range of motion, obstructing the surgeon's field of view or limiting the operating space for surgical tools, impacting surgical efficiency and safety.

[0004] 2. Issues with path feasibility and surgical safety: Medical surgical environments place extremely high demands on the safety of robotic arm paths. Traditional inverse kinematics methods only focus on whether the robotic arm's end reaches the target position and fail to control the arm's position, which can easily lead to collisions between the robotic arm and the surgeon or surgical instruments. Existing methods for planning robotic arm paths lack optimization for the arm's posture, potentially placing the robotic arm in an unstable or unreasonable position during surgery, increasing surgical risk.

[0005] 3. Unsmooth path planning affects surgical fluency: During orthopedic surgery, the robotic arm must frequently adjust its position to coordinate with the surgeon. Traditional path planning can result in drastic changes in the robotic arm's position or path jitter, impacting surgical fluency.

[0006] It can be seen that there is currently a lack of methods in the field of orthopedic surgical robots that can automatically plan the arm angles of seven-axis robotic arms to ensure that the arm is in a reasonable position, thereby reducing interference with the doctor's operation and improving the safety and smoothness of the operation. Summary of the Invention

[0007] In view of this, the embodiments of the present application provide a motion planning method, device and computer equipment for a robotic arm, which are used to automatically plan the arm angle of a seven-axis robotic arm, ensure that the robotic arm body is always in a reasonable position during the operation, reduce the interference to the doctor's operation caused by the adjustment of the robotic arm posture, and help improve the safety and smoothness of the operation.

[0008] A first aspect of an embodiment of the present application provides a motion planning method for a robotic arm, comprising:

[0009] Constructing a three-dimensional simulation environment of a surgical scene, wherein the three-dimensional simulation environment includes objects for simulating a robotic arm and a plurality of human objects;

[0010] Determining a current surgical indication and configuring a state space for the surgical indication, wherein the state space includes a plurality of parameters matching the surgical indication;

[0011] Training the three-dimensional simulation environment based on the state space to obtain a motion planning model that matches the surgical indication;

[0012] The motion planning model is used to plan the motion of the robotic arm during surgery.

[0013] A second aspect of an embodiment of the present application provides a motion planning device for a robotic arm, comprising:

[0014] An environment construction module, configured to construct a three-dimensional simulation environment of a surgical scene, wherein the three-dimensional simulation environment includes objects for simulating a robotic arm and a plurality of human objects;

[0015] A state configuration module, configured to determine a current surgical indication and configure a state space for the surgical indication, wherein the state space includes a plurality of parameters matching the surgical indication;

[0016] A model training module, configured to train the three-dimensional simulation environment based on the state space to obtain a motion planning model that matches the surgical indication;

[0017] The motion planning module is used to plan the motion of the robotic arm during surgery using the motion planning model.

[0018] A third aspect of an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the computer device implements the method described in the first aspect above.

[0019] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method described in the first aspect above is implemented.

[0020] A fifth aspect of the embodiments of the present application provides a computer program product, including a computer program, which, when executed, enables the method described in the first aspect to be executed.

[0021] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0022] In this embodiment of the present application, by constructing a 3D simulation environment for the surgical scenario, after determining the current surgical indication and configuring a state space for that surgical indication, the 3D simulation environment can be trained based on the configured state space to obtain a motion planning model that matches the surgical indication. During the actual intraoperative scenario, this motion planning model can be used to plan the motion of the robotic arm, allowing it to move according to the planned path, ensuring the safety and smoothness of the surgery.

[0023] By applying the method provided in the embodiment of the present application to perform motion planning of the robotic arm, it is possible to plan the arm angle of the robotic arm, and then plan the arm posture of the robotic arm by integrating environmental factors. Compared with the incomplete consideration of environmental factors in traditional motion planning, the planning of the arm posture of the robotic arm is also incomplete. The embodiment of the present application adopts a reinforcement learning algorithm to comprehensively consider various environmental factors, which can systematically plan the arm posture, and is more conducive to improving the rationality of the planning results.

[0024] Secondly, the application of this method can significantly improve the safety of surgery. The embodiment of this application still uses the traditional motion planning interpolation method for planning the end position of the robotic arm, and the planned path has extremely high accuracy and safety. While taking into account the safety of the surgery, the arm angle planning is optimized through the reinforcement learning algorithm, which further effectively avoids the collision between the robotic arm and the doctor or surgical instruments, significantly improving the safety of the surgery.

[0025] Third, the application of this method can also improve the smoothness of the operation. The embodiment of the present application introduces an incentive mechanism to ensure the smoothness of joint movement, reduce the possibility of drastic changes in the robot arm posture or path jitter, and improve the smoothness of the surgical operation.

[0026] Fourth, the method provided in the embodiments of the present application for motion planning of the robotic arm can improve data utilization. The reinforcement learning algorithm can reuse the same batch of samples multiple times, significantly improving data utilization, accelerating model convergence, and reducing training time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0028] Figure 1This is a logic block diagram of a motion planning method for a robotic arm provided in an embodiment of the present application;

[0029] Figure 2 is a schematic diagram of a motion planning method for a robotic arm provided in an embodiment of the present application;

[0030] Figure 3 2 is a schematic diagram of a possible implementation of S203 in a motion planning method for a robotic arm provided in an embodiment of the present application;

[0031] Figure 4 is a schematic diagram of a motion planning device for a robotic arm provided in an embodiment of the present application;

[0032] Figure 5 This is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present application with unnecessary details.

[0034] An important indicator for measuring the performance of a robotic arm is the number of degrees of freedom (DoF), which is commonly referred to as the number of axes of the robotic arm. For a robotic arm with seven axes or more degrees of freedom, it is extremely difficult to calculate its inverse solution and perform motion planning. Chinese patent application No. 202311300415.6 discloses a motion planning method, device and robotic arm for a seven-axis robotic arm, and gives a detailed method for obtaining an analytical solution for a seven-axis robotic arm. In summary, the method disclosed in this patent is used to solve the analytical solution of the robotic arm during motion. As long as the end pose of the robotic arm at a certain moment and the arm angle are determined, and joint configuration coefficients , we can solve the value of the robot arm joint axis at that moment. However, in motion path planning, for the arm angle and joint configuration coefficients Planning is challenging because these two parameters primarily relate to arm posture, which is related not only to the robot's own state but also to the surrounding environment. For medical assistance scenarios, the primary purpose of arm posture design is to minimize the impact of the robot's body on the surgeon's surgical operation and field of view. However, different surgical indications have different requirements for arm posture, making it difficult to use the same standards for motion path planning.

[0035] To address the above issues, this application provides a motion planning method, device, and computer equipment for a robotic arm. This method uses a reinforcement learning algorithm called proximal policy optimization (PPO) to plan the motion of the robotic arm. While taking into account surgical safety, it comprehensively considers various environmental factors and systematically plans the arm posture to ensure the smoothness of joint movement, reduce the violent jitter caused by changes in the robotic arm posture, and improve the fluency of surgical operations. Furthermore, the motion planning method implemented based on the PPO algorithm can reuse the same batch of samples multiple times, significantly improving data utilization, accelerating model convergence, and reducing training time.

[0036] Specifically, if Figure 1 The figure is a logic block diagram of a motion planning method for a robotic arm provided by an embodiment of the present application. Figure 1 In the process shown, a computer device can construct an initial model and obtain an initial planning strategy. Using this initial motion planning model, the robot arm's motion path can be planned, resulting in the corresponding end-point pose. Based on this end-point pose, the initial model can be trained over multiple rounds using a reinforcement learning algorithm. During this process, the arm angles and joint configuration coefficients output from each training round can be combined with the end-point pose to obtain an inverse solution based on this end-point pose using an inverse kinematics algorithm. Based on the inverse solution, the arm angles and joint configuration coefficients can be further adjusted before the next round of model training. Furthermore, environmental factors can be considered by incorporating them into the intraoperative scenario of the robot arm. For example, environmental factors such as the external torque of the robot arm can be incorporated into the reinforcement learning process. Ultimately, through multiple rounds of training, a motion planning model tailored to the surgical indication is obtained. This model can be used to plan the robot arm's motion path in actual surgical scenarios, ensuring the safety and smoothness of the procedure.

[0037] The technical solution of this application is described below through specific embodiments.

[0038] Reference Figure 2 , which shows a schematic diagram of a motion planning method for a robotic arm provided in an embodiment of the present application, which may specifically include the following steps:

[0039] S201 : Construct a three-dimensional simulation environment of a surgical scene, wherein the three-dimensional simulation environment includes objects for simulating a robotic arm and multiple human objects.

[0040] It should be noted that the embodiments of the present application can be applied to computer devices, that is, the execution subject of the present method is a computer device, which is a device with data processing and other related functions. For example, a desktop computer, a cloud server, and other devices. In the actual field of computer-assisted medical technology, by implementing the relevant functions of the method provided in the embodiments of the present application, the computer device can plan the motion path of a seven-axis robotic arm with redundant degrees of freedom, such as a robotic arm, to meet the needs of actual surgical scenarios.

[0041] Before implementing the method provided in the embodiment of the present application, a three-dimensional simulation environment of the surgical scene can be constructed through modeling. The three-dimensional simulation environment may include objects for simulating robotic arms and multiple human objects. These human objects may include doctors, nurses, patients, etc. in the surgical scene.

[0042] Specifically, a robotic arm, a doctor, a nurse, and a patient can be constructed in a three-dimensional simulation environment through modeling for subsequent model training. For ease of explanation, unless otherwise specified in the subsequent introduction, the robotic arm and human objects mentioned in the model training process in the embodiments of this application refer to the robotic arm and human objects in the three-dimensional simulation environment.

[0043] As an example of an embodiment of the present application, the data required to construct a three-dimensional simulation environment may include:

[0044] 1) Data related to the robotic arm:

[0045] a. DH parameters of the manipulator, including link length (a), link torsion (α), link offset (d), and joint rotation (θ);

[0046] b. The motion range limit parameters of each joint of the robotic arm;

[0047] c. Dynamic parameters of the robotic arm (mass, inertia, etc.);

[0048] d. Information about the location of the robot arm's marker;

[0049] 2) Data related to surgical scenarios:

[0050] a. Standard standing information for doctors and nurses;

[0051] b. The patient's location;

[0052] c. The location and size data of the operating table;

[0053] d. Location information of surgical instruments.

[0054] In an embodiment of the present application, different three-dimensional simulation environments can be constructed for different surgical indications; alternatively, surgical indications with certain similarities can use the same three-dimensional simulation environment, or a universal three-dimensional simulation environment can be pre-constructed, and then the universal three-dimensional simulation environment can be adjusted according to different surgical indications to obtain a three-dimensional simulation environment that matches the specific surgical indication, thereby speeding up the construction efficiency of the three-dimensional simulation environment.

[0055] In one possible implementation of the embodiment of the present application, markers may be set on a robotic arm and multiple human objects in a three-dimensional simulation environment, and each marker may be set at a specific marker point of the corresponding object. The marker point may be the location of the marker.

[0056] For example, multiple markers can be set on the robot arm, and these markers can be set with the same color. For example, each marker on the robot arm can be set with a blue marker. The setting of markers on the robot arm should follow the following principles:

[0057] 1) Markers must be set at key joint positions:

[0058] a. Shoulder joint Ps location;

[0059] b. Elbow joint Pe location;

[0060] c.Wrist joint Pw location;

[0061] 2) Set markers at key locations on the robotic arm:

[0062] a. The midpoint between the upper and lower arms of the robot;

[0063] b. The protruding parts of the outer contour of the robotic arm;

[0064] c. Parts where collision may occur during the movement of the robotic arm.

[0065] When placing markers on a robotic arm, the locations of each marker must meet the corresponding distribution requirements. For example, the marker locations should be evenly distributed on the robotic arm surface; adjacent markers should maintain an appropriate distance, such as a distance greater than 5 centimeters (cm) between two adjacent markers; and the placement of markers should not affect the normal movement of the robotic arm.

[0066] On the other hand, markers can be set on human objects. The markers set on each human object can be set with the same color as the markers on the robot arm. For example, the markers on all human objects can be set with red markers to distinguish them from the blue markers on the robot arm. The setting of markers on each human object should follow the following principles:

[0067] 1) Place markers on key locations on the doctor:

[0068] a. Head position (avoid obstruction of vision);

[0069] b. Shoulder position (indicating operating space);

[0070] c. Hand position (indicating the operating area);

[0071] d. Surgical tool holding position;

[0072] 2) Place markers on key locations on the nurse:

[0073] a.Nurse standing position;

[0074] b. The nurse's hand position (indicating the operating area);

[0075] 3) Place markers at key locations on the patient:

[0076] a. Key points around the surgical site;

[0077] b. Parts of the patient's body that may affect the movement of the robotic arm.

[0078] Similarly, when placing markers on various human subjects, the locations of each marker should also meet certain distribution requirements. For example, the markers should cover the key activity areas of the subject; the markers should reflect the subject's real-time position and posture; and the markers should not affect the subject's normal activities.

[0079] In addition, markers are set on the robotic arm and each human object, and the selection of each marker point should also meet considerations such as the effectiveness, real-time performance and safety of collision detection.

[0080] For example, in terms of effectiveness, the selection of markers should accurately reflect the actual distance between objects, the distribution of each marker should cover all possible collision areas, and the number of markers should be moderate. Too many markers will increase the computational burden, and too few markers may also miss collision detection. In terms of real-time performance, the selection of markers should be easy for the visual system to identify and track, should not be blocked by other objects, and the markers should also have sufficient contrast to distinguish them from non-markers. In terms of safety, the material of the markers should be safe and harmless, the setting of markers should not affect surgical operations, and they should also be easy to disinfect and clean.

[0081] In one possible implementation of the present invention, the positions of the markers on the robotic arm and each person object can be optimized. For example, the positions of the markers can be optimized based on historical data analysis, simulation verification results, or actual test results.

[0082] For example, based on historical data analysis, collision data from surgical procedures can be collected to analyze the frequency and location of collisions, and the position of markers can be adjusted based on the analysis results. Another example is that based on simulation verification results, different marker points can be tested in a 3D simulation environment to evaluate the accuracy and efficiency of collision detection, and then the optimal marker point configuration scheme can be selected. Another example is that based on actual test results, tests can be conducted in a real surgical environment, feedback from doctors and nurses can be collected, and the marker settings can be optimized based on this feedback.

[0083] By placing markers on the robotic arm and the human object, the distance between the robotic arm and each human object can be calculated based on the markers on the robotic arm and each human object. This distance value can include the distance between each marker calculated after each movement of the robotic arm or human object. In other words, in actual use, when the robotic arm or human object moves, the computer device can calculate the distance between the robotic arm and the human object in real time based on the position of the markers.

[0084] Specifically, when calculating the distance value, the coordinate values ​​of any marker on the robot arm and any marker on each human object can be determined separately. For example, assuming that the marker on the robot arm is a blue marker point, the coordinate set composed of these blue marker points is , the markers on each character object are red markers, and the coordinate set composed of these red markers is .in, and respectively. In this way, the distance value between any marker on the robot arm and any marker on each of the person objects can be calculated based on the coordinate values, i.e. the distance between each of the blue markers and each of the red markers in the above example can be calculated. Taking the Euclidean distance as an example, the Euclidean distance between the i-th blue marker and the j-th red marker can be expressed as .

[0085] After all the distance values are calculated, the minimum value in the distance values can be found as the minimum collision distance between the markers .

[0086] S202, determine the current surgical indication and configure a state space for the surgical indication, the state space including a plurality of parameters matching the surgical indication.

[0087] Different surgical indications can be used to represent different types of surgeries, such as hip replacement surgery, ankle replacement surgery, etc. For different types of surgeries, the required surgical scene is also different, and the three-dimensional simulation environment constructed accordingly is also different.

[0088] Determining the current surgical indication can be performed before constructing the three-dimensional simulation model of the surgical scene, or after constructing the three-dimensional simulation model of the surgical scene. If the surgical indication is determined before constructing the three-dimensional simulation model of the surgical scene, a matching three-dimensional simulation environment can be constructed based on different surgical indications, so that the three-dimensional simulation environment constructed is as close as possible to the surgical scene corresponding to the surgical indication. Alternatively, after the three-dimensional simulation environment of the surgical scene has been constructed, the three-dimensional simulation environment constructed can be adjusted in a targeted manner by determining the current surgical indication, so that the adjusted three-dimensional simulation environment is as close as possible to the surgical scene corresponding to the actual surgical indication.

[0089] After the foregoing three-dimensional simulation environment is constructed, a state space can be configured for the current surgical indication, the state space including a plurality of parameters matching the surgical indication.

[0090] Exemplarily, the state space can include the following parameters:

[0091] joint angle of the robot arm ;

[0092] end pose of the robot arm (i.e. position + attitude of the end of the robot arm) ;

[0093] arm angle of the robot arm ;

[0094] Joint configuration coefficient of a robot arm ;

[0095] External torque of a joint of a robot arm ;

[0096] Minimum collision distance between a robot arm and a character object ;

[0097] Surgical indication type .

[0098] In addition, an action space of the robot arm can be defined, which can include parameters such as a change amount of an arm angle of the robot arm .

[0099] In the embodiments of the present application, the parameters in the state space should be in an initial state before the start of each training round of the model, that is, the three-dimensional simulation environment needs to be restored to the initial state at the start of each training round. For example, the state of the robot arm is initialized, such as joint angle , end pose , and arm angle , etc., to ensure that the robot arm is in a safe position. At the same time, the positions of the doctor, nurse, and patient character objects are initialized to ensure that the positions of the character objects meet the specific requirements of the corresponding surgical scene of the corresponding surgical indication. In addition, the positions of the markers on the robot arm and the character objects also need to be initialized. The parameters in the initialized state space can be recorded by a computer.

[0100] S203, training the three-dimensional simulation environment based on the state space to obtain a motion planning model matching the surgical indication.

[0101] After the three-dimensional simulation environment corresponding to the current surgical indication is constructed, the three-dimensional simulation environment can be trained by initializing the parameters in the state space to obtain a motion planning model matching the surgical indication.

[0102] In the embodiments of the present application, the training process can include two parts of environment initialization and policy learning. In the environment initialization, the three-dimensional simulation environment can be initialized according to the surgical indication type; the initial positions of the robot arm, the doctor, the nurse, and the patient are set; the marker positions on the robot arm and the character objects are initialized; and the training parameters (maximum number of steps, incentive coefficient, etc.) are set.

[0103] In the policy learning process, the PPO algorithm can be used for training, and each training round can include the following steps:

[0104] a. Obtain the current state (joint angle of the robot arm, end pose, arm angle, etc.);

[0105] b. Select an action (arm angle change) based on the current strategy;

[0106] c. Execute the action and calculate the incentive value;

[0107] d. Update the strategy network and value network;

[0108] e. Determine whether the termination condition is met.

[0109] Based on the above training process, Figure 3 FIG. 2 shows a possible implementation method of training a three-dimensional simulation environment based on the state space in the above S203 to obtain a motion planning model that matches the surgical indication. The process may specifically include the following steps S2031-S2033:

[0110] S2031. Calculate an arm angle variation of the robotic arm according to a first state corresponding to the state space, where the arm angle variation is within a feasible arm angle region of the robotic arm.

[0111] In the embodiment of the present application, the first state corresponding to the state space may refer to the initial state of the state space. Before each training round, the computer device needs to initialize the three-dimensional simulation environment so that the robotic arm and each human object represented by each parameter in the state space are in the corresponding state.

[0112] After initializing the state space to the first state, the computer device can calculate the arm angle change of the robotic arm in this state and ensure that the arm angle change is within the feasible arm angle region of the robotic arm.

[0113] Taking the seven-axis robotic arm as an example, the feasible arm angle range of the seven-axis robotic arm is Ψ range，all It can be determined by the following formula:

[0114]

[0115] in, Ψ range，1 ~Ψ range，7 It is the feasible arm angle set area of ​​the end pose pose, GC2, GC6 when the limit angle of each axis of the robot arm is determined. This set area can be obtained by analyzing the limit arm angle corresponding to the axis limit angle position of each joint axis. Ψ After determining the feasible arm angle set area for each joint axis, the feasible arm angle area applicable to each joint axis can be obtained by taking the intersection. Ψ range，all In the feasible arm angle area Ψ range，allwithin the range, take any arm angle Ψ The situation that a joint axis exceeds its limit will not occur.

[0116] S2032, based on the arm angle change, the inverse kinematics formula is used to solve the update position of the mechanical arm.

[0117] After determining the arm angle change, the computer device can update the mechanical arm, and use the inverse kinematics (IK) formula to solve the new joint angle, which can be used to represent the position to which the mechanical arm needs to move subsequently, that is, the update position.

[0118] As for how to calculate the arm angle change of the mechanical arm and how to use the inverse kinematics formula to solve the new joint angle of the mechanical arm according to the arm angle change, please refer to the introduction in the Chinese patent application with the application number 202311300415.6, which will not be repeated here. The joint angle solution obtained by the above solving process is an analytical solution.

[0119] S2033, moving the mechanical arm to the update position and obtaining a second state corresponding to the state space after moving, which will be used as the first state to recalculate the arm angle change of the mechanical arm in the next training until the training termination condition is met, obtaining the motion planning model matching the surgical indication.

[0120] After calculating the update position of the mechanical arm, the computer device can control the mechanical arm to move to the update position. In the update position, the joint angles of each joint of the mechanical arm are the same as the angles obtained by using the inverse kinematics formula.

[0121] Then, the computer device can obtain the second state corresponding to the state space when the mechanical arm is in the update angle, that is, obtain the information of each parameter in the state space at the update position, and continue the training.

[0122] In the process of continuing the training, the computer device can recalculate the arm angle change of the mechanical arm based on the second state at this time. In this step, the second state can be taken as the first state, and the new arm angle change can be calculated in the same way as the foregoing S2031, and then the inverse kinematics formula is used to solve the new joint angle of the mechanical arm based on the new arm angle change, and the mechanical arm is moved to the update position corresponding to the new joint angle. By repeatedly executing the steps of S2031-S2033, the training of the current round can be ended when the training termination condition is met, and the motion planning model matching the current surgical indication is obtained.

[0123] In a possible implementation of an embodiment of the present application, the above-mentioned second state includes the distance value between the robotic arm and each human object. After obtaining the second state corresponding to the state space after movement, the computer device can also determine whether the robotic arm collides with the human object based on the distance value. Since the robotic arm and the human object both include multiple markers, the calculated distance value also includes multiple ones. In the case where any distance value indicates that the robotic arm collides with the human object, the current update position needs to be adjusted, and the adjusted update position should meet the condition that the robotic arm and the human object will not collide. After adjusting the update position, the computer device can move the robotic arm to the adjusted update position.

[0124] In another possible implementation of the present embodiment, the computer device may also incorporate an incentive mechanism into model training, incentivizing each movement of the robotic arm. Specifically, after moving the robotic arm to the updated position, the computer device may determine the incentive value generated by this movement. This incentive value may include a positive incentive value and / or a negative incentive value. In this way, the computer device can determine the rationality and safety of this movement based on the incentive value.

[0125] In the embodiment of the present application, the incentive value may be determined based on one or more of the following:

[0126] 1) The positional relationship between the arm angle and the feasible arm angle area;

[0127] 2) The distance between the robot arm and the human object;

[0128] 3) the smoothness of the joint motion of the robotic arm; and,

[0129] 4) Calculate the number of times the robot arm updates its position;

[0130] 5) the magnitude of the external force; and,

[0131] 6) Potential energy state of the robotic arm.

[0132] The following describes each of the above factors.

[0133] The positional relationship between the arm angle and the feasible arm angle range can be used to indicate whether the arm angle is within the feasible arm angle range. In practical applications, the arm angle of the robotic arm should always be within the feasible arm angle range. Once the arm angle exceeds the feasible arm angle range, it will have an adverse impact on surgical safety. Therefore, if the arm angle exceeds the feasible arm angle range, the computer device can negatively excite the movement, generating a corresponding negative excitation value; if the arm angle is within the feasible arm angle range, the computer device can positively excite the movement, generating a corresponding positive excitation value.

[0134] Specifically, if the arm angle exceeds the feasible arm angle range, the corresponding incentive value generated can be the maximum negative incentive value, for example, -1000. If the arm angle is within the feasible arm angle range, the corresponding incentive value generated can be a positive incentive value. The magnitude of the positive incentive value is negatively correlated with the distance between the arm angle at that time and the axis of the feasible arm angle range. In other words, the closer the arm angle is to the center axis of the feasible arm angle range, the greater the positive incentive value.

[0135] As an example of an embodiment of the present application, when the arm angle is within the feasible arm angle region, the corresponding excitation mode can be expressed as:

[0136]

[0137] in, is the current arm angle value, is the standard deviation.

[0138] The distance between the robotic arm and a human object can be used to indicate whether the robotic arm has collided with the human object during movement. In practical applications, the robotic arm should avoid colliding with any human object. If the arm collides with other objects, it will affect surgical safety. Therefore, if the robotic arm collides with a human object, the computer device can apply negative incentives to the movement, generating a corresponding negative incentive value; if no collision occurs, the computer device can apply positive incentives to the movement, generating a corresponding positive incentive value.

[0139] Specifically, if any distance value between the arm of the robotic arm and the human object is less than a preset threshold, or any distance value indicates that the arm collides with the human object, the corresponding incentive value generated may be a maximum negative incentive value, such as the incentive value of -1000 in the above example. In the case where any distance value between the arm of the robotic arm and the human object is greater than or equal to a preset threshold, the corresponding incentive value generated may be a positive incentive value, and the magnitude of the positive incentive value is positively correlated with the distance value between the arm and the human object. In other words, the farther the distance between the arm and the human object, the greater the positive incentive value.

[0140] As an example of an embodiment of the present application, when the arm does not collide with the human object, the corresponding excitation mode can be expressed as:

[0141]

[0142] in, is the distance standard deviation, the preset threshold can be 0.01 meters (m), so it should meet .if , that is, when the distance value is less than the preset threshold, the arm of the robot and the human object are approximately in collision, and the maximum negative excitation will be generated at this time.

[0143] The smoothness of a robot's joint motion can be used to indicate the jitter of each joint during movement. A greater smoothness indicates less joint jitter, while a lower smoothness indicates greater joint jitter. Therefore, the incentive value generated based on the smoothness of the robot's joint motion can be positively correlated with the smoothness. A greater smoothness indicates less joint jitter and a correspondingly greater incentive value, while a lower smoothness indicates a lower incentive value.

[0144] As an example of an embodiment of the present application, the excitation mode corresponding to the joint motion smoothness can be expressed as:

[0145]

[0146]

[0147] in, is the angle of the kth joint, is the velocity of the kth joint, is the acceleration of the kth joint, and is a weight coefficient used to balance the effects of velocity and acceleration on smoothness, and n is the total number of joints in the robot arm. For example, for a seven-axis robot arm, n=7.

[0148] The number of times the updated position of the robotic arm is calculated can be used to represent the number of times the model is trained. The computer device can pre-set a maximum number of training cycles. If the number of times the updated position is calculated exceeds this maximum number of training cycles, it indicates that the training cannot achieve a result within the maximum number of training cycles, and a corresponding negative incentive can be applied. Specifically, if the number of times the updated position of the robotic arm is calculated exceeds the maximum number of training cycles, the corresponding incentive value can be the maximum negative incentive value, such as the aforementioned incentive value of -1000.

[0149] The magnitude of the external force can be used to represent the magnitude of the external force that the manipulator can perceive, as calculated by the manipulator's dynamic model. In practical applications, external forces acting on the manipulator should be avoided as much as possible. Therefore, if the external force is calculated based on the dynamic model, a negative excitation can be applied to this movement. Since the negative excitation value is a negative number, the excitation value of the negative excitation is negatively correlated with the magnitude of the external force and positively correlated with the absolute value of the negative excitation value. That is, the absolute value of the negative excitation value corresponding to the magnitude of the external force calculated based on the manipulator's dynamic model is positively correlated with the magnitude of the external force. The greater the external force, the greater the absolute value of the negative excitation value, and the smaller the negative excitation value represented by a negative number.

[0150] As an example of an embodiment of the present application, the excitation mode corresponding to the external force can be expressed as:

[0151]

[0152] in, F ext is the magnitude of the external force.

[0153] Different surgical indications have different requirements for the potential energy state of the robotic arm. For example, some surgical indications require a high potential energy state, while others require a low potential energy state. The potential energy state of the robotic arm can be the sum of the potential energies of each link of the robotic arm, that is, the potential energy state of the robotic arm can be expressed as:

[0154]

[0155] In the above expression, mi is the mass of the i-th connecting rod, is the direction of gravity, is the homogeneous transformation matrix from the base coordinate system to the i-th link coordinate system.

[0156] For high potential energy states, the corresponding excitation mode can be expressed as:

[0157]

[0158] For the low potential energy state, the corresponding excitation mode can be expressed as:

[0159]

[0160] After each movement of the robotic arm, the incentive value for that movement can be determined based on one or more of the above factors. The resulting incentive value is then used to comprehensively evaluate the rationality and safety of the movement. The rationality and safety of the movement are positively correlated with the incentive value. A larger positive incentive value indicates a more rational and safer movement. Conversely, a negative incentive value indicates the need for adjustments to the movement process.

[0161] The termination conditions of model training may include exceeding the training number limit or completing the training task, for example, the total incentive value obtained during the training process exceeds a certain threshold, etc.

[0162] After training is completed to obtain a motion planning model that matches the surgical indication, the computer device can save the surgical indication and the matching motion planning model accordingly. Accordingly, the strategy obtained from model training will also be used as the strategy for subsequent use, or as the initial strategy for updating training when it is necessary to update the training. In this way, when a robotic arm motion planning task for the same surgical indication is subsequently received, the saved motion planning model can be updated and trained based on the three-dimensional simulation environment corresponding to the actual surgical scenario required for the surgical indication, thereby improving the utilization rate of the training data.

[0163] S204: Using the motion planning model to plan the motion of the robotic arm during surgery.

[0164] The trained motion planning model can be applied in subsequent actual intraoperative scenarios to plan the movement of the robotic arm during surgery, ensuring that the robotic arm moves along the planned path without collision and with high smoothness.

[0165] By applying the motion planning method provided in the embodiments of this application, a three-dimensional simulation environment for the surgical scenario is constructed. After determining the current surgical indication and configuring a state space for that surgical indication, the three-dimensional simulation environment can be trained based on the configured state space to obtain a motion planning model that matches the surgical indication. In actual intraoperative scenarios, this motion planning model can be used to plan the motion of the robotic arm, allowing it to follow the planned path and ensure the safety and smoothness of the surgery.

[0166] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0167] Reference Figure 4, shows a schematic diagram of a motion planning device for a robotic arm provided in an embodiment of the present application, which may specifically include an environment construction module 401, a state configuration module 402, a model training module 403, and a motion planning module 404, wherein:

[0168] An environment construction module 401 is used to construct a three-dimensional simulation environment of a surgical scene, wherein the three-dimensional simulation environment includes objects for simulating a robotic arm and multiple human objects;

[0169] A state configuration module 402 is configured to determine a current surgical indication and configure a state space for the surgical indication, wherein the state space includes a plurality of parameters matching the surgical indication;

[0170] A model training module 403 is configured to train the three-dimensional simulation environment based on the state space to obtain a motion planning model that matches the surgical indication;

[0171] The motion planning module 404 is used to plan the motion of the robotic arm during surgery using the motion planning model.

[0172] In the embodiment of the present application, the model training module 403 can be specifically used to:

[0173] Calculating an arm angle variation of the robotic arm according to a first state corresponding to the state space, wherein the arm angle variation is within a feasible arm angle region of the robotic arm;

[0174] Based on the arm angle change, an inverse kinematics formula is used to solve the updated position of the robotic arm;

[0175] The robotic arm is moved to the updated position, and the second state corresponding to the state space after the movement is obtained. The second state will be used as the first state in the next training to recalculate the arm angle change of the robotic arm until the training termination condition is reached, thereby obtaining a motion planning model that matches the surgical indication.

[0176] In a possible implementation of the embodiment of the present application, the second state includes a distance value between the robotic arm and each of the human objects, and the model training module 403 may further be used to:

[0177] When any distance value indicates that the robotic arm collides with the human object, the updated position is adjusted, and the robotic arm is moved to the adjusted updated position.

[0178] In another possible implementation of the embodiment of the present application, the model training module 403 may also be used to:

[0179] Determine an incentive value generated by moving the robotic arm to the updated position, the incentive value including a positive incentive value and / or a negative incentive value, and the incentive value is determined based on one or more of the following:

[0180] The positional relationship between the arm angle of the robotic arm and the feasible arm angle range;

[0181] The distance between the arm of the robotic arm and the human object;

[0182] The smoothness of the joint motion of the robotic arm;

[0183] Solving and obtaining the number of updated positions of the robotic arm;

[0184] the magnitude of the external force; and,

[0185] The potential energy state of the robotic arm.

[0186] In the embodiment of the present application, if the arm angle of the robotic arm exceeds the feasible arm angle range, the corresponding incentive value generated is the maximum negative incentive value; when the arm angle of the robotic arm is within the feasible arm angle range, the corresponding incentive value generated is a positive incentive value, and the positive incentive value is negatively correlated with the distance between the arm angle and the axis in the feasible arm angle range;

[0187] If any distance value between the arm body of the robotic arm and the human object is less than a preset threshold, or any of the distance values ​​indicates that the arm body and the human object have collided, the corresponding incentive value generated is a maximum negative incentive value; if any distance value between the arm body of the robotic arm and the human object is greater than or equal to the preset threshold, the corresponding incentive value generated is a positive incentive value, and the positive incentive value is positively correlated with the distance value between the arm body and the human object;

[0188] The incentive value generated based on the joint motion smoothness of the robotic arm is positively correlated with the joint motion smoothness;

[0189] When the number of times the updated position of the robotic arm is solved exceeds the maximum number of training times, the corresponding generated incentive value is the maximum negative incentive value;

[0190] The absolute value of the negative excitation value generated corresponding to the magnitude of the external force calculated based on the dynamic model of the robotic arm is positively correlated with the magnitude of the external force.

[0191] In a possible implementation of the embodiment of the present application, the objects used to simulate the robotic arm and the plurality of human objects each have corresponding markers, and the apparatus may further include:

[0192] The distance value calculation module is used to calculate the distance value between the robotic arm and each of the human objects based on the markers on the robotic arm and each of the human objects.

[0193] In the embodiment of the present application, the distance value calculation module can be specifically used to:

[0194] respectively determining the coordinate values ​​of any marker on the robotic arm and any marker on each of the human objects;

[0195] The distance value between any marker on the robotic arm and any marker on each of the human objects is calculated based on the coordinate values; wherein the minimum value of the distance values ​​is the minimum collision distance between each of the markers.

[0196] In a possible implementation of the embodiment of the present application, the model training module 403 may also be used to:

[0197] Correspondingly saving the surgical indication and the matching motion planning model;

[0198] When a robotic arm motion planning task for the same surgical indication is received, the saved motion planning model is updated and trained based on the three-dimensional simulation environment corresponding to the surgical scene actually required for the surgical indication.

[0199] The present application provides a motion planning device for a robotic arm, which can be a computer device or a processing unit in a computer device. The device can be used to implement the various steps in the aforementioned method embodiments.

[0200] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0201] Reference Figure 5 , shows a schematic diagram of a computer device provided by an embodiment of the present application. Figure 5 As shown, the computer device 500 in the embodiment of the present application includes: a processor 510, a memory 520, and a computer program 521 stored in the memory 520 and executable on the processor 510. When the processor 510 executes the computer program 521, the steps in each embodiment of the above-mentioned motion planning method for the robotic arm are implemented, such as Figure 2 Alternatively, when the processor 510 executes the computer program 521, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 4 Functions of modules 401 to 404 are shown.

[0202] Exemplarily, the computer program 521 may be divided into one or more modules / units, which are stored in the memory 520 and executed by the processor 510 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which may be used to describe the execution process of the computer program 521 in the computer device 500. For example, the computer program 521 may be divided into an environment construction module, a state configuration module, a model training module, and a motion planning module, with the specific functions of each module being as follows:

[0203] An environment construction module, configured to construct a three-dimensional simulation environment of a surgical scene, wherein the three-dimensional simulation environment includes objects for simulating a robotic arm and a plurality of human objects;

[0204] A state configuration module, configured to determine a current surgical indication and configure a state space for the surgical indication, wherein the state space includes a plurality of parameters matching the surgical indication;

[0205] A model training module, configured to train the three-dimensional simulation environment based on the state space to obtain a motion planning model that matches the surgical indication;

[0206] The motion planning module is used to plan the motion of the robotic arm during surgery using the motion planning model.

[0207] The computer device 500 may be a device capable of implementing the functions of each step in the aforementioned method embodiments. The computer device 500 may be a computing device such as a desktop computer, a cloud server, etc. The computer device 500 may include, but is not limited to, a processor 510 and a memory 520. It will be understood by those skilled in the art that Figure 5 This is merely an example of the computer device 500 and does not constitute a limitation of the computer device 500 . The computer device 500 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device 500 may also include input and output devices, network access devices, buses, etc.

[0208] The processor 510 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0209] The memory 520 can be an internal storage unit of the computer device 500, such as a hard drive or memory of the computer device 500. The memory 520 can also be an external storage device of the computer device 500, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the computer device 500. Furthermore, the memory 520 can include both an internal storage unit of the computer device 500 and an external storage device. The memory 520 is used to store the computer program 521 and other programs and data required by the computer device 500. The memory 520 can also be used to temporarily store data that has been output or is about to be output.

[0210] An embodiment of the present application further discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the methods described in the aforementioned embodiments are implemented.

[0211] An embodiment of the present application further discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a computer, the methods described in the aforementioned embodiments are implemented.

[0212] An embodiment of the present application further discloses a computer program product, including a computer program. When the computer program is run on a computer, the computer is caused to execute the methods described in the aforementioned embodiments.

[0213] The above-described embodiments are merely used to illustrate the technical solutions of the present application, but not to limit the same. Although the present application is described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some of the technical features can be replaced equivalently, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A motion planning method for a robotic arm, characterized in that: include: Constructing a three-dimensional simulation environment of a surgical scene, wherein the three-dimensional simulation environment includes objects for simulating a robotic arm and a plurality of human objects; Determining a current surgical indication and configuring a state space for the surgical indication, wherein the state space includes a plurality of parameters matching the surgical indication; Training the three-dimensional simulation environment based on the state space to obtain a motion planning model that matches the surgical indication; Using the motion planning model to plan the motion of the robotic arm during surgery; The training of the three-dimensional simulation environment based on the state space to obtain a motion planning model matching the surgical indication includes: Calculating an arm angle variation of the robotic arm according to a first state corresponding to the state space, wherein the arm angle variation is within a feasible arm angle region of the robotic arm; Based on the arm angle change, an inverse kinematics formula is used to solve the updated position of the robotic arm; The robotic arm is moved to the updated position, and a second state corresponding to the state space after the movement is obtained. The second state is used as the first state in the next training to recalculate the arm angle change of the robotic arm until the training termination condition is met, thereby obtaining a motion planning model that matches the surgical indication. After obtaining the second state corresponding to the state space after the movement, the method further includes: determining an incentive value generated by moving the robotic arm to the updated position, the incentive value including a positive incentive value and / or a negative incentive value, and the incentive value being determined based on a positional relationship between an arm angle of the robotic arm and the feasible arm angle range; If the arm angle of the robotic arm exceeds the feasible arm angle range, the corresponding excitation value generated is the maximum negative excitation value; when the arm angle of the robotic arm is within the feasible arm angle range, the corresponding excitation value generated is a positive excitation value, and the positive excitation value is negatively correlated with the distance value between the arm angle and the axis in the feasible arm angle range.

2. The method according to claim 1, characterized in that The second state includes the distance value between the robotic arm and each of the human objects. After obtaining the second state corresponding to the state space after the movement, the method further includes: When any distance value indicates that the robotic arm collides with the human object, the updated position is adjusted, and the robotic arm is moved to the adjusted updated position.

3. The method according to claim 1 or 2, characterized in that The incentive value is also determined based on one or more of the following: The distance between the arm of the robotic arm and the human object; The smoothness of the joint movement of the robotic arm; Solving and obtaining the number of updated positions of the robotic arm; the magnitude of the external force; as well as, The potential energy state of the robotic arm.

4. The method according to claim 3, characterized in that If any distance value between the arm body of the robotic arm and the human object is less than a preset threshold, or any of the distance values ​​indicates that the arm body and the human object have collided, the corresponding incentive value generated is the maximum negative incentive value; When any distance value between the arm body of the robotic arm and the human object is greater than or equal to the preset threshold, the corresponding incentive value generated is a positive incentive value that is positively correlated with the distance value between the arm body and the human object; The incentive value generated based on the joint motion smoothness of the robotic arm is positively correlated with the joint motion smoothness; When the number of times the updated position of the robotic arm is solved exceeds the maximum number of training times, the corresponding generated incentive value is the maximum negative incentive value; The absolute value of the negative excitation value generated corresponding to the magnitude of the external force calculated based on the dynamic model of the robotic arm is positively correlated with the magnitude of the external force.

5. The method according to any one of claims 1, 2 or 4, characterized in that: The objects used to simulate the robotic arm and the plurality of human objects each have corresponding markers, and the method further includes: The distance value between the robotic arm and each of the human objects is calculated based on the markers on the robotic arm and each of the human objects.

6. The method according to claim 5, characterized in that The calculating, based on the markers on the robotic arm and the human objects, a distance value between the robotic arm and each of the human objects, comprises: respectively determining the coordinate values ​​of any marker on the robotic arm and any marker on each of the human objects; The distance value between any marker on the robotic arm and any marker on each of the human objects is calculated based on the coordinate values; wherein the minimum value of the distance values ​​is the minimum collision distance between each of the markers.

7. The method according to any one of claims 1, 2, 4 or 6, characterized in that: After the three-dimensional simulation environment is trained based on the state space to obtain a motion planning model that matches the surgical indication, the method further includes: Correspondingly saving the surgical indication and the matching motion planning model; When a robotic arm motion planning task for the same surgical indication is received, the saved motion planning model is updated and trained based on the three-dimensional simulation environment corresponding to the surgical scene actually required for the surgical indication.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the computer device is caused to implement the method according to any one of claims 1 to 7.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed, the method according to any one of claims 1 to 7 is performed.

Images