Arthroscopic robotic arm planning method, device, equipment and medium

Abstract

The disclosed embodiments relate to an arthroscopic manipulator planning method, device, equipment and medium, the method comprising: obtaining the position of the end of the manipulator arm in the joint cavity based on navigation information, determining the obstacle posture information, entering the automatic mode based on the preset photoelectric sensor detecting that the target object is not on the power handle, obtaining the current posture of the manipulator arm, and obtaining the pre-planned target bone channel posture, performing real-time planning based on the current posture, obstacle posture information and target bone channel posture, obtaining an obstacle avoidance trajectory, and controlling the manipulator arm to move to the target position point corresponding to the target bone channel posture based on the obstacle avoidance trajectory. The above technical solution is adopted to ensure that after the end of the manipulator arm enters the joint cavity, the manipulator arm is controlled to move to the pre-planned target position point based on the real-time planned obstacle avoidance trajectory, and has the ability of active obstacle avoidance planning, which solves the technical problem that the existing technology cannot ensure that the incision position is on the planned bone channel, resulting in additional incisions or secondary planning of the bone channel.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent medical control technology, and in particular to an arthroscopic robotic arm planning method, device, equipment and medium. Background Art

[0002] Usually, in anterior cruciate ligament robotic surgery, the skin needs to be opened first, and the probe is inserted from the incision position to obtain the target bone tunnel position after alignment. The robotic arm obtains the target bone tunnel position, adjusts the posture and non-axis position outside the skin, and then pushes the robotic arm from outside the skin into the joint cavity.

[0003] However, there is no guarantee that the incision is located on the planned bone tract, resulting in additional incisions or secondary planning of the bone tract. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an arthroscopic robotic arm planning method, device, equipment and medium.

[0005] The present disclosure provides an arthroscopic robotic arm planning method, the method comprising:

[0006] Based on the navigation information, the end of the robotic arm is located in the joint cavity, and the obstacle position information is determined;

[0007] When the preset photoelectric sensor detects that the target object is not on the power handle, it enters the automatic mode, obtains the current posture of the robot arm, and obtains the pre-planned target bone tunnel posture;

[0008] Perform real-time planning based on the current posture, the obstacle posture information and the target bone tunnel posture to obtain an obstacle avoidance trajectory;

[0009] Based on the obstacle avoidance trajectory, the robotic arm is controlled to move to a target position point corresponding to the target bone channel posture.

[0010] The present disclosure also provides an arthroscopic robotic arm planning device, the device comprising:

[0011] A determination module is used to obtain the position of the end of the robot arm in the joint cavity based on the navigation information and determine the obstacle posture information;

[0012] An acquisition module, used to enter the automatic mode when a preset photoelectric sensor detects that the target object is not on the power handle, acquire the current posture of the robot arm, and acquire the pre-planned target bone tunnel posture;

[0013] A planning module, used for performing real-time planning based on the current posture, the obstacle posture information and the target bone tunnel posture to obtain an obstacle avoidance trajectory;

[0014] A control module is used to control the robot arm to move to a target position point corresponding to the target bone channel posture based on the obstacle avoidance trajectory.

[0015] An embodiment of the present disclosure also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the arthroscopic robotic arm planning method provided in the embodiment of the present disclosure.

[0016] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the arthroscopic robotic arm planning method provided in the embodiment of the present disclosure.

[0017] The present disclosure also provides a computer program product, including a computer program, wherein the computer program is executed by a processor as the arthroscopic robotic arm planning method provided in the embodiment of the present application.

[0018] The technical solution provided by the embodiment of the present disclosure has the following advantages over the prior art: the arthroscopic manipulator planning solution provided by the embodiment of the present disclosure obtains the position of the manipulator end in the joint cavity based on navigation information, determines the obstacle posture information, enters the automatic mode when the preset photoelectric sensor detects that the target object is not on the power handle, obtains the current posture of the manipulator, and obtains the pre-planned target bone channel posture, performs real-time planning based on the current posture, obstacle posture information and target bone channel posture, obtains the obstacle avoidance trajectory, and controls the manipulator to move to the target position point corresponding to the target bone channel posture based on the obstacle avoidance trajectory. The above technical solution is adopted to ensure that after the end of the manipulator enters the joint cavity, the manipulator is controlled to move to the pre-planned target position point based on the real-time planning obstacle avoidance trajectory, and has the ability of active obstacle avoidance planning, which solves the technical problem that the prior art cannot ensure that the incision position is on the planned bone channel, resulting in an increase in the incision or secondary planning of the bone channel. In addition, it can sense the user's intention and allow the user to change the motion trajectory, that is, the user is allowed to manually push and pull the manipulator to correct the trajectory at any time, and the manipulator re-plans the trajectory at the corrected position, and the whole process is safe and controllable. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A schematic diagram of a flow chart of an arthroscopic robotic arm planning method provided in an embodiment of the present disclosure;

[0021] Figure 2 A schematic flow chart of another arthroscopic robotic arm planning method provided by an embodiment of the present disclosure;

[0022] Figure 3 A schematic flow chart of another arthroscopic robotic arm planning method provided in an embodiment of the present disclosure;

[0023] Figure 4 An example diagram of a process for planning an arthroscopic robotic arm provided in an embodiment of the present disclosure;

[0024] Figure 5 An example diagram of a 4th-order curve obstacle avoidance trajectory planning provided in an embodiment of the present disclosure;

[0025] Figure 6 An example diagram of another fourth-order curve obstacle avoidance trajectory planning provided in an embodiment of the present disclosure;

[0026] Figure 7 A schematic diagram of the structure of an arthroscopic robotic arm planning device provided in an embodiment of the present disclosure;

[0027] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0031] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] At present, in the method of positioning the robot arm, it is usually only based on the positioning method of the bone channel, and it is difficult to ensure that the incision is exactly satisfied. In order to meet the positioning of the bone channel, the incision will be increased. Without the automatic positioning method of the robot arm, it is difficult to manually adjust the hand drill with the mark to the planned posture.

[0035] In view of the above problems, the present invention proposes an arthroscopic robotic arm planning method, including: 1) the user pokes the probe on the bone surface through an incision for registration, and the navigation software plans the target bone tunnel; 2) the robotic arm automatically adjusts its posture according to the position of the target bone tunnel, and the position is selected as the extension line of the bone tunnel; 3) the user can drag the robotic arm (only the position can be changed, not the posture) to enter the joint cavity through the incision (changing the position can enter the joint cavity from the incision, and obstacle avoidance in the joint cavity can also rely solely on position adjustment); 4) the robotic arm determines that it has entered the joint cavity based on navigation information (i.e., visual information), and the user holds the robotic arm with his hand. When the robotic arm senses that the external force values ​​in the X, Y, and Z directions are all less than the threshold, the robotic arm adjusts the position based on medical images such as CT (Computed Tomography) images, MR (Magnetic Resonance Imaging) images, etc. The robot automatically plans the trajectory from the current position to the target bone tunnel position by means of magnetic resonance imaging (MRI) images, etc.; when the robot senses that any of the external forces in the X, Y, and Z directions is greater than a threshold, the robot moves along with the external force value until the external force value is lower than the threshold; the robot updates the starting position and plans the trajectory to the target bone tunnel position until it reaches the target bone tunnel position.

[0036] Therefore, after planning the bone tunnel, the user drags the robotic arm from the incision position into the joint cavity. The robotic arm plans the obstacle avoidance trajectory based on the current posture, target bone tunnel posture (the posture remains unchanged, and the target position point is the starting position of the target bone tunnel posture), CT, MR images, etc., and the user is allowed to manually push and pull the robotic arm to correct the trajectory at any time. The robotic arm re-plans the trajectory at the corrected position, that is, with the automatic positioning of the robotic arm, the incision position is met, combining automatic planning and manual intervention.

[0037] Figure 1 The present invention provides a flowchart of an arthroscopic robot arm planning method, which can be performed by an arthroscopic robot arm planning device, wherein the device can be implemented by software and / or hardware, and can generally be integrated into an electronic device. Figure 1 As shown, the method includes:

[0038] Step 101: Based on the navigation information, the end of the robot arm is obtained in the joint cavity, and the obstacle posture information is determined.

[0039] Specifically, the navigation information includes the current position information of the robotic arm, so that the obstacle posture information can be determined when the distal end of the robotic arm is in the joint cavity based on the navigation information; wherein, taking into account the situation that the incision position and the planned bone tunnel are not colinear, the user is allowed to drag the robotic arm from the incision position into the joint cavity. In order to avoid adding a new incision, the embodiment of the present disclosure controls how to control the robotic arm to move to the target position point corresponding to the target bone tunnel posture when the robotic arm enters the joint cavity, that is, the starting point of the planned bone tunnel.

[0040] In the embodiment of the present disclosure, the obstacle posture information refers to information such as the number and posture of obstacles.

[0041] In the embodiments of the present disclosure, there are many ways to determine the obstacle posture information. In some embodiments, the first obstacle posture is obtained based on the medical image, the second obstacle posture is obtained from the navigation host computer image, the third obstacle is determined through the arthroscopy, and the third obstacle posture corresponding to the third obstacle is collected through the peripheral device, the fourth obstacle posture is obtained based on the pre-installed second force sensor, and the obstacle posture information is determined based on the first obstacle posture, the second obstacle posture, the third obstacle posture and the fourth obstacle posture. In other embodiments, the obstacle posture information is determined based on the obstacle posture obtained from the medical image and the obstacle posture obtained based on the pre-installed force sensor.

[0042] Step 102: When the preset photoelectric sensor detects that the target object is not on the power handle, the automatic mode is entered to obtain the current posture of the robot arm and the pre-planned target bone channel posture.

[0043] Among them, the photoelectric sensor is installed on the power handle to detect whether the target object (such as a human hand) is placed on the power handle. Therefore, when the photoelectric sensor detects that the target object is not on the power handle, it enters the automatic mode to obtain the current posture of the robotic arm and the pre-planned target bone tunnel posture.

[0044] Among them, the current posture includes information such as the current position point of the operating tool; the target bone tunnel posture refers to information such as the target position point of the planned bone tunnel. The target bone tunnel posture can be obtained by determining the incision position, aligning the probe on the target bone surface based on the incision position, and planning based on the alignment result using the navigation planning software.

[0045] Step 103: Perform real-time planning based on the current posture, obstacle posture information and target bone posture to obtain an obstacle avoidance trajectory.

[0046] Step 104: Control the robotic arm to move to a target position point corresponding to the target bone channel posture based on the obstacle avoidance trajectory.

[0047] In the disclosed embodiment, real-time planning is performed based on the current posture, obstacle posture information and target bone channel posture to obtain an obstacle avoidance trajectory. Specifically, a starting position point is determined based on the current posture, a certain number of obstacle position points are determined based on the obstacle posture information, a target position point is determined based on the target bone channel posture, and an N-order curve is determined as the obstacle avoidance trajectory based on the starting position point, a certain number of control points and the target position point; wherein N is a positive integer greater than or equal to 4.

[0048] Specifically, the starting position point is determined according to the current posture, and the obstacle avoidance trajectory is planned in real time with a certain number of obstacle position points as control points and the starting point of the target bone channel as the target position point (final position); more specifically, the obstacle avoidance trajectory is generated according to all obstacle position points, the current position points of the operating tool and the target position points, using at least a fourth-order curve such as a Bezier curve, that is, from the stage when the end of the robotic arm enters the joint cavity to the final position of the starting point of the target bone channel, the robotic arm is in automatic movement mode when there is no manual dragging, and the robotic arm is controlled to move to the target position point corresponding to the posture of the target bone channel based on the obstacle avoidance trajectory.

[0049] The arthroscopic manipulator planning scheme provided by the disclosed embodiment obtains the position of the manipulator arm in the joint cavity based on navigation information, determines the obstacle posture information, enters automatic mode when the preset photoelectric sensor detects that the target object is not on the power handle, obtains the current posture of the manipulator arm, and obtains the pre-planned target bone channel posture, performs real-time planning based on the current posture, obstacle posture information and target bone channel posture, obtains the obstacle avoidance trajectory, and controls the manipulator arm to move to the target position point corresponding to the target bone channel posture based on the obstacle avoidance trajectory. The above technical scheme is adopted to ensure that after the end of the manipulator arm enters the joint cavity, the manipulator arm is controlled to move to the pre-planned target position point based on the real-time planned obstacle avoidance trajectory, and has the ability of active obstacle avoidance planning, which solves the technical problem that the existing technology cannot ensure that the incision position is on the planned bone channel, resulting in additional incisions or secondary planning of the bone channel.

[0050] Figure 2 This is a flow chart of another arthroscopic robot arm planning method provided by the present disclosure. Based on the above embodiment, this embodiment further optimizes the above arthroscopic robot arm planning method. Figure 2 As shown, the method also includes:

[0051] Step 201: Switch to manual mode when a photoelectric sensor detects that a target object is on the power handle, and obtain a real-time external force value in any target direction of the robotic arm based on a preset first force sensor.

[0052] Step 202: When the real-time external force value in any target direction of the robotic arm is greater than or equal to a preset first threshold, the robotic arm is controlled to update the movement trajectory.

[0053] Step 203: When the photoelectric sensor detects that the target object is not on the power handle, the mode is switched to the automatic mode, and the current updated posture of the robot arm is obtained when the real-time external force value is less than the first threshold.

[0054] Step 204: Update the obstacle avoidance trajectory based on the current updated posture, the target bone posture and the obstacle posture information.

[0055] In the disclosed embodiment, on the basis of automatically planning the trajectory, the user is allowed to drag the robot arm to manually modify the trajectory at any time, ensuring that the whole process is safe and controllable and the operation is in line with the user's habits. Among them, the first threshold can be set to a specific value according to the application scenario and adjusted as needed during use.

[0056] In the embodiment of the present disclosure, the first force sensor at the end flange of the robotic arm, such as a six-dimensional force sensor, is connected at the end flange of the robotic arm and the power handle to detect the trend of artificial movement, that is, the robotic arm and the power handle are connected on the end flange of the robotic arm to detect human movement intention, that is, the real-time external force value in any target direction of the robotic arm can be obtained.

[0057] Specifically, when it is detected that the real-time external force value in any direction of the X, Y, and Z directions is greater than or equal to a certain first threshold, the robotic arm can comply with the external force, change the current updated posture (current position point) of the operating tool, but do not change the posture, and regenerate the obstacle avoidance trajectory.

[0058] Figure 3 This is a flow chart of another arthroscopic robot arm planning method provided by the present disclosure. Based on the above embodiment, this embodiment further optimizes the above arthroscopic robot arm planning method. Figure 3 As shown, the method includes:

[0059] Step 301, based on the navigation information, the end of the robotic arm is obtained in the joint cavity, the first obstacle posture is determined based on the medical image, the second obstacle posture is obtained by obtaining the navigation host image, and the third obstacle is determined by the arthroscope, and the third obstacle posture corresponding to the third obstacle is collected by the peripheral device.

[0060] Step 302: When the preset photoelectric sensor detects that the target object is not on the power handle, the automatic mode is entered to obtain the current posture of the robot arm and the pre-planned target bone channel posture.

[0061] Step 303: determine a starting position point based on the current posture, determine a candidate number of obstacle position points based on the obstacle posture information, and determine a target position point based on the target bone tunnel posture.

[0062] Step 304: When the number of all current position points is less than a preset number threshold, determine the target number of control points based on the candidate number of obstacle position points; wherein, when the target number is less than the number threshold, determine an N-order curve as the obstacle avoidance trajectory based on the starting position point, the target number of control points and the target position point; wherein N is a positive integer greater than or equal to 4.

[0063] In some embodiments, determining the target number of control points based on the candidate number of obstacle position points includes: determining a supplementary number based on the difference between the candidate number and the target number, determining at least one obstacle position point from the candidate number of obstacle position points and supplementing it according to the supplementary number to obtain the target number of control points. The number threshold can be set according to the application scenario, and the target order curve can be determined according to the accuracy requirements of the obstacle avoidance trajectory, so as to determine the number threshold according to the value of the target order. For example, in the case of a fourth-order curve, the number threshold is set to 5.

[0064] Step 305: When the number of all current position points is greater than or equal to a preset number threshold, a target number of obstacle position points are determined from the candidate number of obstacle position points, and an N-order curve is determined as an obstacle avoidance trajectory based on the target number of control points corresponding to the starting position point and the target number of obstacle position points.

[0065] Step 306, when the real-time external force value of any target direction of the robotic arm obtained based on the first force sensor is greater than or equal to a preset second threshold, and / or the change value of the real-time external force value is greater than or equal to a preset first change threshold, and / or the torque of the robotic arm is greater than or equal to a preset second change threshold, and / or the torque change rate of the robotic arm is greater than or equal to a preset third change threshold, control the robotic arm to stop moving.

[0066] Step 307: When the real-time external force value in any target direction of the robotic arm is greater than or equal to a preset first threshold, the robotic arm is controlled to update the movement trajectory until the real-time external force value is less than the first threshold, the current updated posture of the robotic arm is obtained, and the obstacle avoidance trajectory is updated based on the current updated posture, the target bone channel posture and the obstacle posture information.

[0067] Step 308: Control the robotic arm to move to the target position point corresponding to the target bone channel posture based on the obstacle avoidance trajectory.

[0068] Among them, the quantity threshold, the first threshold, the second threshold, the first change threshold, the second change threshold and the third change threshold can all be selected and set according to actual applications.

[0069] Specifically, when the incision position is not in line with the planned bone channel, if the robot arm is used to automatically position outside the skin, the robot arm cannot make a hole and needs to add an incision (usually done by the user using other tools). Therefore, when the incision position is not in line with the planned bone channel, in order to avoid adding an incision, how to generate an obstacle avoidance trajectory based on the current posture of the operating tool, the posture of the target bone channel, and the position of the obstacle after the operating tool enters the joint cavity through the original incision position.

[0070] Specifically, Figure 4 As shown, step 4.1 the robotic arm reaches the drilling preparation position; step 4.2 drags the robotic arm from the incision into the joint cavity; step 4.3 determines whether it enters the joint cavity; if so, executes step 4.4 to generate obstacle points based on CT and MR images; step 4.5 points out obstacles on the navigation image; step 4.6 generates an obstacle avoidance trajectory based on the current point, target point and obstacle; step 4.7 determines whether there is an obstacle under arthroscopy; if so, executes step 4.8 to push the robotic arm to avoid obstacles and change the trajectory; step 4.9 executes the trajectory; step 4.10 determines whether it reaches the target point; if so, ends.

[0071] In the disclosed embodiment, the current position of the operating tool is acquired in real time (adapting to the user manually pushing and pulling the robotic arm to change the position), the position of the target bone tunnel is determined by the doctor making an incision and poking the probe on the bone surface for registration, and the navigation software plans the target bone tunnel (the starting point of the target bone tunnel is the final position); the obstacle position is based on the information obtained from CT and MR images, and the obstacle position can be manually added on the host computer, plus the position determined by the arthroscopy and foot pedal; according to the positions of all obstacles, the current position of the operating tool and the target position, a fourth-order curve is used, such as a Bezier curve, to generate an obstacle avoidance trajectory (starting from entering the joint cavity to the target At the final position of the starting point of the bone track, the robot is in automatic motion mode when there is no human dragging, and in manual mode when there is a human hand on the power handle. The trajectory of this stage is planned in real time, for example, based on five points, including the current point (planning starting point) and the nearest four points); in this process, if it is detected that the force in any direction of the X, Y, and Z directions is greater than a certain threshold, the robot can comply with the external force, change the current point position of the surgical tool, but do not change the posture, and regenerate the obstacle avoidance trajectory; at the same time, if an obstacle is detected by the second force sensor (such as a three-dimensional force sensor), it will stop.

[0072] Specifically, there are many ways to determine obstacles. When entering the joint cavity, the robot arm can generate some obstacles based on CT and MR images (such as some parts that cannot be touched by physiological characteristics); it can also plan the trajectory of obstacle avoidance based on the points selected by the user on the image of the navigation host computer as obstacles. The user observes through the arthroscope, determines the obstacles, and collects the obstacle posture information through peripherals such as pedals (the arthroscope is not installed at the end of the robot arm, but is handheld and enters the joint cavity through the incision. When the user observes and finds obstacles through the feedback image of the arthroscope, the system sends the collection information through peripherals such as pedals, and the system collects the target point information installed on the arthroscope for positioning to determine the obstacle posture information); the obstacles are automatically detected based on the installed three-dimensional force sensor.

[0073] Specifically, the first sensor is installed on the flange at the end of the robotic arm, connecting the robotic arm and the power handle, and is mainly used to detect human movement intention; the second sensor is installed at the connection between the drill bit and the power handle, and a three-dimensional force sensor is added (compared with the six-dimensional force sensor, the three-dimensional force sensor cannot detect torque, but has strong overload capacity, high resolution and high accuracy, so it is used at the connection between the drill bit and the power handle, and only detects external force when the robotic arm moves automatically). When the robotic arm automatically executes the trajectory in the joint cavity, it determines whether it encounters an obstacle based on the force information.

[0074] Specifically, a photoelectric sensor is installed on the power handle to detect whether a target object, such as a human hand, is placed on the power handle. When the target object is detected on the power handle, the robotic arm is in manual mode. In the process of reaching the target bone tunnel, the user can push the robotic arm to change its trajectory at any time according to the arthroscopic image. When the user does not push the robotic arm, the photoelectric proximity sensor detects that the target object has left, and the robotic arm will re-plan the obstacle avoidance trajectory based on the current point, target point, and obstacle information. The curve is planned using a multi-terminal fourth-order polynomial. The trajectory has the characteristics of first-order, second-order, and third-order differentiability, ensuring that the speed, acceleration, and jerk are controllable.

[0075] It should be noted that in an emergency, the user can twist the hand drill vigorously, and the robotic arm will control the robotic arm to stop according to the torque, the rate of change of the torque, and the rate of change of the force.

[0076] As an example, the robot arm system is equipped with three sensors on the traditional basis. One is a first force sensor at the end flange of the robot arm, such as a six-dimensional force sensor, which is connected to the end flange of the robot arm and the power handle to detect the manual movement trend; one is a photoelectric sensor on the power handle to detect whether the human hand is placed on the power handle. A second force sensor at the connection between the drill bit and the power handle is used to detect the resistance encountered by the robot arm during automatic operation.

[0077] Specifically, the robotic arm is positioned to the drilling preparation position according to the navigation information. The drilling preparation position means that the posture is consistent with the posture of the bone tunnel and the position is on the extension line of the bone tunnel, allowing the doctor to drag and change the position of the robotic arm. and The doctor drags the robotic arm through the incision into the joint cavity.

[0078] Specifically, the end of the robot arm is judged whether it is in the joint cavity based on the navigation information. When in the joint cavity, the tissue structure in the joint cavity is complex. For the robot, it is necessary to first determine the obstacle information, and the obstacle information is determined in a variety of ways. For example, obstacles are determined based on CT and MR images (points that cannot be touched under physiological characteristics); for example, points selected by users on the navigation interface; for example, under arthroscopes (arthroscopy is equipped with target points for positioning), doctors use pedals and other peripherals to determine obstacle information by observing the actual situation, and perform automatic obstacle avoidance trajectory planning.

[0079] Specifically, the robot arm trajectory automatic planning method, that is, the current point (starting point) and the obstacle are the control points. In order to ensure the continuous and smooth curve of the robot arm's automatic operation, the multi-end splicing continuous smooth trajectory planning method is adopted. The first-order differentiable represents the controllable speed, the second-order differentiable represents the controllable acceleration, and the third-order differentiable represents the controllable acceleration. Therefore, at least a fourth-order polynomial is selected to determine the trajectory, and at least 4 obstacle points are selected. The mathematical expression of the curve can be:

[0080] c(t)=P0(1-t) 4 +4P1t(1-t) 3 +4P2t 2 (1-t) 2 +4P3t 3 (1-t)+P4t 4 Among them, P0 is

[0081] The coordinates of the previous position point, P1 to P4 are the coordinates of the obstacle position points, and t is the weight parameter.

[0082] It should be noted that the last obstacle is not necessarily the end point planned by the navigation host computer.

[0083] As an example, if the current point plus the number of remaining obstacle points ≥ 5, such as Figure 5 As shown, P0 is the current point (planning starting point), P1, P2, and P3 are selected as control points, the control points are located inside the obstacle (that is, the control points are on the obstacle), and P4 is located outside the obstacle (that is, P4 is not on the obstacle, because if the last point is on the fourth-order curve, there will be a collision).

[0084] As an example, if the current point plus the number of remaining obstacle points is less than 5, then the control points are supplemented according to the number and obstacle information, such as Figure 6 As shown, the order of the curve remains unchanged. Figure 6 In the example, two points are selected on obstacle 1 and one point is selected on obstacle 2. Alternatively, you can select one point on obstacle 1 and two points on obstacle 2. The specific settings are selected according to the application scenario.

[0085] It should be noted that the CT and MR images and the points to be avoided confirmed by the user on the navigation host computer may not fully reflect all obstacle information. The user is required to observe in real time under the arthroscopy and adjust the robot arm trajectory at any time according to the situation. When it is detected that the force in any direction of the X, Y, and Z directions is greater than a certain threshold, the robot arm complies with the external force and changes the trajectory until the external force is less than a certain threshold and the human hand is detected to leave. Then the current point is set as the starting point, and the known obstacle is used as the control point for automatic obstacle avoidance trajectory planning.

[0086] This ensures that after the end of the robotic arm penetrates into the human joint cavity, it has the ability to actively avoid obstacles and plan, and can sense the user's intentions, allowing the user to change the motion trajectory, and the entire process is safe and controllable.

[0087] Figure 7 This is a schematic structural diagram of an arthroscopic robotic arm planning device provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device.

[0088] like Figure 7 As shown, the device comprises:

[0089] The determination module 401 is used to obtain the position information of the obstacle based on the navigation information to determine that the end of the robot arm is in the joint cavity.

[0090] The acquisition module 402 is used to enter the automatic mode when the preset photoelectric sensor detects that the target object is not on the power handle, acquire the current posture of the robot arm, and acquire the pre-planned target bone channel posture.

[0091] The planning module 403 is used to perform real-time planning based on the current posture, the obstacle posture information and the target bone tunnel posture to obtain an obstacle avoidance trajectory.

[0092] The control module 404 is used to control the robot arm to move to the target position point corresponding to the target bone channel posture based on the obstacle avoidance trajectory.

[0093] Optionally, the device further includes:

[0094] A first switching acquisition module, configured to switch to a manual mode when the photoelectric sensor detects that the target object is on the power handle, and to acquire a real-time external force value in any target direction of the robotic arm based on a preset first force sensor;

[0095] A first updating module, used for controlling the robotic arm to update the moving trajectory when the real-time external force value in any target direction of the robotic arm is greater than or equal to a preset first threshold;

[0096] A second switching acquisition module is used to switch to the automatic mode when the photoelectric sensor detects that the target object is not on the power handle, and to acquire the current updated posture of the robotic arm when the real-time external force value is less than the first threshold value;

[0097] The second updating module is used to update the obstacle avoidance trajectory based on the current updated posture, the target bone channel posture and the obstacle posture information.

[0098] Optionally, the determining module 401 is specifically configured to:

[0099] The end of the robotic arm is obtained to be in the joint cavity based on the navigation information, and the position of the first obstacle is obtained based on the medical image;

[0100] Obtain the second obstacle pose from the navigation host computer image;

[0101] Determine a third obstacle through an arthroscopy, and acquire a third obstacle posture corresponding to the third obstacle through an external device;

[0102] Acquire a fourth obstacle posture based on a pre-installed second force sensor;

[0103] Obstacle pose information is determined based on the first obstacle pose, the second obstacle pose, the third obstacle pose, and the fourth obstacle pose.

[0104] Optionally, the planning module 403 is specifically configured to:

[0105] Determine a starting position point based on the current posture;

[0106] Determine a candidate number of obstacle position points based on the obstacle posture information;

[0107] Determining the target position point based on the target bone tunnel posture;

[0108] When the number of all current position points is less than a preset number threshold, determining a target number of control points based on the candidate number of obstacle position points; wherein the target number is less than the number threshold;

[0109] Based on the starting position point, the target quantity control point and the target position point, an N-order curve is determined as the obstacle avoidance trajectory; wherein N is a positive integer greater than or equal to 4.

[0110] Optionally, the planning module 403 is further configured to:

[0111] When the number of all current position points is greater than or equal to a preset number threshold, determining a target number of obstacle position points from the candidate number of obstacle position points;

[0112] An N-th order curve is determined as the obstacle avoidance trajectory based on the target number of control points corresponding to the starting position point and the target number of obstacle position points.

[0113] Optionally, determining a target number of control points based on the candidate number of obstacle position points includes:

[0114] Determine the supplement quantity based on the difference between the candidate quantity and the target quantity;

[0115] At least one obstacle position point is determined from the candidate number of obstacle position points and supplemented according to the supplementary number to obtain the target number of control points.

[0116] Optionally, the arthroscopic robotic arm planning device further comprises:

[0117] A control stop module is used to control the robotic arm to stop moving when the real-time external force value of any target direction of the robotic arm obtained by the first force sensor is greater than or equal to a preset second threshold, and / or the change value of the real-time external force value is greater than or equal to a preset first change threshold, and / or the torque of the robotic arm is greater than or equal to a preset second change threshold, and / or the torque change rate of the robotic arm is greater than or equal to a preset third change threshold.

[0118] The arthroscopic robotic arm planning device provided in the embodiments of the present disclosure can execute the arthroscopic robotic arm planning method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0119] The embodiments of the present disclosure also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the arthroscopic robotic arm planning method provided by any embodiment of the present disclosure.

[0120] Figure 8 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 8 , which shows a schematic diagram of the structure of an electronic device 800 suitable for implementing the embodiment of the present disclosure. The electronic device 800 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0121] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0122] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0123] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the arthroscopic robotic arm planning method of the embodiment of the present disclosure are executed.

[0124] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0125] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0126] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0127] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains that the end of the robotic arm is in the joint cavity based on the navigation information, determines the obstacle posture information, enters the automatic mode when the preset photoelectric sensor detects that the target object is not on the power handle, obtains the current posture of the robotic arm, and obtains the pre-planned target bone channel posture, performs real-time planning based on the current posture, obstacle posture information and target bone channel posture, obtains the obstacle avoidance trajectory, and controls the robotic arm to move to the target position point corresponding to the target bone channel posture based on the obstacle avoidance trajectory.

[0128] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0129] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0131] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0132] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0133] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0134] processor;

[0135] a memory for storing instructions executable by the processor;

[0136] The processor is used to read the executable instructions from the memory and execute the instructions to implement any arthroscopic robotic arm planning method provided in the present disclosure.

[0137] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program is used to execute any of the arthroscopic robotic arm planning methods provided by the present disclosure.

[0138] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0139] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0140] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An arthroscopic robotic arm planning device, characterized in that: include: A determination module is used to obtain the position of the end of the robot arm in the joint cavity based on the navigation information and determine the obstacle posture information; An acquisition module, used to enter the automatic mode when a preset photoelectric sensor detects that the target object is not on the power handle, acquire the current posture of the robot arm, and acquire the pre-planned target bone tunnel posture; A planning module, used for performing real-time planning based on the current posture, the obstacle posture information and the target bone tunnel posture to obtain an obstacle avoidance trajectory; A control module, configured to control the robot arm to move to a target position point corresponding to the target bone path posture based on the obstacle avoidance trajectory; Wherein, the planning module is specifically used for: Determine a starting position point based on the current posture; Determine a candidate number of obstacle position points based on the obstacle posture information; Determining the target position point based on the target bone tunnel posture; The target number control point is on the obstacle, and when the number of all current position points is less than a preset number threshold, the target number control point is determined based on the candidate number of obstacle position points; wherein the target number is less than the number threshold; Determine an N-order high-order curve as the obstacle avoidance trajectory based on the starting position point, the target number control point and the target position point; wherein N is a positive integer greater than or equal to 4; Wherein, determining the target number of control points based on the candidate number of obstacle position points includes: Determining a supplementary quantity based on a difference between the candidate quantity and the target quantity of control points; At least one obstacle position point is determined from the candidate number of obstacle position points and supplemented according to the supplementary number to obtain the target number of control points.

2. The arthroscopic robotic arm planning device according to claim 1, characterized in that: Also includes: A first switching acquisition module, configured to switch to a manual mode when the photoelectric sensor detects that the target object is on the power handle, and to acquire a real-time external force value in any target direction of the robotic arm based on a preset first force sensor; A first updating module, used for controlling the robotic arm to update the moving trajectory when the real-time external force value in any target direction of the robotic arm is greater than or equal to a preset first threshold; A second switching acquisition module is used to switch to the automatic mode when the photoelectric sensor detects that the target object is not on the power handle, and to acquire the current updated posture of the robotic arm when the real-time external force value is less than the first threshold value; The second updating module is used to update the obstacle avoidance trajectory based on the current updated posture, the target bone channel posture and the obstacle posture information.

3. The arthroscopic robotic arm planning device according to claim 1, characterized in that: The determination module is specifically used for: The end of the robotic arm is obtained to be in the joint cavity based on the navigation information, and the position of the first obstacle is obtained based on the medical image; Obtain the second obstacle pose from the navigation host computer image; Determine a third obstacle through an arthroscopy, and acquire a third obstacle posture corresponding to the third obstacle through an external device; Acquire a fourth obstacle posture based on a pre-installed second force sensor; Obstacle pose information is determined based on the first obstacle pose, the second obstacle pose, the third obstacle pose, and the fourth obstacle pose.

4. The arthroscopic robotic arm planning device according to claim 1, characterized in that: The planning module is further specifically used for: When the number of all current position points is greater than or equal to a preset number threshold, determining a target number of obstacle position points from the candidate number of obstacle position points; An N-order high-order curve is determined as the obstacle avoidance trajectory based on the target number of control points corresponding to the starting position point and the target number of obstacle position points.

5. The arthroscopic robotic arm planning device according to claim 1, characterized in that: Also includes: A control stop module is used to control the robotic arm to stop moving when the real-time external force value of any target direction of the robotic arm obtained based on the first force sensor is greater than or equal to a preset second threshold, and / or the change value of the real-time external force value is greater than or equal to a preset first change threshold, and / or the torque of the robotic arm is greater than or equal to a preset second change threshold, and / or the torque change rate of the robotic arm is greater than or equal to a preset third change threshold.

6. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement: Based on the navigation information, the end of the robotic arm is located in the joint cavity, and the obstacle position information is determined; When the preset photoelectric sensor detects that the target object is not on the power handle, it enters the automatic mode, obtains the current posture of the robot arm, and obtains the pre-planned target bone tunnel posture; Perform real-time planning based on the current posture, the obstacle posture information and the target bone tunnel posture to obtain an obstacle avoidance trajectory; Control the robot arm to move to the target position point corresponding to the target bone path posture based on the obstacle avoidance trajectory; The real-time planning based on the current posture, the obstacle posture information and the target bone tunnel posture to obtain the obstacle avoidance trajectory includes: Determine a starting position point based on the current posture; Determine a candidate number of obstacle position points based on the obstacle posture information; Determining the target position point based on the target bone tunnel posture; The target number control point is on the obstacle, and when the number of all current position points is less than a preset number threshold, the target number control point is determined based on the candidate number of obstacle position points; wherein the target number is less than the number threshold; Determine an N-order high-order curve as the obstacle avoidance trajectory based on the starting position point, the target number control point and the target position point; wherein N is a positive integer greater than or equal to 4; Wherein, determining the target number of control points based on the candidate number of obstacle position points includes: Determining a supplementary quantity based on a difference between the candidate quantity and the target quantity of control points; At least one obstacle position point is determined from the candidate number of obstacle position points and supplemented according to the supplementary number to obtain the target number of control points.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute: Based on the navigation information, the end of the robotic arm is located in the joint cavity, and the obstacle position information is determined; When the preset photoelectric sensor detects that the target object is not on the power handle, it enters the automatic mode, obtains the current posture of the robot arm, and obtains the pre-planned target bone tunnel posture; Perform real-time planning based on the current posture, the obstacle posture information and the target bone tunnel posture to obtain an obstacle avoidance trajectory; Control the robot arm to move to the target position point corresponding to the target bone path posture based on the obstacle avoidance trajectory; The real-time planning based on the current posture, the obstacle posture information and the target bone tunnel posture to obtain the obstacle avoidance trajectory includes: Determine a starting position point based on the current posture; Determine a candidate number of obstacle position points based on the obstacle posture information; Determining the target position point based on the target bone tunnel posture; The target number control point is on the obstacle, and when the number of all current position points is less than a preset number threshold, the target number control point is determined based on the candidate number of obstacle position points; wherein the target number is less than the number threshold; Determine an N-order high-order curve as the obstacle avoidance trajectory based on the starting position point, the target number control point and the target position point; wherein N is a positive integer greater than or equal to 4; Wherein, determining the target number of control points based on the candidate number of obstacle position points includes: Determining a supplementary quantity based on a difference between the candidate quantity and the target quantity of control points; At least one obstacle position point is determined from the candidate number of obstacle position points and supplemented according to the supplementary number to obtain the target number of control points.

Images