Path planning method, system and device for interventional operation robot and readable storage medium

Abstract

The embodiment of the invention provides a path planning method, system and device for an interventional operation robot and a readable storage medium. The method comprises the steps that multiple puncture positions in operation plan information are acquired; a plurality of path planning sub-areas extending in the first straight line direction are determined, the plurality of path planning sub-areas fully cover the plurality of puncture positions, and each path planning sub-area covers at least part of the plurality of puncture positions; and planning a moving path of the end effector based on the path planning sub-region.

Description

Technical Field

[0001] This specification relates to the field of robotics technology, and in particular to a path planning method, system, device, and readable storage medium for an interventional surgical robot. Background Art

[0002] Interventional procedures are a common technique in modern surgery, particularly in the field of minimally invasive surgery. For example, needle procedures, guided by imaging and other sensors, penetrate target soft tissue lesions to deliver medications, biopsies, local anesthesia, radiation therapy, and ablation. Interventional procedures are widely used in the diagnosis and treatment of organs and tissues such as the prostate, lungs, liver, kidneys, and spine.

[0003] Traditional interventional surgery has disadvantages such as the need for multiple scan verifications, long operation time, high technical requirements for doctors, and poor repeatability. There are also many problems such as direct exposure of doctors to radiation and limited operating space. The end effector carried by the interventional surgery robot can assist or replace the doctor in performing interventional surgery. However, since interventional surgery usually requires interventional operations to be performed at multiple interventional locations, and medical devices (such as puncture needles) may be left on the patient's body after the operation. Therefore, it is necessary to design a path planning method and system for the interventional surgery robot to assist in positioning, so as to improve the success rate of the interventional surgery robot in assisting or performing interventional surgery. Summary of the Invention

[0004] One embodiment of this specification provides a path planning method for an interventional surgical robot. The interventional surgical robot includes a robotic arm and an end effector connected to the end of the robotic arm. The method includes: obtaining multiple puncture locations from surgical plan information; determining multiple path planning sub-areas extending along a first straight line direction, wherein the multiple path planning sub-areas fully cover the multiple puncture locations, and each path planning sub-area covers at least a portion of the multiple puncture locations; and planning a movement path for the end effector based on the path planning sub-areas.

[0005] In some embodiments, planning the movement path of the end effector based on the path planning sub-area includes: determining a sorting result of the multiple puncture positions based on the path planning sub-area; and planning the movement path of the end effector based at least on the sorting result.

[0006] In some embodiments, determining the sorting results of the multiple puncture positions based on the path planning sub-areas includes: sorting the path planning sub-areas along a preset direction, where the preset direction is parallel to the horizontal plane and perpendicular to the first straight line direction; and sorting the puncture positions covered by each of the path planning sub-areas.

[0007] In some embodiments, the end effector includes an end clamp, which is used to clamp the puncture needle; the sorting results of the multiple puncture positions determined based on the path planning sub-area also include: determining the opening direction of the end clamp at the multiple puncture positions; and determining the preset direction based on the opening direction.

[0008] In some embodiments, sorting the puncture positions covered by each of the path planning sub-areas includes: sorting the multiple puncture positions in the same direction along the first straight line in each of the path planning sub-areas.

[0009] In some embodiments, the end effector includes an end clamp, which is used to clamp the puncture needle; the sorting of the puncture positions covered by each path planning sub-area includes: determining the opening direction of the end clamp at the multiple puncture positions; and determining the sorting of the puncture positions covered by the path planning sub-area based on the opening direction.

[0010] In some embodiments, planning the movement path of the end effector at least based on the sorting result includes: acquiring a collision model; and planning the movement path of the end effector based on the collision model and the sorting result.

[0011] In some embodiments, the movement path includes one or more sub-paths, and planning the movement path of the end effector based on the collision model and the sorting result includes: sequentially planning the sub-paths between each group of adjacent puncture positions in the sorting result.

[0012] In some embodiments, the sequential planning of the sub-path between each group of adjacent puncture positions in the sorting results includes: for each group of adjacent puncture positions, planning an avoidance point based on the front puncture position among the adjacent puncture positions, and planning a displacement path segment based on the front puncture position and the avoidance point, the displacement path segment being the path segment between the front puncture position and the avoidance point; adding the surgical instrument model corresponding to the front puncture position among the adjacent puncture positions to the collision model to update the collision model; and planning an avoidance path segment between the avoidance point and the rear puncture position among the adjacent puncture positions based on the updated collision model, the avoidance path segment and the displacement path segment constituting the sub-path.

[0013] In some embodiments, the sub-path includes an execution sub-path, the avoidance point includes an execution avoidance point, the displacement path segment includes an execution displacement path segment, the avoidance path segment includes an execution avoidance path segment, and the execution sub-path is composed of the execution displacement path segment and the execution avoidance path segment. The method also includes: when planning the execution displacement path segment fails, obtaining a first correction position of the end effector; planning the execution displacement path segment based on the first correction position and the execution avoidance point, or re-planning the execution avoidance point based on the first correction position, and planning the execution displacement path segment based on the first correction position and the execution avoidance point, or determining the first correction position as the re-planned execution avoidance point; and / or, when planning the execution avoidance path segment fails, obtaining a second correction position of the end effector; planning the execution avoidance path segment based on the second correction position and the rear puncture position among the adjacent puncture positions.

[0014] In some embodiments, the method further includes: obtaining an external safety point; planning the movement path of the end effector at least based on the sorting result, further including: planning a first movement sub-path from the initial position of the end effector to the external safety point based on the collision model; and planning a second movement sub-path between the external safety point and the first puncture position in the sorting result based on the collision model.

[0015] In some embodiments, the method further includes: determining the preset orientation based on a position of the path planning sub-area relative to the robotic arm base.

[0016] In some embodiments, the movement path includes an execution path. Planning the movement path of the end effector based on the path planning sub-region includes: moving the punctured object outside the aperture of the medical imaging device; and planning and executing the execution path outside the aperture of the medical imaging device.

[0017] One embodiment of this specification provides a path planning system for an interventional surgical robot, the interventional surgical robot comprising a robotic arm and an end effector connected to the end of the robotic arm. The system comprises: a plan acquisition module configured to acquire multiple puncture locations from surgical plan information; a first determination module configured to determine multiple path planning sub-regions extending along a first straight line direction, wherein the multiple path planning sub-regions fully cover the multiple puncture locations, and each of the path planning sub-regions covers at least a portion of the multiple puncture locations; and a path planning module configured to plan a movement path for the end effector based at least on the path planning sub-regions.

[0018] In some embodiments, the system further includes a second determination module configured to determine a sorting result of the multiple puncture positions based on the path planning sub-area; the path planning module is further configured to plan a movement path of the end effector based at least on the sorting result.

[0019] One of the embodiments of this specification provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the path planning method described in any of the above embodiments are implemented.

[0020] One of the embodiments of the present specification provides a path planning device for an interventional surgical robot, the path planning device comprising an interventional surgical robot and a processor; the interventional surgical robot comprising a robotic arm and an end effector connected to the end of the robotic arm; the processor being configured to execute: obtaining multiple puncture positions in surgical plan information; determining multiple path planning sub-areas extending along a first straight line direction, the multiple path planning sub-areas fully covering the multiple puncture positions, and each of the path planning sub-areas covering at least part of the multiple puncture positions; and planning a moving path of the end effector based on the path planning sub-areas.

[0021] One of the embodiments of this specification provides another path planning method for an interventional surgical robot, wherein the interventional surgical robot includes a robotic arm and an end effector connected to the end of the robotic arm, and the method includes: obtaining multiple puncture positions in surgical plan information; determining a sorting result of the multiple puncture positions; obtaining a collision model; and planning a moving path of the end effector based on the collision model and the sorting result.

[0022] One of the embodiments of the present specification provides a path execution method for an interventional surgical robot, the method comprising: obtaining multiple puncture positions in surgical planning information; determining multiple path planning sub-areas extending along a first straight line direction, the multiple path planning sub-areas fully covering the multiple puncture positions, and each of the path planning sub-areas covering at least part of the multiple puncture positions; planning a moving path of the end effector based on the path planning sub-areas, the moving path including an execution path; and controlling the interventional surgical robot to execute the execution path.

[0023] One of the embodiments of the present specification provides a path execution system for an interventional surgical robot, the system comprising: a plan acquisition module, configured to acquire multiple puncture positions in surgical plan information; a first determination module, configured to determine multiple path planning sub-areas extending along a first straight line direction, the multiple path planning sub-areas fully covering the multiple puncture positions, and each of the path planning sub-areas covers at least part of the multiple puncture positions; a path planning module, configured to plan the movement path of the end effector based on the path planning sub-areas, the movement path including an execution path; and a path execution module, configured to control the interventional surgical robot to execute the execution path. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0025] Figure 1 is an application scenario diagram of a surgical robot device system shown in some embodiments of this specification;

[0026] Figure 2 is a schematic diagram of the three-dimensional structure of a surgical robot device shown in some embodiments of this specification;

[0027] Figure 3 is a side structural schematic diagram of a surgical robot device shown in some embodiments of this specification;

[0028] Figure 4 is a module diagram of a path planning system for a surgical robot as shown in some embodiments of this specification;

[0029] Figure 5A is an exemplary flow chart of a path planning method for a surgical robot as shown in some embodiments of this specification;

[0030] Figure 5B is an exemplary flow chart of a path execution method of a surgical robot as shown in some embodiments of this specification;

[0031] Figure 6 This is a schematic diagram of a path planning sub-area in a top-down state in the path planning method described in some embodiments of this specification;

[0032] Figure 7 is a schematic diagram of the sorting of multiple puncture positions in a path planning sub-area in the path planning method shown in some embodiments of this specification;

[0033] Figure 8A yes Figure 7 A magnified schematic diagram of the sorting of multiple puncture locations in the middle path planning sub-area;

[0034] Figure 8B yes Figure 8A Schematic diagram of the end effector of the surgical robot shown in;

[0035] Figure 9A It is an enlarged schematic diagram of the order of multiple puncture positions within a path planning sub-area in the path planning method shown in other embodiments of this specification;

[0036] Figure 9B yes Figure 9A Schematic diagram of the end effector of the surgical robot shown in;

[0037] Figure 10 is a schematic diagram of a workflow for planning a moving path based on a path planning sub-area in a path planning method shown in some embodiments of this specification;

[0038] Figure 11 is a schematic diagram of a workflow for planning a moving path based on a sorting result in a path planning method shown in some embodiments of this specification;

[0039] Figure 12 is a schematic diagram of a workflow for planning a check sub-path in a path planning method shown in some embodiments of this specification;

[0040] Figure 13 is a schematic diagram of the overall mobile path planning method shown in some embodiments of this specification;

[0041] Figure 14 is a schematic diagram of the end retraction of the end effector of the surgical robot shown in some embodiments of this specification;

[0042] Figure 15 is a schematic diagram of a front view of the end effector of a surgical robot as shown in some embodiments of this specification when the end effector is retracted;

[0043] Figure 16 This is a schematic diagram of the retraction direction of the end effector of the surgical robot shown in some embodiments of this specification;

[0044] Figure 17A is a schematic diagram of a workflow for executing a sub-path between each group of adjacent puncture positions in the path planning method shown in some embodiments of this specification;

[0045] Figure 17B is a schematic diagram of a workflow for executing a sub-path between each group of adjacent puncture positions in the path execution method shown in some embodiments of this specification;

[0046] Figure 18is a schematic diagram of the overall workflow of a surgical robot device in an offline interventional surgery as shown in some embodiments of this specification;

[0047] Figure 19 is a schematic diagram of overall moving path planning in the path planning method shown in some embodiments of this specification;

[0048] Figure 20 is a schematic diagram of the workflow of the path planning method shown in other embodiments of this specification;

[0049] Figure 21 This is a flowchart of overall movement path planning verification in the path planning method of the surgical robot shown in some embodiments of this specification;

[0050] Figure 22 This is a workflow diagram of intraoperative movement path planning and movement path execution in the path planning method of the surgical robot shown in some embodiments of this specification;

[0051] Figure 23 This is an exemplary flowchart of another surgical robot path planning method shown in other embodiments of this specification. DETAILED DESCRIPTION

[0052] To more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly describes the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this specification. Those skilled in the art can apply this specification to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0053] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0054] As used in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not refer to the singular but also include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0055] Flowcharts are used throughout this specification to illustrate the operations performed by systems according to embodiments of this specification. It should be understood that preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0056] Interventional surgical treatment is a minimally invasive treatment performed using modern high-tech means, that is, under the guidance of medical imaging equipment, surgical instruments are introduced into the patient's body to diagnose and locally treat internal pathologies. The embodiments of this specification provide a path planning method for a surgical robot and a path planning system for a surgical robot. The surgical robots in the embodiments of this specification can all be interventional surgical robots, that is, the path planning method for the surgical robot and the path planning system for the surgical robot in the embodiments of this application can be applied to the scenario of multi-needle puncture surgery for interventional surgical robots in interventional surgery. In some embodiments of this specification, in a remote control operation scenario, the surgical robot generally receives a control signal from the main control end, and then calculates and determines the motion trajectory of the robotic arm based on the control signal, and then the robotic arm moves according to the determined motion trajectory. It should be noted that the path planning method and system for the surgical robot in the embodiments of this specification can be applied to both real-time interventional surgery and offline interventional surgery. Take offline interventional surgery as an example. During offline interventional surgery, the patient lies on a mobile bed, which carries the patient to the cavity of the medical imaging device. The patient is first scanned by the medical imaging device, and then the patient is removed from the cavity of the medical imaging device. The end of the surgical robot's robotic arm carries the end effector (for example, an auxiliary locator or end gripper) and moves to the puncture position. The end effector then performs the puncture operation (i.e., the puncture needle is inserted into the target needle track). In general offline interventional surgery, the doctor needs to perform multiple needle punctures on one or more lesions of the patient outside the cavity of the medical imaging device. The robotic arm needs to assist in positioning and complete the robotic arm path planning for all puncture needle tracks. Multiple puncture needles are inserted into multiple target needle tracks. During the operation, it is necessary to ensure that the robotic arm and the end effector do not collide with surrounding objects. Obstacles with high collision risks in the environment are mainly puncture needles left on the patient's body surface, the patient's body, the outer wall of the medical imaging device cavity, etc. Therefore, it is necessary to plan the motion path of the surgical robot for multiple puncture positions in the surgical plan information before the robotic arm moves. Therefore, the embodiments of this specification provide a path planning method and a path planning system for a surgical robot, so as to plan the path of the surgical robot in advance based on multiple puncture positions in the surgical plan information during offline interventional surgery, so as to avoid collision of the end effector during the surgery and avoid collision between multiple puncture needles.

[0057] The embodiments of this specification provide a path planning method and a path planning system for a surgical robot, which are applied to offline interventional surgery outside the cavity of a medical imaging device. The surgical robot includes a robotic arm and an end effector connected to the end of the robotic arm. The path planning method includes: obtaining multiple puncture positions in the surgical plan information; determining multiple path planning sub-areas extending along a first straight line direction, the multiple path planning sub-areas fully covering the multiple puncture positions, and each of the path planning sub-areas covers at least part of the multiple puncture positions; and planning the movement path of the end effector based on the path planning sub-areas. In some embodiments, the path planning method and the path planning system for a surgical robot provided in the embodiments of this specification can also be applied to other scenarios, for example, a scenario in which a user controls the end effector based on the planned movement path by using a control handle near the surgical robot.

[0058] Figure 1 This is an application scenario diagram of a surgical robot equipment system according to some embodiments of this specification.

[0059] The path planning method and path planning system of the surgical robot provided in this specification can be applied to a surgical robot device system. The surgical robot device system 1000 is a bionic robot system used in interventional surgery in the medical field, which can assist doctors in completing surgical operations with high medical difficulty. In some embodiments, such as Figure 1 As shown, the surgical robot device system 1000 may include a surgical robot device 100, a network 200, a terminal 300, a processing device 400 and a storage device 500. In some embodiments, the surgical robot device system 1000 can be applied to medical fields that require complex operations, such as real-time interventional surgery and offline interventional surgery. The path planning method provided in this specification is applied to offline interventional surgery. Offline interventional surgery refers to puncture surgery outside the cavity of a medical imaging device. The patient's lesion position is first scanned by a medical imaging device, and then the patient is moved out of the cavity of the medical imaging device. The end of the robotic arm carries an auxiliary locator and moves to the puncture position. The doctor then performs the puncture surgery using the auxiliary locator (an instrument clamped at the end of the robotic arm of the surgical robot for puncture auxiliary positioning). In offline interventional surgery, a surgical robot can be used to move the robotic arm to a designated position according to a preset path during surgical preparation to perform puncture assistance.

[0060] The surgical robot device 100 can perform corresponding operations according to the received instructions (e.g., control signal instructions). When the surgical robot device 100 receives data or instructions sent by other devices or system components, the surgical robot device 100 can perform surgical assistance based on the instructions (e.g., offline interventional surgical assistance). In some embodiments, the surgical robot includes at least a robotic arm and an end effector (e.g., an auxiliary positioner). When the surgical robot receives data or instructions sent by other devices or system components, the robotic arm on the surgical robot moves so that the end effector moves to the position indicated by the instruction to assist the doctor in performing the surgical action (e.g., inserting the puncture needle into the patient's body through the guidance of the auxiliary positioner). It should be noted that the end effector refers to a device provided at the end of the robotic arm for performing related operations. It can be an auxiliary device (e.g., an auxiliary positioner) for assisting surgical instruments (e.g., a puncture needle), or a component for clamping the puncture needle (e.g., an end clamp). In some embodiments, the surgical robot can also be provided with sensors to detect the kinematic parameters (e.g., position, angle, speed, etc.) during the movement of the connecting rod, and feed the kinematic parameters back to the processing device 400 or the terminal 300. In some embodiments, the surgical robot may also be provided with a camera to obtain images of the surgical robot and its environment, and send the images to the processing device 400 or the terminal 300. For more information about surgical robots, please refer to Figure 2 、 Figure 3 The relevant description is not repeated here.

[0061] The network 200 may include any suitable network capable of facilitating information and / or data exchange among the surgical robot device 100. In some embodiments, at least one component of the surgical robot device system 1000 (e.g., the surgical robot device 100, the terminal 300, the processing device 400, the storage device 500) may exchange information and / or data with at least one other component of the surgical robot device system 1000 via the network 200. For example, the processing device 400 may obtain a control signal input by a user from the terminal 300 via the network 200. In some embodiments, the network 200 may include at least one network access point. For example, the network 200 may include a wired and / or wireless network access point (e.g., a base station and / or an Internet exchange point), and at least one component of the surgical robot device system 1000 may connect to the network 200 via the access point to exchange data and / or information.

[0062] The terminal 300 can communicate and / or connect with the surgical robot device 100, the processing device 400, and / or the storage device 500. In some embodiments, the terminal 300 may include a mobile device 310, a tablet computer 320, a laptop computer 330, or the like, or any combination thereof. For example, the mobile device 310 may include a mobile control handle, a personal digital assistant (PDA), a smartphone, or the like, or any combination thereof. In some embodiments, the terminal 300 may include a display that can be used to display information or images related to the surgical procedure, such as current surgical data of the surgical robot device 100 or an image of the environment in which the surgical robot is located.

[0063] In some embodiments, the terminal 300 may include an input device. The input device may be a keyboard input, a touch screen input (e.g., with tactile or haptic feedback), a voice input, an eye tracking input, a gesture tracking input, a brain monitoring system input, an image input, a video input, or any other similar input mechanism. The input information received by the input device may be transmitted to the processing device 400 via a bus, for example, for further processing. Other types of input devices may include a cursor control device, such as a mouse, a trackball, or cursor direction keys. In some embodiments, a user may input control signals through the input device. In some embodiments, the terminal 300 may include an output device. The output device may include a display, a speaker, a printer, or the like, or any combination thereof. The output device may be used to output parameters related to the surgical robot device 100 determined by the processing device 400, etc. In some embodiments, the terminal 300 may be part of the processing device 400.

[0064] The processing device 400 can process data and / or information obtained from the surgical robot apparatus 100, at least one terminal 300, the storage device 500, or other components of the surgical robot's path planning system 2000. For example, the processing device 400 can obtain control signals from the terminal 300 for further calculation of the surgical robot's motion trajectory at the next moment. For another example, the processing device 400 can obtain system structural parameters of the surgical robot apparatus 100, such as relevant descriptive parameters of the surgical robot and medical imaging device (e.g., modeling parameters), relevant descriptive parameters of the surgical robot's robotic arm and end effector (e.g., end effector model), and relevant descriptive parameters of the patient (e.g., patient medical condition information), from the storage device 500, for further calculation of the surgical robot's path trajectory. In some embodiments, the processing device 400 can be a single server or a server group. The server group can be centralized or distributed. In some embodiments, the processing device 400 can be local or remote. For example, the processing device 400 can access information and / or data from the surgical robot apparatus 100, the storage device 500, and / or the terminal 300 via the network 200. For another example, the processing device 400 can be directly connected to the surgical robot device 100, the terminal 300, and / or the storage device 500 to access information and / or data. For another example, the processing device 400 can be installed on the surgical robot device 100. In some embodiments, the processing device 400 can be implemented on a cloud platform. For example, the cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud cloud, a multi-cloud, or any combination thereof.

[0065] The storage device 500 can store data, instructions, and / or any other information. For example, the storage device 500 can store system configuration parameters of the surgical robot device 100, etc. In some embodiments, the storage device 500 can store data obtained from the surgical robot device 100, the terminal 300, and / or the processing device 400. In some embodiments, the storage device 500 can store data and / or instructions used by the processing device 400 to execute or use to complete the exemplary methods described in this specification. In some embodiments, the storage device 500 can include a large-capacity memory, a removable memory, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. In some embodiments, the storage device 500 can be implemented on a cloud platform.

[0066] In some embodiments, the storage device 500 can be connected to the network 200 to communicate with at least one other component (e.g., the processing device 40, the terminal 300) in the surgical robot device system 1000. At least one component in the surgical robot device system 1000 can access data stored in the storage device 500 through the network 200. In some embodiments, the storage device 500 can be part of the processing device 400. In some embodiments, the processing device 400 and the storage device 500 can be integrated into the surgical robot device 100.

[0067] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this application. For those skilled in the art, various changes and modifications can be made under the guidance of the contents of this application. The features, structures, methods and other features of the exemplary embodiments described in this application can be combined in various ways to obtain additional and / or alternative exemplary embodiments. For example, the storage device 500 can be a data storage device 500 including a cloud computing platform, such as a public cloud, a private cloud, a community and a hybrid cloud. However, these changes and modifications will not deviate from the scope of this specification.

[0068] Figure 2 is a schematic diagram of the three-dimensional structure of a surgical robot device according to some embodiments of this specification; Figure 3 It is a side view structural diagram of a surgical robot device according to some embodiments of this specification.

[0069] In some embodiments, as Figure 2 、 Figure 3As shown, the surgical robot device 100 may include a surgical robot 110, a medical imaging device 120, and a mobile bed 130. In some embodiments, when the surgical robot device 100 is used to perform an offline interventional surgery, the patient lies on the mobile bed 130, and the mobile bed 130 carries the patient to move into the scanning cavity of the medical imaging device 120. After the scan is completed, the mobile bed 130 carries the patient to move outside the scanning cavity of the medical imaging device 120, and the end of the robotic arm of the surgical robot 110 moves with the end effector to a specified position (e.g., a puncture position). During an offline interventional surgery (e.g., a multi-needle puncture surgery), the doctor is required to perform multi-needle puncture treatment on one or more lesions of the patient outside the cavity of the medical imaging device 120, that is, to obtain one or more puncture positions. During the offline interventional surgery, the end effector (e.g., an auxiliary positioner) on the surgical robot 110 moves to the first puncture position for auxiliary positioning before the doctor can perform interventional actions (e.g., inserting the puncture needle from the patient's body surface to the lesion location in a preset posture through the guidance of the auxiliary positioner). After completing an interventional action, the end effector will retreat a certain distance and then move to the next puncture position until the interventional actions at all puncture positions are completed. During the operation, the robotic arm and end effector of the surgical robot 110 need to avoid the medical imaging equipment 120, the patient's phantom, and the surgical instruments on the patient's body surface (e.g., the puncture needle).

[0070] The surgical robot 110 may be a mechanical electronic device that anthropomorphizes the functions of an arm, wrist, and hand. In some embodiments, the surgical robot may include a robotic arm base, a robotic arm, and an end effector, wherein the robotic arm is connected to the robotic arm base, and the end effector is connected to the end of the robotic arm. The robotic arm base may be a fixed base or a movable base (such as a trolley). For example, Figure 2 As shown, the surgical robot 110 may include a robotic arm base 111, a robotic arm 112 connected to the robotic arm base 111, and an end effector 113 connected to the end of the robotic arm 112. In some embodiments, the robotic arm 111 may include at least two connecting rods movably connected in sequence. When the robotic arm 111 receives data or instructions sent by other devices or system components, the joints connected to the connecting rods on the robotic arm 111 move to move the robotic arm 111 to the position indicated by the instructions, thereby moving the end effector 113 to the position indicated by the instructions. In some embodiments, the end of the robotic arm 112 may be a connecting rod at the farthest end of the robotic arm 111, which is used to clamp the end effector 113 (e.g., an auxiliary positioner).

[0071] In some embodiments, the origin of the end effector 113 (e.g. Figure 16The end effector coordinate system is established based on the point O2 in the image. The position and posture of the end effector 113 can be represented by the end effector coordinate system and the modeling parameters of the end effector 113. In some embodiments, the origin of the end effector 113 can be the end position of the end effector 113. For more information about the end effector coordinate system, please refer to the description of step 4100 and will not be repeated here.

[0072] The medical imaging device 120 can be used to scan the patient and obtain medical images, so that the doctor can observe the patient's lesions and determine one or more puncture locations. In some embodiments, the medical imaging device 112 can be a magnetic resonance imaging device (MRI), a computer tomography (CT), or other image scanning device.

[0073] The mobile bed 130 is used to carry a patient, and the mobile bed 130 can move in the cavity aperture of the medical imaging device 120 to implement a medical imaging examination on the patient on the mobile bed 130 .

[0074] Figure 4 It is a module diagram of a path planning system for a surgical robot according to some embodiments of this specification.

[0075] like Figure 4 As shown, the path planning system 2000 includes a plan acquisition module 2100, a first determination module 2200, and a path planning module 2400. In some embodiments, the plan acquisition module 2100, the first determination module 2200, and the path planning module 2400 can be implemented by the processing device 400. In some embodiments, the path planning system 2000 can also include a second determination module 2300, which can be implemented by the processing device 400.

[0076] The plan acquisition module 2100 is configured to obtain multiple puncture locations from the surgical plan information. In some embodiments, the surgical plan information may include information and data required for offline interventional surgery, such as the patient's condition information, lesion location, etc. The plan acquisition module 2100 may obtain multiple puncture locations from the surgical plan information. In some embodiments, offline interventional surgery requires the simultaneous puncture of multiple surgical instruments (e.g., puncture needles). After the puncture, the surgical instruments are likely to remain temporarily on the patient's body surface. When the robotic arm plans and executes the path of the subsequent surgical instrument, the robotic arm's motion space will be affected by the previous surgical instrument. Therefore, in order to maximize the robotic arm's motion space, the puncture sequence of the multiple surgical instruments needs to be determined before the surgery officially begins. The multiple surgical instruments are path-planned and executed in the optimal order to ensure the success rate of the robotic arm's path planning during the surgery. In some embodiments, the plan acquisition module 2100 may also obtain three-dimensional information from the medical imaging device 120 and three-dimensional information from the patient. In some embodiments, the location information of the multiple puncture locations may be preset in the processing device 400 by the doctor based on the patient's condition and lesion location. In some embodiments, the three-dimensional information of the medical imaging device 120 can be preset in the processing device 400 based on the three-dimensional structure and model of the medical imaging device 120. For the location information of the multiple puncture positions and the three-dimensional information of the medical imaging device 120, please refer to the detailed description of step 3100 and will not be repeated here.

[0077] The first determining module 2200 is configured to determine a plurality of path planning sub-areas extending along a first straight line direction, wherein the plurality of path planning sub-areas fully cover a plurality of puncture positions. In some embodiments, the first straight line direction may be a direction parallel to the moving direction of the mobile bed 130 relative to the medical imaging device 120 (such as the long axis of the mobile bed 130) (such as the long axis of the mobile bed 130). Figure 6 In some embodiments, the first straight line direction may be a direction parallel to the short axis of the mobile bed 130 (eg Figure 9A ). It should be noted that the first straight line direction in the embodiments of this specification may be a direction within a horizontal plane, where the horizontal plane refers to a reference plane parallel to the bed surface of the mobile bed 130. It should be further noted that the first straight line direction in the embodiments of this specification is not limited to a specific orientation. For example, the first straight line direction may also be a direction inclined relative to the major axis or minor axis of the mobile bed 130. The specific method for determining the multiple path planning sub-areas extending along the first straight line direction can be found in the detailed description of step 3200 and is not further elaborated here.

[0078] The second determination module 2300 is configured to determine the order of the multiple puncture locations based on the path planning sub-region. In some embodiments, the order of the multiple puncture locations is determined according to a preset ordering rule, which is related to the shape of the end effector on the robotic arm, as described below for the L-shaped auxiliary positioner and I-shaped auxiliary positioner. The specific method for determining the order of the multiple puncture locations can be found in the detailed description of step 3300 and is not further described here.

[0079] The path planning module 2400 is configured to plan the movement path of the end effector based at least on the path planning sub-areas. In some embodiments, the path planning module 2400 is further configured to plan the movement path of the end effector based at least on the sorting result. The specific method for planning the movement path of the end effector 113 based at least on the sorting result can be found in the detailed description of step 3300 and is not further described here.

[0080] In some embodiments, the path planning system 2000 further includes an adjustment warning module 2500. The adjustment warning module 2500 can be configured to execute an adjustment warning operation. In some embodiments, the adjustment warning module 2500 can execute an adjustment warning operation based on a failure in simulating the movement path of the end effector. The triggering conditions for the adjustment warning operation can be found in the detailed description of step 3400 and are not further elaborated here.

[0081] It should be noted that the above description of the path planning system and its modules is for convenience only and does not limit this specification to the scope of the embodiments. It is understandable that those skilled in the art, after understanding the principles of the system, may arbitrarily combine the modules or form subsystems connected with other modules without deviating from the principles. For example, Figure 4 The plan acquisition module 2100, first determination module 2200, second determination module 2300, and path planning module 2400 disclosed herein may be different modules within a system, or a single module may implement the functions of two or more of the aforementioned modules. For another example, each module may share a storage module, or each module may have its own storage module. Such variations are within the scope of protection of this specification.

[0082] The present specification also provides a path execution system, which includes a plan acquisition module, a first determination module, a path planning module, and a path execution module. In some embodiments, the plan acquisition module, the first determination module, the path planning module, and the path execution module can be implemented by a processing device 400. The configuration of the plan acquisition module, the first determination module, and the path planning module is similar to that of the path planning system 2000, which can be seen in the previous text. Figure 4The path execution module is configured to control the surgical robot to execute the execution path.

[0083] See below for Figure 5A-Figure 22 The path planning method of surgical robot is explained in detail.

[0084] like Figure 5A As shown, some embodiments of the present specification provide a path planning method for a surgical robot, and the process 3000 of the path planning method may include the following steps. The process 3000 can be executed by a processing device (e.g., the processing device 400). For example, the process 3000 can be implemented as an instruction set (e.g., an application) stored in a memory external to, for example, a storage device 500, a surgical robot (e.g., the surgical robot 110) and accessible to a path planning system. The processing device can execute the instruction set, and when executing the instructions, it can be configured to execute the process 3000. The operational diagram of the process 3000 presented below is illustrative. In some embodiments, the process can be completed using one or more additional operations not described and / or one or more operations not discussed. In addition, Figure 5A The order in which the operations of process 3000 are illustrated and described below is not intended to be limiting.

[0085] Step 3100 , obtaining multiple puncture positions in the surgical plan information. In some embodiments, step 3100 may be performed by the processing device 400 or the plan obtaining module 2100 .

[0086] The surgical plan information may include the patient's condition information, the patient's three-dimensional information, the location of the lesion requiring surgery, etc. In some embodiments, the patient's condition information may include the patient's name, gender, height, weight, medical history, and the location of the lesion to be treated in this interventional surgery. In some embodiments, the patient's surgical plan information may be pre-stored in the processing device 400 or the information acquisition module 2100. The user (e.g., a doctor) may input the patient's name through the terminal 300 to read the patient's surgical plan information. In some embodiments, the patient's lesion location information may be determined by scanning a medical image obtained by a medical imaging device.

[0087] In some embodiments, an offline interventional procedure requires the simultaneous insertion of multiple surgical instruments (e.g., puncture needles). Based on the patient's condition and medical images obtained by scanning with a medical imaging device, and based on clinical surgical experience, the doctor can plan multiple needle tracts on the patient's body. The surgical instruments (e.g., puncture needles) enter the patient's body through these tracts. A needle tract can be understood as the surgical instrument path formed from the surgical instrument's entry point (the point where the surgical instrument contacts the patient's skin is called the entry point) to the target lesion that the surgical instrument needs to reach. Multiple needle tracts can be input by the doctor using an input device on terminal 300. For example, terminal 300 can be provided with a three-dimensional software input interface, on which the doctor can draw the posture information of multiple needle tracts, that is, determine the target posture information of the multiple surgical instruments when entering the patient's body and reaching the lesion. In some embodiments, the surgical instrument can be a puncture needle. The doctor can input the needle's first and last endpoints (e.g., the coordinates corresponding to the two endpoints) on the device. The straight line connecting the two endpoints is the target posture information of the puncture needle. After the doctor completes the needle track drawing operation, the entry point of each surgical instrument is determined. The puncture positions described in this manual all represent the entry points of the surgical instruments.

[0088] In some embodiments, in order to ensure the accuracy of the posture of the surgical instrument when it intervenes in the patient's body, after the end effector 113 is installed at the end of the robotic arm 112, the end effector 113 (for example, the guide groove of the auxiliary locator) can be adjusted to the target posture first, and then the robotic arm 112 is moved to move the end effector 113 to the preset position (i.e., the corresponding puncture position) according to the planned movement path.

[0089] The three-dimensional information of the medical imaging device 120 may refer to the three-dimensional information of the outer contour of the medical imaging device 120. In some embodiments, the medical imaging device 120 can be three-dimensionally modeled by photographing or three-dimensional scanning, so that the three-dimensional information of the medical imaging device 120 can be stored in the processing device or the plan acquisition module 2100. In some embodiments, the three-dimensional information of the medical imaging device 120 can be pre-stored in the processing device 400 or the plan acquisition module 2100. For example, the processing device 400 or the plan acquisition module 2100 may be pre-stored with various types and models of medical imaging devices. The user can select a corresponding medical imaging device through the terminal 300. The processing device 400 can then receive the user input signal from the terminal 300 via the network 200 to retrieve the three-dimensional information of the medical imaging device 120 appropriate for the current interventional procedure. When performing path planning for the surgical robot, the three-dimensional information of the medical imaging device 120 can serve as obstacle avoidance information for the robotic arm 112 and the end effector 113.

[0090] In some embodiments, the processing device 400 may include multiple processors, each of which processes a set of information simultaneously. For example, different processors may be used to acquire the posture information of the end effector 113 and the three-dimensional information of the medical imaging device 120. By simultaneously acquiring and processing multiple sets of information, the speed and real-time nature of information acquisition can be improved.

[0091] Step 3200 : Determine multiple path planning sub-areas extending along the first straight line direction, wherein the multiple path planning sub-areas fully cover multiple puncture positions. In some embodiments, step 3200 may be performed by the processing device 400 or the first determination module 2200 .

[0092] In some embodiments, multiple puncture locations are widely distributed, and each puncture location requires surgical instrument intervention. After the intervention is completed, the surgical instrument (for example, a puncture needle) will remain on the patient's body surface, and the surgical instrument after intervention will become an obstacle on the moving path of the end effector.

[0093] like Figure 6-9B As shown, the puncture area can be divided into multiple strip-shaped path planning sub-areas along the first straight line. The puncture area covers all puncture locations obtained in step 3100. That is, the multiple planned path planning sub-areas fully cover the multiple puncture locations, and each path planning sub-area covers at least a portion of the multiple puncture locations. In some embodiments, a coordinate system for the mobile bed 130 can be established as a mobile bed coordinate system. The direction of movement of the mobile bed 130 along the scanning cavity aperture of the medical imaging device 120 is set as the Z1 axis direction, the horizontal direction perpendicular to the Z1 axis is set as the X1 axis direction, and the direction perpendicular to the horizontal direction is set as the Y1 axis direction. The horizontal direction can be a direction parallel to the bed surface of the mobile bed.

[0094] like Figure 6-9B As shown, according to the structural configuration of the end effector 113 or the position of the robot base 111 relative to the mobile bed 130 in the actual scene, the first straight line direction can be the mobile bed coordinate system (i.e. Figure 6-9B The direction of the Z1 axis of the CT bed coordinate system shown in FIG may also be the direction of the X1 axis of the mobile bed coordinate system. In some embodiments, the first straight line direction may be determined based on the structural configuration of the end effector 113.

[0095] In some embodiments, as Figure 6-8BAs shown, the first linear direction is the Z1-axis direction of the mobile bed coordinate system. The puncture area is divided into multiple strip-shaped path planning sub-areas. The multiple path planning sub-areas are divided into strips of equal width along the X1-axis. The width of each strip-shaped path planning sub-area can be configured as w, which is suitable for situations where multiple needle tracts are evenly distributed. The width w of the path planning sub-area can be manually set by the surgeon based on the size of the surgical instrument, the shape of the end effector 113, and the density of the multiple puncture locations. In this embodiment, the multiple strip-shaped path planning sub-areas can be referred to as multiple columns of path planning sub-areas. When the end effector 113 is L-shaped, it is more appropriate to set the first linear direction as the Z1-axis direction of the mobile bed coordinate system. For details, see the description of step 3310 below. In some embodiments, the multiple path planning sub-areas can also be divided into strips of varying widths along the X1-axis. This is suitable for situations where multiple needle tracts are unevenly distributed. For example, areas with dense needle tracts can be divided into strips of smaller widths, while areas with sparse needle tracts can be divided into strips of larger widths.

[0096] In some embodiments, as Figure 9A As shown, the first linear direction is the X1-axis direction of the mobile bed coordinate system. The puncture area is divided into multiple strip-shaped path planning sub-areas. The multiple path planning sub-areas are divided into strips of equal width along the Z-axis. The width of each strip-shaped path planning sub-area can be configured as w. In this embodiment, the multiple strip-shaped path planning sub-areas can be referred to as multiple rows of path planning sub-areas. When the end effector 113 is I-shaped, it is more appropriate to define the first linear direction as the X1-axis direction of the mobile bed coordinate system. For specific reasons, see the description of step 3310 below.

[0097] Step 3300 : Plan a moving path of the end effector based on the path planning sub-area. In some embodiments, step 3300 may be performed by the processing device 400 or the path planning module 2400 .

[0098] In some embodiments, the movement path may include an execution path. Step 3300 may specifically include: moving the punctured object (e.g., a patient) outside the aperture of a medical imaging device (e.g., medical imaging equipment 120), and then planning and executing the execution path outside the aperture of the medical imaging device.

[0099] In some embodiments, when an offline interventional procedure includes multiple puncture locations, all puncture locations are divided and categorized into multiple strip-shaped path planning sub-regions. In some embodiments, when the multiple path planning sub-regions are arranged in multiple columns, the end effector's movement path can be planned by first planning the puncture locations within a column of path planning sub-regions away from the robotic arm base 111, and then planning the paths for the next column of path planning sub-regions until all puncture locations have been planned.

[0100] In some embodiments, when multiple path planning sub-areas are in the shape of multiple rows, the moving path of the end effector can be planned by first planning the puncture position within a row of path planning sub-areas close to the medical imaging device 120, and then planning the next row of path planning sub-areas in sequence until the paths of all puncture positions are planned.

[0101] In some embodiments, as Figure 10 As shown, an implementation method of planning a moving path of an end effector based on a path planning sub-area is provided. The workflow 3300 of the implementation method includes the following steps:

[0102] Step 3310, determine the sorting results of multiple puncture positions based on the path planning sub-area. In some embodiments, when there are multiple puncture positions in the puncture area, in order to prevent the surgical instrument that punctures first and remains on the patient's body surface from colliding with the end effector 113, the multiple puncture positions in each path planning sub-area can be sorted according to a certain preset sorting rule. The sorting of multiple puncture positions located in the same column or the same row of the path planning sub-area can be sorted according to the preset sorting rules to obtain the sorting results. The preset sorting rules can be determined based on the shape of the end effector at the end of the robotic arm 112. In some embodiments, the end effector includes an auxiliary locator, and the preset sorting rules can be determined based on the shape of the auxiliary locator. For example, see Figure 7-8A ,The small dots in the figure indicate the puncture positions.,When the shape of the end effector is L-shaped (such as Figure 8B ), the preset sorting rule can be set to sort the puncture positions on each path planning sub-area in the Z1 axis direction of the mobile bed coordinate system in the order from outside the scanning cavity aperture of the medical imaging device to inside the scanning cavity aperture of the medical imaging device (for example, when the positive direction of the Z1 axis points to the inside of the aperture, the puncture positions are sorted in ascending order based on the Z1 coordinate value). At the same time, the puncture positions on adjacent path planning sub-areas are sorted in the order from away from the robot arm base 111 to close to the robot arm base 111 (that is, the puncture positions are sorted in ascending order based on the X1 coordinate value). This sorting method can be called an N-shaped sorting method. For another example, see Figure 9A , the shape of the end effector is I-shaped (such as Figure 9B), the preset sorting rule can be set to sort the puncture positions on each path planning sub-area in the X1-axis direction of the mobile bed coordinate system in the order from farthest from the robot arm base 111 to closer to the robot arm base 111 (for example, when the positive direction of the X1-axis points to the base, the puncture positions are sorted in ascending order based on the X1 coordinate value). At the same time, the puncture positions on adjacent path planning sub-areas are sorted in the order from inside the scanning cavity aperture of the medical imaging device to outside the scanning cavity aperture of the medical imaging device (that is, the puncture positions are sorted in descending order based on the Z1 coordinate value). This sorting method can be called a Z-shaped sorting method.

[0103] In some embodiments, the end effector includes an end clamp, and the preset sorting rule can also be determined based on the opening direction of the end clamp. Specifically, the opening direction of the end clamp at multiple puncture positions can be determined first, and the sorting of the puncture positions covered by one or more path planning sub-areas can be determined based on the opening direction. For example, the puncture area includes multiple puncture positions, and each puncture position corresponds to the opening direction of the end clamp assembly of a robotic arm 112. Several puncture positions on each path planning sub-area correspond to the opening directions of the end clamp assemblies of several robotic arms 112. If the opening directions of most end clamp assemblies in the puncture area are toward the inside of the aperture, the preset sorting rule is to sort in the order from inside the scanning cavity aperture of the medical imaging device to outside the scanning cavity aperture of the medical imaging device (for example, when the positive direction of the Z1 axis points to the inside of the aperture, the puncture positions are sorted in descending order based on the Z1 coordinate value). If the openings of most end clamp assemblies in the puncture area are directed toward the outside of the aperture, the preset sorting rule is to sort in the order from outside the scanning cavity aperture of the medical imaging device to inside the scanning cavity aperture of the medical imaging device (for example, when the positive direction of the Z1 axis points into the aperture, the puncture position is sorted in ascending order on the Z1 coordinate value).

[0104] In some embodiments, the opening orientation of the end gripper assembly of the robot arm 112 can be determined by the following method: Figure 16For any puncture position, the position of the origin O2 of the end effector coordinate system at that puncture position is determined based on the needle entry point and target point of the end effector 113 at that puncture position (for example, O2 is the needle entry point at that puncture position) and the orientation of the Z2 axis in the end effector coordinate system (for example, the positive direction of the Z2 axis is from the target point at that puncture position to the needle entry point). Any direction in the plane passing through the origin O2 of the end effector coordinate system and perpendicular to the Z2 axis is taken as the Y2 axis orientation. The X2 axis orientation is determined based on the orientations of the Z2 and Y2 axes, and then the rationality of the end effector coordinate system (before the end effector is retracted) is verified using inverse kinematics. If it is not rational, the Z2 axis of the end effector coordinate system is rotated by the end of the robotic arm 112 to continue searching for a new end effector coordinate system orientation. If it is rational, the position and posture of the end effector coordinate system are determined, and the X2 axis in the end effector coordinate system is the opening orientation of the end gripper assembly.

[0105] In some embodiments, the sorting results of multiple puncture positions determined based on the path planning sub-areas (i.e., step 3310) may include the following steps: sorting the path planning sub-areas along a preset direction, the preset direction is parallel to the horizontal plane and perpendicular to the first straight line direction, wherein the horizontal plane may be parallel to the bed surface of the mobile bed; and sorting the puncture positions covered by each of the path planning sub-areas. It should be noted that this embodiment does not limit the order of the above steps. For example, the puncture positions covered by each of the path planning sub-areas may be sorted first, and then the path planning sub-areas may be sorted. For another example, the path planning sub-areas may be sorted first, and then the puncture positions covered by each of the path planning sub-areas may be sorted. For another example, sorting the path planning sub-areas and sorting the puncture positions covered by each of the path planning sub-areas may be performed simultaneously.

[0106] In some embodiments, the preset orientation can be determined based on the position of the path planning sub-area relative to the robot base. Figure 7-8B The preset direction can be the X1-axis direction of the mobile bed coordinate system, that is, the puncture positions can be sorted in the order from farthest from the robot arm base 111 to closer to the robot arm base 111 along the X1-axis direction. Each path planning sub-area is sorted in sequence along the preset direction (X1-axis direction), and multiple puncture positions located in the same column of path planning sub-areas are sorted according to the preset sorting rules. This sorting method can be called a quasi-N-shaped sorting method.

[0107] In some embodiments, see Figure 7-8B ,When the shape of the end effector is L-shaped, each column of path planning sub-areas is sorted in sequence along the X1-axis direction of the mobile bed coordinate system, and the multiple puncture positions located in the same column of path planning sub-areas are sorted in ascending order according to the Z1 coordinate value of the puncture position in the mobile bed coordinate system.

[0108] In some embodiments, the preset orientation can be determined based on the position of the path planning sub-area relative to the medical imaging device 120. Figure 9A , the preset direction may be the negative direction of the Z1 axis of the mobile bed coordinate system, that is, the path planning sub-areas may be sorted in the Z-axis direction in the order from the aperture close to the medical imaging device 120 to the aperture far away from the medical imaging device 120. Each path planning sub-area is sorted in sequence along the preset direction (the negative direction of the Z1 axis of the mobile bed coordinate system), and the multiple puncture positions located in the same row of path planning sub-areas are sorted according to the preset sorting rules. This sorting method may be referred to as a Z-shaped sorting method. In other embodiments, the preset direction may be the positive direction of the Z1 axis of the mobile bed coordinate system, that is, the path planning sub-areas may be sorted in the Z-axis direction in the order from the aperture far away from the medical imaging device 120 to the aperture close to the medical imaging device 120.

[0109] In some embodiments, see Figure 9A ,When the shape of the end effector is I-shaped, each column of path planning sub-areas is sorted in sequence along the Z1-axis direction of the mobile bed coordinate system, and the multiple puncture positions located in the same column of path planning sub-areas are sorted in ascending order according to the X1 coordinate value of the puncture position in the mobile bed coordinate system.

[0110] In some embodiments, the end effector includes an end clamp, which is used to clamp the puncture needle, and the preset orientation can be determined based on the opening direction of the end clamp. In some embodiments, the processing device can first determine the opening orientation of the end clamp at multiple puncture positions, and then determine the preset orientation based on the opening orientation. In some embodiments, when the opening orientation of the end clamp at most puncture positions is toward the inside of the aperture, the preset orientation can be away from the aperture (i.e., toward the outside of the aperture). In some embodiments, when the opening orientation of the end clamp at most puncture positions is toward the outside of the aperture, the preset orientation can be toward the aperture (i.e., toward the inside of the aperture).

[0111] The sorting method mentioned in this specification can be sorted by a sorting algorithm. In some embodiments, the sorting method can also be sorted by the user, or by using a sorting algorithm combined with user sorting.

[0112] In some embodiments, the preset orientation may be determined based on a position of the path planning sub-area relative to the robot arm base 111 .

[0113] In some embodiments, see Figure 6-8B, the position of the robot arm base 111 is located on the right side of the path planning sub-area, and the preset direction is the direction from small to large in the X1 coordinate value of the mobile bed coordinate system. In some embodiments, the position of the robot arm base 111 can also be located on the left side of the path planning sub-area (not shown in the figure), and the preset direction is the direction from large to small in the X1 coordinate value of the mobile bed coordinate system.

[0114] In some embodiments, in each path planning sub-area in step 3310 (ie, in the same column or the same row), multiple puncture positions are sorted along the same direction of the first straight line to obtain a sorting result.

[0115] In some embodiments, see Figure 8A In this embodiment, the direction of the first straight line is the Z1 axis direction of the mobile bed coordinate system. When sorting multiple puncture positions in the same column path planning sub-area, they can be sorted according to the same direction, such as Figure 8A As shown by the dashed arrows, the multiple puncture positions in each column of the path planning sub-area can be sorted from small to large (or from large to small) based on the Z1 coordinate value of the mobile bed coordinate system. It should be noted that in this embodiment, when sorting the puncture positions in each path planning sub-area, they are all sorted in the same direction.

[0116] In some embodiments, see Figure 9A In this embodiment, the first straight line direction is the X1-axis direction of the mobile bed coordinate system. When the robotic arm 112 is located on the right side of the puncture area, the multiple puncture positions in the same row of the path planning sub-area can be sorted in ascending order based on their X1 coordinate values ​​in the mobile bed coordinate system. When the robotic arm 112 is located on the left side of the puncture area, the multiple puncture positions in the same row of the path planning sub-area can be sorted in descending order based on their X1 coordinate values ​​in the mobile bed coordinate system.

[0117] In each column or row of the path planning sub-region, the moving paths between the puncture positions may also be planned one by one according to a certain fixed direction, and it is not necessary to sort the multiple puncture positions.

[0118] Step 3320: Plan a movement path of the end effector based at least on the sorting result.

[0119] In some embodiments, the movement path of the end effector may include one or more sub-paths. In some embodiments, the path between each two adjacent puncture positions in the sorting results may be considered a sub-path. Adjacent puncture positions in the sorting results refer to two puncture positions that are immediately adjacent in sequence in the sorting results.

[0120] In some embodiments, as Figure 11 As shown, after determining the sorting results of the plurality of puncture positions, a specific method workflow 3320 for planning the movement path of the end effector based on the sorting results may include the following steps:

[0121] Step 3321: Acquire a collision model. The collision model can also be understood as acquiring obstacle avoidance information on the movement path of the end effector 113. In some embodiments, step 3321 can be performed by the processing device 400 or the plan acquisition module 2100.

[0122] The collision model mainly includes the three-dimensional information of the medical imaging device 120, the three-dimensional information of the patient's body, and / or the three-dimensional information of other medical devices near the puncture area. The method for obtaining the collision model is described in the above step 3100.

[0123] Step 3322 : Plan the movement path of the end effector 113 based on the collision model and the sorting result. In some embodiments, step 3322 may be performed by the processing device 400 or the path planning module 2400 .

[0124] In some embodiments, the path planning algorithm used in this specification can be the joint linear interpolation algorithm MoveJ, or other path planning algorithms (such as A* algorithm, Dijkstra algorithm, D algorithm, artificial potential field method, etc.), or an algorithm based on sampling search (such as PRM algorithm, RRT algorithm, etc.), or an intelligent bionic path planning algorithm (such as neural network algorithm, ant colony algorithm, genetic algorithm, etc.), or other path planning algorithms, which are not limited here.

[0125] In some embodiments, step 3322 may include sequentially planning a subpath between each group of adjacent puncture locations in the sorted results. In some embodiments, considering that during actual interventional surgery, after a surgical robot performs a puncture operation at each puncture location, a medical device (e.g., a puncture needle) typically remains at that puncture location. To prevent interference with the remaining medical device during subsequent movement of the surgical robot, avoidance points may be planned for the subpaths, and the subpaths may be divided into two path segments using the avoidance points. In some embodiments, the two path segments of the subpath divided by the avoidance points may be a displacement path segment and an avoidance path segment. Specifically, the method for sequentially planning the subpath between each group of adjacent puncture locations in the sorted results may include: for each group of adjacent puncture locations, planning an avoidance point based on a preceding puncture location among the adjacent puncture locations; and planning a displacement path segment between the preceding puncture location and the avoidance point based on the preceding puncture location and the avoidance point; adding the surgical instrument model corresponding to the preceding puncture location among the adjacent puncture locations to a collision model to update the collision model; and planning an avoidance path segment between the avoidance point and a following puncture location among the adjacent puncture locations based on the updated collision model.

[0126] In some embodiments, the movement path planned for the end effector 113 in step 3322 includes a verification path, which may include one or more verification sub-paths. After determining the sorting results for the multiple puncture positions, a simulation is performed on the movement path of the end effector 113, which was planned based on the collision model and sorting results in step 3322. This is to verify the overall movement path plan. This operation process may be referred to as path verification. Based on the planned movement path, it is necessary to ensure that the overall path can successfully reach all puncture positions in sequence. The overall movement path must pass the verification path in its entirety before proceeding to the subsequent intraoperative stage.

[0127] In some embodiments, planning the verification path includes sequentially planning the verification sub-path between each group of adjacent puncture positions in the sorting result. For a specific method of sequentially planning the verification sub-path between each group of adjacent puncture positions in the sorting result, refer to Figure 12 As shown in process 4000 , process 4000 may be executed by the processing device 400 or the path planning module 2400 .

[0128] Verify the shift path segment In some embodiments, such as Figure 12 As shown, the method flow 4000 for verifying the sub-path may include the following steps:

[0129] Step 4100 verifies the shift path segments and avoidance points based on the preceding puncture position plan among the adjacent puncture positions. Adjacent puncture positions refer to the two adjacent puncture positions in the sorted results of the multiple puncture positions obtained in step 3310. In this specification, the preceding puncture position refers to the puncture position with the preceding puncture position in each group of adjacent puncture positions, and the following puncture position refers to the puncture position with the following puncture position in each group of adjacent puncture positions.

[0130] In order to prevent uncontrolled interference and collision between the end effector 113 and the surgical instrument 114 (e.g., puncture needle) left on the patient's body surface after the intervention, the end effector 113 (e.g., auxiliary locator) can be moved horizontally for a distance in the opposite direction of the opening of the positioning groove when moving from the front puncture position to the rear puncture position in the adjacent puncture positions. This process can be called end retraction. Figure 14As shown, Figures (a) to (c) are schematic diagrams of the end retraction process observed from a frontal perspective. Figure (a) is a state diagram when the end effector 113 (for example, an auxiliary locator) reaches the target position. The end effector 113 is mounted and clamped at the end of the robotic arm 112. The end of the end effector 113 (for example, an auxiliary locator) has a positioning groove, which is used to locate the direction of the needle track when the surgical instrument 114 (for example, a puncture needle) is puncturing. Figure (b) is a state diagram of the end effector 113 during end retraction. After the surgical instrument 114 completes the puncture, the end effector 113 is horizontally retracted for a distance in the opposite direction of the opening of the positioning groove of the auxiliary locator (that is, the end retraction direction in Figure (b)). This can avoid a collision between the end effector 113 and the surgical instrument 114 remaining on the patient's body surface. This retraction path can be called a verification shift path segment. Figure (c) is a state diagram after the end effector 113 completes the retraction path and a puncture needle model is added to the front puncture position. In some embodiments, as Figure 14 As shown, when the end effector 113 moves in the verification shift path segment, the posture of the end effector 113 at the end of the robotic arm 112 remains unchanged, the position of the end effector 113 keeps moving horizontally, and the position of the end effector 113 moves horizontally for a distance in the opposite direction of the projection of the X2 coordinate axis of the end effector coordinate system in the horizontal plane.

[0131] In some embodiments, see Figure 13 , a check avoidance point can be determined on the path between the front puncture position and the rear puncture position (i.e. Figure 13 In other words, the verification avoidance point is a retraction point set in the verification sub-path to avoid interference between the end effector (or its model) and the surgical instrument (or its model) left at the front puncture position. The path between the front puncture position and the verification avoidance point is the verification displacement path segment (such as Figure 13 3, 5, 7 and 9 in the figure).

[0132] When planning the verification displacement path segment of the end effector 113 using the joint linear interpolation algorithm, it is necessary to first determine the position of the verification avoidance point. Figure 15 The figure shows the end effector 113 end retraction action diagram, combined with Figure 15 and Figure 16 The end retraction direction of the end effector 113 is the opposite direction of the projection p direction of the X2 axis of the end effector coordinate system in the horizontal plane, and the end retraction distance (ie, the distance of the verification shift path segment) is defined as dis.

[0133] The calculation principle of the verification displacement path segment is as follows: Assume that the transformation matrix T of the end effector coordinate system relative to the robot base coordinate system before the end effector 113 avoids is:

[0134]

[0135] The above transformation matrix T BaseToClutch represents the homogeneous transformation matrix of the end effector coordinate system relative to the robot base coordinate system before the end effector 113 verifies the displacement path segment. x 、a y 、a z represents the projection component of the X2-axis unit vector of the end effector coordinate system on the X, Y, and Z axes of the robot base coordinate system; similarly, b x 、b y 、b z c represents the projection component of the Y2-axis unit vector of the end effector coordinate system on the X, Y, and Z axes of the robot base coordinate system; x 、c y 、c z represents the projection component of the Z2-axis unit vector of the end effector coordinate system on the X, Y, and Z axes of the robot base coordinate system; and p x 、p y 、p z Represents the coordinate value of the origin of the end effector coordinate system in the robot base coordinate system.

[0136] The horizontal plane is the XOY plane of the robot base coordinate system. The avoidance distance of the end effector 113 in the X-axis direction and the Y-axis direction is:

[0137]

[0138]

[0139] Then, after the end effector 113 verifies the displacement path segment, the transformation matrix T of the end effector coordinate system relative to the robot base coordinate system is:

[0140]

[0141] The above transformation matrix T′ BaseToClutch represents the homogeneous transformation matrix of the end effector coordinate system relative to the robot base coordinate system after the end effector 113 verifies the displacement path segment. x 、a y 、a z 、b x 、b y 、b z 、c x 、c y 、c z 、p x 、p y 、p z Same as above.

[0142] T′ after the end effector 113 retreats BaseToClutch By finding the inverse solution, the target joint angle data of the end effector 113 can be obtained. The calculated target joint angle data is the position of the verification avoidance point. Then, the verification shift path segment is planned with the previous puncture position as the starting point and the verification avoidance point as the target point.

[0143] In some embodiments, the planning of the verification shift path segment can adopt a joint linear interpolation algorithm (i.e., MoveJ algorithm), which can connect a straight line path from the starting point (i.e., the front puncture position) to the target point (i.e., the verification avoidance point). If there is no obstacle between the starting point and the target point, the verification shift path segment planning is successful. If there is an obstacle between the starting point and the target point, the verification shift path segment planning fails.

[0144] In step 4200, the surgical instrument model corresponding to the previous puncture position among the adjacent puncture positions is added to the collision model to update the collision model. The collision model of the surgical instrument 114 remaining on the patient's body surface after the intervention needs to be considered in planning the movement path. Therefore, the model of the surgical instrument 114 needs to be added to the collision model at an appropriate time. This specification adopts that after the current surgical instrument 114 is punctured, the robotic arm 112 drives the end effector 113 to move horizontally in the opposite direction of the opening of the positioning slot for a distance. After the end retraction path is planned and executed, the surgical instrument model at the previous puncture position is added to the collision model to update the collision model, and then the subsequent movement path of the end effector 113 is replanned and the subsequent movement path is executed.

[0145] In some embodiments, as Figure 14 As shown, after the surgical instrument 114 completes the puncture action, the collision model of the surgical instrument is not immediately added to the collision model virtual environment (if it is added immediately, the subsequent path cannot be planned, because at the starting point of the path, the robotic arm model and the surgical instrument model are in contact, which will be judged as a collision). Instead, the end effector 113 is horizontally retreated a distance in the opposite direction of the auxiliary locator opening, and then the collision model of the surgical instrument is added to the collision model virtual environment. When the moving path is subsequently planned, the surgical instrument will be considered as obstacle avoidance information on the moving path of the end effector 113.

[0146] In some embodiments of the present specification, after the end effector retracts from the puncture position to the corresponding verification avoidance point, the collision model of the surgical instrument is added to the collision model virtual environment, which can prevent the system from misjudging collisions and ensure the smooth progress of subsequent path planning.

[0147] Step 4300 : Planning a verification avoidance path segment between the verification avoidance point and a subsequent puncture position among adjacent puncture positions based on the updated collision model. The verification avoidance path segment and the verification shift path segment constitute a verification sub-path.

[0148] In some embodiments, as Figure 13 As shown, the verification avoidance path segment between the verification avoidance point and the later puncture position among the adjacent puncture positions is represented by the first verification avoidance path segment (indicated by arrow 4), the second verification avoidance path segment (indicated by arrow 6), and the third verification avoidance path segment (indicated by arrow 8). The verification avoidance path segment and the verification shift path segment constitute a verification sub-path, and thus the verification sub-path represents the movement path from the earlier puncture position to the later puncture position among the adjacent puncture positions.

[0149] In some embodiments, the planning of the verification avoidance path segment can also adopt a joint linear interpolation algorithm (i.e., MoveJ algorithm), which can connect a straight line path from the starting point (i.e., the verification avoidance point) to the target point (i.e., the post-puncture position). If there is no obstacle between the starting point and the target point, the verification avoidance path segment planning is successful. If there is an obstacle between the starting point and the target point, the verification avoidance path segment planning fails.

[0150] In some embodiments of this specification, after determining the sorting results of multiple puncture positions, the movement path of the end effector is planned based on the collision model and the sorting results, and the overall movement path is verified. Only after the verification is successful does the intraoperative stage begin, which can effectively improve the success rate of planning and executing the intraoperative movement path. It should be noted that the above description of process 4000 is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to process 4000 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0151] Some embodiments of this specification provide a path planning method for a surgical robot. The process 3000 of the path planning method may further include the following steps:

[0152] Step 3400 : In response to any verification sub-path planning failure, output a surgical plan information adjustment prompt. In some embodiments, step 3400 may be executed by the processing device 400 or the adjustment warning module 2500 .

[0153] In some embodiments, when planning the moving path and verifying the path, there may be some abnormal situations. If there is a planning failure, the output device (such as a display screen) will issue a prompt of path planning failure, prompting the user (for example, a doctor) to manually drag the robotic arm 112 to a safe position, return to step 3100 to adjust the surgical plan information to obtain new multiple puncture positions, and restart planning the moving path of the end effector 113 until all verification sub-path plans are successful.

[0154] The path planning method for a surgical robot provided in the embodiments of this specification plans the path of the surgical robot in advance based on multiple puncture positions in the surgical plan information, which can avoid collisions between the end effector during the operation and avoid collisions between multiple surgical instruments.

[0155] In some embodiments, the movement path of the end effector 113 planned in step 3322 includes an execution path. After the movement paths of multiple puncture positions are successfully verified, the execution path is activated according to the movement path of the end effector 113 planned in step 3322 based on the collision model and the sorting results, officially entering the interventional surgery phase (referred to as "intraoperative" in this specification).

[0156] After officially entering the interventional surgery stage (abbreviated as "intraoperative"), path planning is performed again according to the path planning method described in the preoperative path planning method process 3000. The difference is that the intraoperative path planning is to plan a path and then execute this path, and then plan and execute the subsequent path until the end of the operation.

[0157] In some embodiments, as Figure 5B As shown, the processing device can plan the execution path of the surgical robot according to the path execution method process 3001 and control the surgical robot to execute the planned execution path. In some embodiments, the process 3001 may include the following steps:

[0158] Step 3101: Acquire multiple puncture positions in the surgical plan information. Step 3101 can refer to the content of step 3100 above and will not be repeated here.

[0159] Step 3201: Determine a plurality of path planning sub-areas extending along a first straight line. Step 3201 may refer to the contents of step 3200 above and will not be described in detail here.

[0160] Step 3301: Plan the movement path of the end effector based on the path planning sub-area. Step 3301 can refer to the content of step 3300 above and will not be repeated here.

[0161] Step 3401: Control the surgical robot to execute the execution path.

[0162] In some embodiments, the execution path may include one or more execution sub-paths. In some embodiments, the execution path includes an execution sub-path between each set of adjacent puncture positions in the sorting result. In some embodiments, the processing device may sequentially plan the execution sub-path between each set of adjacent puncture positions in the sorting result. For specific methods, refer to Figure 17A The workflow 5000 shown in FIG. 5 can be executed by the processing device 400 or the path planning module 2400. In some embodiments, the processing device can sequentially plan the execution sub-paths between each group of adjacent puncture positions in the sorting results, wherein each execution sub-path can include one or more path segments. The processing device can also control the surgical robot to execute the planned path segment of the execution sub-path after each path segment of the execution sub-path is planned. For specific methods, please refer to Figure 17B The workflow 5001 shown may be executed by the processing device 400 or the path planning module 2400 .

[0163] In some embodiments, the method flow 5000 for planning the execution sub-path may include the following steps:

[0164] Step 5100, for each group of adjacent puncture positions, an execution avoidance point is planned based on the front puncture position in the adjacent puncture positions, and an execution shift path segment is planned based on the front puncture position and the execution avoidance point. The execution avoidance point refers to the retraction point set in the execution sub-path to avoid interference between the end effector (or its model) and the surgical instrument (or its model) left at the front puncture position. The execution shift path segment is the last path segment required to move the end effector to the execution avoidance point. For example, the execution shift path segment can be the path segment from the front puncture position to the execution avoidance point. For another example, the execution shift path segment can be the path segment from the first correction position to the execution avoidance point. For more description of the first correction position, please refer to the following text. Among them, the method of planning the execution shift path segment and the execution avoidance point is the same as the method of planning the verification shift path segment and the verification avoidance point in step 4100, and will not be repeated here.

[0165] Step 5200: Add the surgical instrument model corresponding to the preceding puncture position among the adjacent puncture positions to the collision model to update the collision model. The operation method of this step is the same as that of the above-mentioned step 4200 and will not be repeated here.

[0166] Step 5300, based on the updated collision model, plans the execution avoidance path segment between the execution avoidance point and the rear puncture position in the adjacent puncture positions. Among them, the execution avoidance path segment and the execution shift path segment constitute the execution sub-path. The execution avoidance path segment refers to the last path segment in the process of the end effector retracting and then moving to the rear puncture position. For example, the execution avoidance path segment can be the path segment from the execution avoidance point to the rear puncture position. For another example, the execution avoidance path segment can be the path segment from the second correction position to the rear puncture position. For more description of the second correction position, please refer to the following text. The specific method of this step is the same as that of the above step 4300 and will not be repeated here.

[0167] In some embodiments, the method flow 5001 for planning and executing the execution sub-path may include the following steps:

[0168] In step 5101, for each set of adjacent puncture positions, an avoidance point is planned based on the preceding puncture position among the adjacent puncture positions, and a shift path segment is planned based on the preceding puncture position and the avoidance point. The methods for planning the shift path segment and executing the avoidance point are the same as those for planning and verifying the shift path segment and avoidance point in step 4100 and are not further described here.

[0169] Step 5201 : Execute the planned execution displacement path segment to move the end effector to the execution avoidance point.

[0170] Step 5301: Add the surgical instrument model corresponding to the preceding puncture position among the adjacent puncture positions to the collision model to update the collision model. The operation method of this step is the same as that of the above-mentioned step 4200 and will not be repeated here.

[0171] Step 5401 plans an avoidance path segment between the avoidance point and the subsequent puncture point among the adjacent puncture points based on the updated collision model. The avoidance path segment and the shift path segment constitute an execution subpath. The specific method for this step is the same as that for step 4300 above and will not be repeated here.

[0172] Step 5501 : executing the planned avoidance path segment to move the end effector 113 to a later puncture position among adjacent puncture positions.

[0173] In some embodiments, when the planned execution displacement path segment in steps 5100 and 5101 fails, a first correction position of the end effector 113 can be obtained. Since the planned execution displacement path segment may fail during the execution phase of each movement path during the surgical procedure, the doctor can manually drag the end effector 113 to a new relatively safe position (i.e., the first correction position) so that the subsequent path can be replanned using the first correction position as the starting point. In some embodiments, after obtaining the first correction position, the processing device can directly plan the execution displacement path segment based on the first correction position and the planned execution avoidance point, so as to move the end effector from the first correction position to the planned execution avoidance point along the execution displacement path segment. In some embodiments, after obtaining the first correction position, since the position of the end effector has changed to the first correction position, in order to avoid collision, the processing device can replan the execution avoidance point and plan the execution displacement path segment based on the first correction position and the replanned execution avoidance point, so as to move the end effector from the first correction position to the replanned execution avoidance point along the execution displacement path segment. In some embodiments, after obtaining the first calibration position, the processing device may determine whether a usable path can be directly planned from the first calibration position to the rear operating position without interfering with the medical device in the front operating position. If a usable path can be directly planned from the first calibration position to the rear operating position without interfering with the medical device in the front operating position, the processing device may determine the first calibration position as the re-planned execution avoidance point to facilitate subsequent path planning. In this case, the processor may not need to further plan the execution displacement path segment.

[0174] In some embodiments, when the planned execution of the avoidance path segment in steps 5300 and 5401 fails, the second correction position of the end effector 113 is obtained, and the avoidance path segment is planned and executed based on the second correction position and the subsequent puncture position among the adjacent puncture positions. In this case, the executed avoidance path segment is the path segment from the second correction position to the subsequent puncture position. Since the planned execution of the avoidance path segment may fail at each stage of the movement path execution during the operation, the doctor can manually drag the end effector 113 to another new relatively safe position (i.e., the second correction position) so that the subsequent path can be replanned using the second correction position as the starting point.

[0175] It should be noted that, in actual operation scenarios, the above-mentioned failure in planning the execution of the shift path segment and the failure in planning the execution of the avoidance path segment can occur simultaneously, or one of them can occur separately. When both occur simultaneously, the processing device can first process the failure in planning the execution of the shift path segment, thereby planning the execution of the shift path segment, so as to move the end effector from the first correction position to the planned execution avoidance point or the re-planned execution avoidance point, or to keep the end effector at the first correction position and determine the first correction position as the new execution avoidance point. Then, the processing device can process the failure in planning the execution of the avoidance path segment, and plan the execution of the avoidance path segment according to the second correction position of the end effector after the doctor drags it.

[0176] In some embodiments of the present specification, the end effector is allowed to perform an end retraction action at the beginning of the path from the front puncture position to the rear puncture position among two adjacent puncture positions, thereby effectively preventing uncontrollable interference and collision of surgical instruments remaining on the patient's body surface during subsequent movement of the end effector and after the intervention.

[0177] Figure 18 This is a schematic diagram of the overall workflow of the surgical robot device in offline interventional surgery as shown in some embodiments of this specification.

[0178] In some embodiments, as Figure 18 As shown, the overall workflow 6000 of the surgical robot device in offline interventional surgery may include the following steps:

[0179] In step 6100, the offline interventional surgery begins, and the medical imaging device scans and images the patient.

[0180] Step 6200: Determine multiple puncture locations based on the medical image.

[0181] Step 6300: Determine a plurality of path planning sub-areas extending along a first straight line direction.

[0182] Step 6400: Determine a ranking result of multiple puncture positions based on the path planning sub-area.

[0183] Step 6500 : planning the movement path of the end effector based on the collision model and the sorting result.

[0184] Step 6600: Verify the mobile path planning. If the verification fails, return to step 6200 until the verification succeeds.

[0185] Step 6700: Plan and execute the robot arm movement path during the operation to perform the surgical instrument intervention operation.

[0186] Step 6800: All puncture sites have completed the surgical instrument intervention operation and the operation is completed.

[0187] The specific operation methods in the overall workflow 6000 are described above and will not be repeated here.

[0188] In some embodiments, as Figure 20 As shown, after determining the sorting results of the multiple puncture positions, the specific method of planning the movement path of the end effector based on the sorting results may further include the following steps:

[0189] Step 3323 : Obtain an external safety point 121 (OS point for short). Step 3323 may be performed by the processing device 400 or the path planning module 2400 .

[0190] In some embodiments, as Figure 19 As shown, external safety point 121 can be determined based on the three-dimensional information of medical imaging device 120. External safety point 121 is located outside the scanning cavity of medical imaging device 120. In some embodiments, external safety point 121 can be preset as the mounting point of end effector 113. In some embodiments, mounting end effector 113 refers to the act of connecting and securing end effector 113 via the end of robotic arm 111. In some embodiments, external safety point 121 is set before each interventional procedure and can remain fixed throughout the entire interventional procedure.

[0191] In some embodiments, the external space of the medical imaging device 120 is relatively large, and a wide range of external safety points 121 can be selected. The determination of external safety points 121 also needs to consider the pose information of the end effector 113. Since the end effector 113 needs to be mounted to the end of the robotic arm 112 at the external safety point 121, it is necessary to determine an external safety point 121 that facilitates the installation of the end effector 113 based on the pose information of the end effector 113.

[0192] In some embodiments, the end effector 113 corresponding to the external safety point 121 is approximately 1500 mm horizontally away from the outer wall of the medical imaging device 120, making the external safety point 121 relatively far from the medical imaging device 120. The vertical distance between the lowest point of the end effector 113 corresponding to the external safety point 121 and the mobile bed 130 exceeds 500 mm, a distance significantly greater than the thickness of a patient lying flat. Therefore, the external safety point 121 provided in this embodiment is a sufficiently safe position for the end effector 113, effectively preventing collisions with environmental objects such as patients and medical imaging devices 120 when the end effector 113 is mounted on the robotic arm 112.

[0193] In some embodiments, planning the movement path of the end effector 113 based at least on the sorting result (i.e., step 3320) further includes the following steps:

[0194] Step 3324: Plan a first moving subpath from the initial position 122 of the end effector 113 to the external safety point 121 based on the collision model. The first moving subpath can be found in Figure 13 and Figure 19 The path segment indicated by arrow 1.

[0195] Step 3325: Plan a second moving sub-path between the external safety point 121 and the first puncture position in the sorting result based on the collision model. Figure 13 and Figure 19 The path segment indicated by arrow 2.

[0196] In some embodiments, see Figure 13 、 Figure 19 Before the interventional surgery is performed, the robotic arm 112 can be located at the initial point 122 (referred to as the Home point); when the interventional surgery is performed, the robotic arm 112 moves from the initial point 122 to the external safety point 121. After the end effector 113 is installed on the end of the robotic arm 112 at the external safety point 121, the end effector 113 moves along the second moving sub-path to the first puncture position in the puncture area. Then, the doctor performs the interventional action of the surgical instrument (for example, a puncture needle) under the guidance of the end effector 113. After the intervention is completed, the subsequent movement path of the end effector 113 is continued to be planned and executed until the interventional action of the surgical instrument is completed at all puncture positions. The robotic arm 112 drives the end effector 113 back to the external safety point 121. Finally, the robotic arm 112 returns to the initial point 122, and the surgery is completed.

[0197] In some embodiments, after determining the sorting results of the multiple puncture positions, it is necessary to plan the movement paths of the end effectors of all puncture positions. Figure 13 The following example illustrates this: Figure 13 The figure shows the puncture of four needles. Planning is performed from left to right based on the sorting results. The robot arm first moves from the starting point to the external safety point to install the end effector 113 (path 1), then moves to the puncture position of the first needle (path 2). After the puncture is completed, the end effector 113 retracts to the end retraction position (path 3), and then plans to the puncture position of the second needle (path 4). The paths of the next three puncture needles are then planned (path 5-path 6-path 7-path 8). After the last puncture needle is completed, the end effector 113 retracts (path 9), and finally moves to the starting point (path 10). The entire path consists of 10 segments, and all 10 segments must be planned successfully at once before entering the intraoperative execution phase. If any segment fails to be planned, the system returns a failure and prompts the user to adjust the needle path and continue to plan and verify the overall path until it succeeds.

[0198] In some embodiments, as Figure 21 As shown, the workflow 7000 for verifying the overall motion path planning in the path planning method of a surgical robot may include: starting to verify the overall motion path planning, verifying the path of the robot arm 112 from the initial point to the external safety point; verifying the path of the end effector 113 from the external safety point to the first needle puncture position, verifying the path of the end effector 113 retracting to the first needle verification avoidance point, and adding the first needle surgical instrument model; verifying the path from the first needle verification avoidance point to the second needle puncture position, verifying the path of the end effector 113 retracting to the second needle verification avoidance point, and adding the second needle surgical instrument model; and so on, until verifying the path of the end effector 113 retracting to the last needle verification avoidance point, and adding the last needle surgical instrument model; and verifying the path of the end effector 113 from the last needle verification avoidance point to the initial point. If any of the above verification paths fails, a surgical plan information adjustment prompt is output to prompt the user to readjust multiple puncture positions. If all the above paths are verified successfully, the verification is completed and the intraoperative operation process can begin.

[0199] In some embodiments, during intraoperative path planning and path execution, there are often some abnormal situations, such as Figure 22 In the workflow 8000, if there is a planning failure during path planning, the interface will prompt the user to manually drag the robot arm. The user can drag the robot arm once or multiple times, and the robot arm will re-plan the path from the first correction position after dragging to the target point of this section of the path. The user can drag repeatedly until the robot arm successfully plans and executes to the target point. The path planning algorithm after dragging is not limited. Figure 22 After the path planning is successful, the user can also drag the robotic arm while the surgical robot executes the path (for example, when the user needs to perform some other medical operations and needs to temporarily move the robotic arm away). After dragging the robotic arm, the robotic arm will re-plan and execute to the target point of this path from the second correction position after dragging. The path planning algorithm after dragging is not limited.

[0200] The path planning method for a surgical robot provided in some embodiments of this specification adopts a strategy of safe transition points, using external safe points on the overall path to transition the overall path, ensuring that the path is relatively fixed and making path planning faster and more accurate.

[0201] An embodiment of this specification provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the path planning method described in any of the above embodiments are implemented. For specific embodiments, see the relevant descriptions of workflow 3000, workflow 4000, workflow 5000, workflow 6000, workflow 7000, workflow 8000, and workflow 9000.

[0202] An embodiment of this specification provides a path planning device for a surgical robot, which includes a surgical robot and a processor. The surgical robot includes a robotic arm base, a robotic arm connected to the robotic arm base, and an end effector connected to the end of the robotic arm. The processor is configured to execute: obtaining multiple puncture positions in the surgical plan information; determining multiple path planning sub-areas extending along a first straight line direction, the multiple path planning sub-areas fully covering the multiple puncture positions, and each of the path planning sub-areas covers at least part of the multiple puncture positions; planning the movement path of the end effector based on the path planning sub-areas. For specific embodiments, please refer to the relevant descriptions of workflow 3000, workflow 4000, workflow 5000, workflow 6000, workflow 7000, workflow 8000, and workflow 9000.

[0203] Figure 23 This is an exemplary flow chart of another path planning method for a surgical robot shown in other embodiments of this specification. Figure 23 As shown, an embodiment of this specification provides another path planning method for a surgical robot, and the process 9000 of the path planning method may include the following steps.

[0204] Step 9100: Acquire multiple puncture positions in the surgical plan information. The method for acquiring multiple puncture positions is as described in step 3100 above.

[0205] Step 9200: Determine the sorting result of the multiple puncture positions. In some embodiments, the multiple puncture positions can be manually sorted according to the overall arrangement of the multiple puncture positions to obtain the sorting result.

[0206] In some embodiments, the plurality of puncture positions may be sorted according to a preset sorting rule.

[0207] In some embodiments, the preset sorting rule is to sort the puncture positions in the same direction. The preset sorting rule is to sort the puncture positions in the order from far away from the robotic arm base 111 to close to the robotic arm base 111 in the X1 axis direction of the mobile bed coordinate system (that is, sort the puncture positions in ascending order based on the X1 coordinate value). In some embodiments, the preset sorting rule can be to sort the puncture positions in the order from close to the medical imaging device to far away from the medical imaging device in the Z1 axis direction of the mobile bed coordinate system (that is, sort the puncture positions in descending order based on the Z1 coordinate value). In some embodiments, the preset sorting rule can be to sort the puncture positions in the order from far away from the medical imaging device to close to the medical imaging device in the Z1 axis direction of the mobile bed coordinate system (that is, sort the puncture positions in ascending order based on the Z1 coordinate value). In some embodiments, the preset sorting rule can be determined based on the shape of the end effector (e.g., an auxiliary positioner) at the end of the robotic arm 112. For example, when the shape of the end effector is L-shaped, the preset sorting rule can be set to sort the multiple puncture positions in the Z1-axis direction of the mobile bed coordinate system in the order from outside the scanning cavity aperture of the medical imaging device to inside the scanning cavity aperture of the medical imaging device (i.e., sorting the puncture positions in ascending order based on the Z1 coordinate value). At the same time, if there are multiple puncture positions with the same Z1 coordinate value, the multiple puncture positions with the same Z1 coordinate value are sorted in the order from farthest from the robot arm base 111 to closer to the robot arm base 111 (i.e., sorting the puncture positions in ascending order based on the X1 coordinate value). For another example, when the shape of the end effector is I-shaped, the preset sorting rule can be set to sort the multiple puncture positions in the X1-axis direction of the mobile bed coordinate system in the order from farthest from the robot arm base 111 to closer to the robot arm base 111 (i.e., sorting the puncture positions in ascending order based on the X1 coordinate value). At the same time, if there are multiple puncture positions with the same X1 coordinate value, the multiple puncture positions with the same X1 coordinate value are sorted in the order from inside the scanning cavity aperture of the medical imaging device to outside the scanning cavity aperture of the medical imaging device (that is, the puncture positions are sorted in descending order based on their Z coordinate values). It should be noted that when sorting the puncture positions in the same direction, the area where all puncture positions are located can be divided into a path planning sub-area, or they can be directly sorted without dividing the path planning sub-area.

[0208] In some embodiments, the preset sorting rule is to divide the puncture positions into multiple groups, sort each group, and then sort the puncture positions within each group, and finally obtain the sorting of all puncture positions. For example, the puncture positions can be grouped based on the direction of one of the coordinate axes (such as the X1 coordinate axis) of the mobile bed coordinate system, and the groups can be sorted based on the coordinate values ​​of the coordinate axis, and then the puncture positions within each group can be sorted based on another coordinate axis (such as the Z1 coordinate axis) of the mobile bed coordinate system, and finally obtain the sorting of all puncture positions. It should be noted that when grouping the puncture positions, the grouping can be performed by dividing the path planning sub-areas (see the previous description for details), or the grouping can be performed directly according to the range of coordinate values. This embodiment does not limit this.

[0209] Step 9300: Obtain a collision model. For how to obtain the collision model, refer to the description in step 3321 above.

[0210] Step 9400: Plan the movement path of the end effector based on the collision model and the sorting result. The planning method of the movement path is described in the above step 3322.

[0211] The beneficial effects that may be brought about by the embodiments of this specification include but are not limited to: (1) The path planning method of the surgical robot provided in this specification plans the path of the surgical robot based on multiple puncture positions in the surgical plan information in advance, which can avoid collisions between the end effector during the operation and avoid collisions between multiple surgical instruments. (2) The path planning method of the surgical robot provided in some embodiments of this specification plans the movement path of the end effector based on the collision model and the sorting results after determining the sorting results of multiple puncture positions, verifies the overall movement path, and enters the intraoperative stage only after the verification is successful, which can effectively improve the success rate of planning and execution of the intraoperative movement path. (3) The path planning method of the surgical robot provided in some embodiments of this specification allows the end effector to perform an end retraction action on the first section of the path from the front puncture position to the rear puncture position of two adjacent puncture positions, effectively preventing uncontrollable interference and collision of surgical instruments left on the patient's body surface during subsequent movement of the end effector and after the intervention. (4) The path planning method for the surgical robot provided in some embodiments of this specification, after the end effector retracts from the end of the puncture position to the corresponding verification avoidance point, adds the collision model of the surgical instrument to the collision model virtual environment, which can prevent the system from misjudging collisions and ensure the smooth progress of subsequent path planning. (5) The path planning method for the surgical robot provided in some embodiments of this specification adopts a safe transition point strategy, using external safe points on the overall path to transition the overall path, ensuring that the path is relatively fixed and making path planning faster and more accurate.

[0212] It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other possible beneficial effects.

[0213] While the basic concepts have been described above, it will be apparent to those skilled in the art that the detailed disclosure is merely illustrative and does not limit this specification. Although not explicitly stated herein, various modifications, improvements, and revisions to this specification may be made by those skilled in the art. Such modifications, improvements, and revisions are suggested in this specification and remain within the spirit and scope of the exemplary embodiments of this specification.

[0214] This specification also uses specific terms to describe the embodiments of this specification. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "one embodiment," "an embodiment," or "an alternative embodiment" two or more times in different locations in this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of this specification may be appropriately combined.

[0215] Similarly, it should be noted that, in order to simplify the presentation of this specification and thus facilitate understanding of one or more embodiments of the invention, the foregoing descriptions of the embodiments of this specification sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not imply that the subject matter of this specification requires more features than those recited in the claims. In fact, an embodiment may have fewer features than all of the features of a single disclosed embodiment.

[0216] In some embodiments, the numerical parameters used in the specification and claims are approximate values, which may vary depending on the desired features of individual embodiments. In some embodiments, numerical parameters should take into account the specified number of significant digits and adopt a general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0217] Finally, it should be understood that the embodiments described in this specification are intended only to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly described and illustrated in this specification.

Claims

1. A path planning method for an interventional surgical robot, wherein the interventional surgical robot comprises a robotic arm and an end effector connected to the end of the robotic arm, characterized in that: The method comprises: Acquiring multiple puncture positions in surgical planning information; determining a plurality of path planning sub-areas extending along a first straight line direction, wherein the plurality of path planning sub-areas fully cover the plurality of puncture positions, and each of the path planning sub-areas covers at least a portion of the plurality of puncture positions; A moving path of the end effector is planned based on the path planning sub-area.

2. The path planning method according to claim 1, characterized in that: Planning the movement path of the end effector based on the path planning sub-area includes: Determining a ranking result of the plurality of puncture positions based on the path planning sub-area; A moving path of the end effector is planned based at least on the sorting result.

3. The path planning method according to claim 2, characterized in that: The determining of the sorting results of the multiple puncture positions based on the path planning sub-area includes: sorting the path planning sub-areas along a preset direction, wherein the preset direction is parallel to a horizontal plane and perpendicular to the first straight line direction; and The puncture positions covered by each of the path planning sub-areas are sorted.

4. The path planning method according to claim 3, characterized in that: The end effector includes an end clamp, and the end clamp is used to clamp the puncture needle; The determining of the sorting results of the multiple puncture positions based on the path planning sub-areas further includes: determining the orientation of the opening of the end holder at the plurality of puncture locations; The preset orientation is determined based on the opening orientation.

5. The path planning method according to claim 3, characterized in that: The sorting of the puncture positions covered by each of the path planning sub-areas includes: In each of the path planning sub-areas, the puncture positions covered by the path planning sub-area are sorted along the same direction of the first straight line.

6. The path planning method according to claim 3, characterized in that: The end effector includes an end clamp, and the end clamp is used to clamp the puncture needle; The sorting of the puncture positions covered by each of the path planning sub-areas includes: determining the orientation of the opening of the end holder at the plurality of puncture locations; The puncture position order covered by the path planning sub-area is determined based on the opening orientation.

7. The path planning method according to claim 2, characterized in that: The step of planning the movement path of the end effector at least based on the sorting result includes: Get the collision model; A moving path of the end effector is planned based on the collision model and the sorting result.

8. The path planning method according to claim 7, characterized in that: The movement path includes one or more sub-paths; and planning the movement path of the end effector based on the collision model and the sorting result includes: The subpaths between each group of adjacent puncture positions in the sorting results are planned in sequence.

9. The path planning method according to claim 8, characterized in that: The sequentially planning of subpaths between each group of adjacent puncture positions in the sorting results includes: For each group of adjacent puncture positions, a avoidance point is planned based on a preceding puncture position among the adjacent puncture positions, and a displacement path segment is planned based on the preceding puncture position and the avoidance point, wherein the displacement path segment is a path segment between the preceding puncture position and the avoidance point; adding a surgical instrument model corresponding to a front puncture position among the adjacent puncture positions to the collision model to update the collision model; An avoidance path segment between the avoidance point and a later puncture position among the adjacent puncture positions is planned based on the updated collision model, and the avoidance path segment and the displacement path segment constitute the sub-path.

10. The path planning method according to claim 9, characterized in that: The subpath includes an execution subpath, the avoidance point includes an execution avoidance point, the displacement path segment includes an execution displacement path segment, the avoidance path segment includes an execution avoidance path segment, the execution subpath is composed of the execution displacement path segment and the execution avoidance path segment, and the method further includes: When planning the execution displacement path segment fails, obtaining a first correction position of the end effector; planning an execution shift path segment based on the first corrected position and the execution avoidance point, or re-planning the execution avoidance point based on the first corrected position and planning an execution shift path segment based on the first corrected position and the re-planned execution avoidance point, or determining the first corrected position as the re-planned execution avoidance point; and / or, When planning the execution avoidance path segment fails, obtaining a second corrected position of the end effector; An avoidance path segment is planned and executed based on the second correction position and a subsequent puncture position among the adjacent puncture positions.

11. The path planning method according to claim 7, characterized in that: The method further comprises: Get external security points; The step of planning the movement path of the end effector at least based on the sorting result further includes: Planning a first movement subpath from the initial position of the end effector to the external safety point based on the collision model; and A second movement subpath between the external safety point and the first puncture position in the sorting result is planned based on the collision model.

12. A path planning system for an interventional surgical robot, the interventional surgical robot comprising a robotic arm and an end effector connected to the end of the robotic arm, characterized in that: The system comprises: a plan acquisition module, configured to acquire multiple puncture positions in the surgical plan information; a first determining module configured to determine a plurality of path planning sub-areas extending along a first straight line direction, wherein the plurality of path planning sub-areas fully cover the plurality of puncture positions, and each of the path planning sub-areas covers at least a portion of the plurality of puncture positions; A path planning module is configured to plan a moving path of the end effector based at least on the path planning sub-area.

13. The system according to claim 12, wherein: The system further comprises: a second determining module, configured to determine the order of the plurality of puncture positions based on the path planning sub-area; The path planning module is further configured to plan a moving path of the end effector based at least on the sorting result.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the path planning method according to any one of claims 1 to 11 are implemented.

15. A path planning device for an interventional surgical robot, characterized by: The path planning device includes an interventional surgical robot and a processor; the interventional surgical robot includes a robotic arm and an end effector connected to an end of the robotic arm; the processor is configured to: obtain multiple puncture positions in surgical plan information; determine multiple path planning sub-areas extending along a first straight line direction, the multiple path planning sub-areas fully covering the multiple puncture positions, and each of the path planning sub-areas covering at least a portion of the multiple puncture positions; A moving path of the end effector is planned based on the path planning sub-area.

Images