Interventional medical system

Abstract

The embodiment of the invention provides an interventional medical system, the system is in data connection with a medical imaging system, the medical imaging system comprises a scanning device and a scanning bed, the system comprises an interventional surgical robot system and a processor, and the interventional surgical robot system comprises a mechanical arm and an end effector. The processor is configured to obtain first position information of the scanning device, second position information of the scanning bed and to-be-executed target operation plan information, wherein the target operation plan information comprises a plurality of target operation positions and a plurality of target operation channels corresponding to the target operation positions; and sequentially planning and executing an execution sub-path between each group of adjacent target operation positions, namely controlling the scanning equipment, the scanning bed and the mechanical arm to be linked to execute the sub-path based on the inclination of a target operation channel corresponding to the target operation position included in the execution sub-path to be executed relative to the second direction.

Description

Technical Field

[0001] This specification relates to the medical field, and in particular to an interventional medical system. Background Art

[0002] In the current medical field, the use of robotic manipulators for interventional surgery is becoming increasingly common. In interventional multi-needle puncture procedures, the procedure often needs to be performed within the scanning bore (i.e., cavity) of a scanning device, such as a computed tomography (CT) device or a magnetic resonance imaging (MRI) device. Taking a CT device as an example, after performing a CT scan of the patient's location, a preoperative simulation is performed to determine the needle tract position. The robotic arm then grips the puncture needle and moves it to the interventional puncture point. The CT scan performs real-time exposure, and the doctor can observe the entire needle insertion process outside the CT room through the real-time exposure. However, since the effectiveness of preoperative simulation is difficult to guarantee, during actual surgery, the end of the robotic arm can easily collide with other objects, such as the patient, an existing puncture needle, or the scanning device. Furthermore, during real-time exposure, in some cases, the puncture needle's insertion trajectory cannot be seen. For example, during chest or abdominal puncture procedures, the needle tract may be tilted to the head and feet, which makes the surgery more difficult and affects the surgical outcome.

[0003] Therefore, it is hoped to provide an interventional medical system to ensure the effectiveness of interventional multi-needle puncture surgery. Summary of the Invention

[0004] One of the embodiments of this specification provides an interventional medical system, which is data-connected to a medical imaging system, wherein the medical imaging system includes a scanning device and a scanning bed, and the interventional medical system includes: an interventional surgical robot system and a processor, wherein the interventional surgical robot system includes a robotic arm and an end effector, and the processor is configured to: obtain first position information of the scanning device, second position information of the scanning bed, and target surgical plan information to be performed, wherein the target surgical plan information includes multiple target operation positions and multiple target operation channels corresponding to the multiple target operation positions; sequentially plan and execute an execution sub-path between each group of adjacent target operation positions, wherein at least a portion of each of the execution sub-paths is executed within a scanning hole of the scanning device, and the sequential planning and execution of the execution sub-path between each group of adjacent target operation positions includes: based on the inclination of the target operation channel corresponding to the target operation position included in the execution sub-path to be executed relative to a second direction, controlling the scanning device, the scanning bed, and the robotic arm to link and execute the sub-path.

[0005] In some embodiments, based on the inclination of the rear target operating channel corresponding to the rear target operating position among the adjacent target operating positions relative to a second direction, the robotic arm and the end effector can be controlled to withdraw from the front target operating position among the adjacent target operating positions from the scanning aperture of the scanning device to a safe position, where the second direction is parallel to the long axis direction of the scanning bed; the scanning bed is controlled to withdraw from the scanning aperture of the scanning device, and then the scanning device is controlled to rotate in the second direction so that the scanning aperture of the scanning device is inclined by a second angle relative to the second direction, where the second angle corresponds to the inclination angle of the rear target operating channel relative to the second direction; the scanning bed is controlled to move at least along the second direction and enter the scanning aperture of the scanning device so that the rear target operating channel is within the scanning range of the scanning device; and the robotic arm and the end effector are planned and controlled to execute the execution sub-path from the safe position to the rear target operating position.

[0006] In some embodiments, based on the inclination of the target operating channel corresponding to the first target operating position in the target surgical plan information relative to a second direction, the scanning device can be controlled to rotate in the second direction so that the scanning hole of the scanning device is inclined at a first angle relative to the second direction, and the first angle corresponds to the inclination angle of the target operating channel corresponding to the first target operating position relative to the second direction, and the second direction is parallel to the long axis direction of the scanning bed; the scanning bed is controlled to move at least along the second direction and enter the scanning hole of the scanning device so that the target operating channel corresponding to the first target operating position is within the scanning range of the scanning device; the robotic arm and the end effector are planned and controlled to execute the execution sub-path from the initial safe position to the first target operating position.

[0007] In some embodiments, based on the inclination of the rear target operating channel corresponding to the rear target operating position among the adjacent target operating positions relative to a second direction, and the front target operating channel corresponding to the front target operating position among the adjacent target operating positions and the rear target operating channel are located in the same scanning layer, and the front target operating channel and the rear target operating channel have different inclination angles relative to the second direction, the scanning device can be controlled to rotate toward the second direction, so that the scanning hole of the scanning device is inclined by a second angle relative to the second direction, so that the rear target operating channel is within the scanning range of the scanning device, the second angle corresponds to the inclination angle of the rear target operating channel relative to the second direction, and the second direction is parallel to the long axis direction of the scanning bed; the robotic arm and the end effector are planned and controlled to execute the execution sub-path from the front target operating position to the rear target operating position.

[0008] In some embodiments, before controlling the scanning device to rotate in the second direction so that the scanning aperture of the scanning device is tilted by a second angle relative to the second direction, the scanning device may be controlled to perform a gantry zeroing operation.

[0009] In some embodiments, based on the inclination of the rear target operation channel corresponding to the rear target operation position among the adjacent target operation positions relative to a second direction, and the front target operation channel corresponding to the front target operation position among the adjacent target operation positions and the rear target operation channel are located in the same scanning layer, the robotic arm and the end effector can be planned and controlled to execute the execution sub-path from the front target operation position to the rear target operation position, where the second direction is parallel to the long axis direction of the scanning bed.

[0010] In some embodiments, a collision model may be determined based on the first position information and the second position information, where the collision model includes a patient model, a scanning device model, and a robotic arm end model.

[0011] In some embodiments, after controlling the rotation of the scanning device and / or controlling the movement of the scanning bed, the collision model and the position information of the subsequent target operation position in the adjacent target operation positions can be updated; and the execution sub-path is planned based on the updated collision model and the position information of the subsequent target operation position.

[0012] In some embodiments, the processor can also be configured to: plan the movement path of the end effector based on the first position information, the second position information and the surgical plan information; and determine the target surgical plan information in response to the successful planning of the movement path of the end effector.

[0013] In some embodiments, the movement path of the end effector may be planned based on the collision model and the sorting result of the operation positions in the surgical planning information.

[0014] One of the embodiments of this specification provides another interventional medical system, which is data-connected to a medical imaging system, wherein the medical imaging system includes a scanning device and a scanning bed, and the interventional medical system includes: an interventional surgical robot system and a processor, wherein the interventional surgical robot system includes a robotic arm, and the robotic arm includes an end effector, and the processor is configured to: obtain first position information of the scanning device, second position information of the scanning bed, and surgical plan information, wherein the surgical plan information includes at least one operating position and at least one operating channel corresponding to the at least one operating position; plan a movement path of the end effector based on the first position information, the second position information, and the surgical plan information; and determine target surgical plan information in response to successful planning of the movement path of the end effector. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0016] Figure 1 is a schematic diagram of an application scenario of an interventional medical system according to some embodiments of this specification;

[0017] Figure 2 is a schematic diagram of an interventional medical system according to some embodiments of this specification;

[0018] Figure 3 is an exemplary flow chart of a surgical path planning method according to some embodiments of this specification;

[0019] Figure 4 is an exemplary flow chart of another surgical path planning method according to some embodiments of this specification;

[0020] Figure 5 is a schematic diagram of sorting needle tracks according to some embodiments of this specification;

[0021] Figure 6 is another schematic diagram of sorting needle channels according to some embodiments of this specification;

[0022] Figure 7 is another schematic diagram of sorting needle channels according to some embodiments of this specification;

[0023] Figure 8 is a schematic diagram of an end effector of a surgical robot according to some embodiments of this specification;

[0024] Figure 9 is a schematic diagram of an end effector of another surgical robot according to some embodiments of this specification;

[0025] Figure 10 is a schematic diagram of a collision between a robot end model and a patient model resulting in a failure in path planning according to some embodiments of this specification;

[0026] Figure 11 is a schematic diagram of adjusting the needle path after path planning fails according to some embodiments of this specification;

[0027] Figure 12 is a schematic diagram of a collision between a robot end model and a scanning device model resulting in a failure in path planning according to some embodiments of this specification;

[0028] Figure 13is a schematic diagram of adjusting the needle path after another path planning failure according to some embodiments of this specification;

[0029] Figure 14 is a schematic diagram of a collision between a robot arm end model and a surgical instrument model resulting in a failure in path planning according to some embodiments of this specification;

[0030] Figure 15 is a schematic diagram of adjusting the needle path after another path planning failure according to some embodiments of this specification;

[0031] Figure 16 is a schematic diagram of a needle tract tilted to the head and foot side of a puncture without tilting the CT gantry according to some embodiments of this specification;

[0032] Figure 17 is a schematic diagram of calculating the tilt angle of a CT gantry according to some embodiments of this specification;

[0033] Figure 18 is a schematic diagram of calculating the vertical movement amount of a CT bed according to some embodiments of this specification;

[0034] Figure 19 It is a schematic diagram of calculating the horizontal movement amount of the CT bed according to some embodiments of this specification. DETAILED DESCRIPTION

[0035] To more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly describes the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this specification. Those skilled in the art can apply this specification to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0036] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0037] As used in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not refer to the singular but also include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0038] Flowcharts are used throughout this specification to illustrate the operations performed by systems according to embodiments of this specification. It should be understood that preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0039] Figure 1 It is a schematic diagram of an application scenario of a medical system according to some embodiments of this specification.

[0040] In this specification, the interventional medical system 100 is referred to as the system 100. Figure 1 As shown, in some embodiments, the system 100 may include a medical imaging system 110 , an interventional surgery robot system 120 , a processing device 130 , a terminal 140 , a storage device 150 , and a network 160 .

[0041] The medical imaging system 110 refers to a system capable of reproducing the internal structures of the human body as images. In some embodiments, the medical imaging system 110 may include a scanning device and a scanning bed. The scanning device may be any medical imaging device capable of imaging or treating a designated part of a patient's body, such as a CT device, an MR device, a positron emission tomography (PET) device, a single-photon emission computed tomography (SPECT) device, etc. The scanning bed may be used to position the patient. This specification will be described below using a CT scanning device as an example.

[0042] like Figure 2 As shown, in some embodiments, the medical imaging system 110 may include a scanning device 111 and a scanning bed 112. Figure 2 The scanning device 111 is a CT machine, and the scanning bed 112 is an operating table (the operating table corresponding to the CT machine can also be called a CT bed). The CT scans and / or treats the patient placed on the operating table. In some embodiments, the scanning device 111 may include a gantry ( Figure 2 The scanning aperture 113 is located in the center of the gantry and is a hole-like structure formed by the inner side of the gantry. When a patient is scanned, they are placed on the scanning bed 112. The rotation of the gantry and the movement of the scanning bed 112 place the patient's scan area within the scanning aperture. In some embodiments, the scanning aperture can include the cavity of a single medical imaging device or a cavity formed by multiple medical imaging devices.

[0043] In some embodiments, the medical imaging system 110 can exchange data and / or information with other components of the system 100 (e.g., the interventional surgical robot system 120, the processing device 130, the storage device 150, and the terminal 140) via the network 160. In some embodiments, the medical imaging system 110 can be directly connected to other components of the system 100. In some embodiments, one or more components of the system 100 (e.g., the processing device 130, the storage device 150) can be included in the medical imaging system 110.

[0044] The interventional surgical robotic system 120 refers to a medical robotic system capable of performing surgical treatment on patients. The interventional surgical robotic system 120 may include various types of surgical robots used to perform interventional procedures, such as percutaneous puncture surgical robots. In some embodiments, the interventional surgical robotic system 120 may include a robotic arm that can be used to position surgical instruments relative to the patient. The robotic arm is connected to the surgical instruments via an end effector. An end effector is a device used to position surgical instruments and / or connect the robotic arm to the surgical instruments, such as an end gripper or an end assist device. In some embodiments, the end effector can secure the surgical instruments to the end of the robotic arm in the surgical robotic system, fixing their position relative to the robotic arm. Surgical instruments are instruments used to perform surgery on patients and can include various types, such as puncture needles, scalpels, and laser transmitters. In some cases, because the surgical instruments are connected to the end effector, in some embodiments, the end portion of the robotic arm, the end effector, and the surgical instruments connected to the robotic arm via the end effector can be considered as a single entity, collectively referred to as the robotic arm end. In some embodiments, when a surgical instrument is connected to the end effector, the movement path of the end effector may also represent the movement path of the surgical instrument.

[0045] like Figure 2 As shown, in some embodiments, the interventional surgery robot system 120 may include a robotic arm 121 and an end effector 122. Figure 2 The robotic arm 121 in the figure is an interventional surgical robot. The end effector 122 is connected to a puncture needle. The interventional surgical robot uses the puncture needle fixed to its end to perform interventional puncture procedures on patients on the operating table. In some embodiments, the interventional surgical robotic system 120 can perform procedures on patients based on a surgical plan generated by the processing device 130, such as multi-needle puncture procedures.

[0046] As an example only, the process of interventional multi-needle puncture surgery can be as follows: first, a CT scan is performed on the patient's surgical site on the CT bed; after the scan, the user (for example, a doctor) determines all puncture needle track positions on the interface and sorts the needle tracks; a needle track simulation verification is performed; after the needle track simulation verification is passed, the operating table is moved to the planned position, and the CT frame is tilted to the planned angle corresponding to the position; the end of the surgical robot's robotic arm clamps the puncture needle and moves to the corresponding puncture needle track position according to the planned path, and the real-time CT exposure guides the user to perform the puncture surgery.

[0047] The processing device 130 can process data and / or information obtained from other devices or system components and, based on this data, information, and / or processing results, execute the surgical path planning methods described in some embodiments of this specification to accomplish one or more functions described in some embodiments of this specification. For example, the processing device 130 can plan the movement path of the end effector 122 based on the position information of the scanning device 111 and the scanning bed 112 in the medical imaging system 110 and the surgical plan information to determine the target surgical plan. For another example, the processing device 130 can prompt the user if the path planning fails. For another example, during chest or abdominal puncture procedures, for a cranio-pedis-tilted needle path, the processing device 130 can tilt the gantry of the scanning device 111 so that the user can see the puncture needle trajectory during real-time exposure of the scanning device 111. In some embodiments, the processing device 130 can send the processed data, such as the target surgical plan and the ranking results of target operating positions, to the storage device 150 for storage. In some embodiments, the processing device 130 can obtain pre-stored data and / or information from the storage device 150, such as surgical plan information, component location information, etc., for executing the surgical path planning method shown in some embodiments of this specification.

[0048] In some embodiments, the processing device 130 may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-core processing device). By way of example only, the processing device 130 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, or any combination thereof.

[0049] Terminal 140 can control surgical operations. A physician can use terminal 140 to issue operational instructions to the medical imaging system 110, causing the medical imaging system 110 to perform a specified operation, such as irradiating and imaging a specified part of a patient's body. A physician can also use terminal 140 to issue operational instructions to the interventional surgical robotic system 120, causing the interventional surgical robotic system 120 to perform a specified operation, such as executing a target surgical plan and performing surgery on a patient. In some embodiments, terminal 140 can, through instructions, cause the processing device 130 to execute the surgical path planning methods described in some embodiments of this specification. In some embodiments, terminal 140 may include a display component, through which processing device 130 can display prompts to the user, such as notifications indicating failure and / or success of surgical path planning. In some embodiments, terminal 140 can be any one of a mobile device 140-1, a tablet computer 140-2, a laptop computer 140-3, a desktop computer 140-4, or any combination thereof, having input and / or output capabilities.

[0050] The storage device 150 can store data or information generated by other devices. In some embodiments, the storage device 150 can store data and / or information collected by the medical imaging system 110, such as scanned images of target objects such as phantoms (i.e., phantoms) and patients. In some embodiments, the storage device 150 can store data and / or information processed by the processing device 130, such as target surgical plans, parameters for the tilt of the scanning device gantry, etc. The storage device 150 can include one or more storage components, each of which can be an independent device or part of another device. The storage device can be local or implemented through the cloud.

[0051] The network 160 can connect the various components of the system and / or connect the system to external resources. The network 160 enables communication between the various components and with other components outside the system, facilitating the exchange of data and / or information. In some embodiments, one or more components in the system 100 (e.g., the medical imaging system 110, the interventional surgical robot system 120, the processing device 130, the storage device 150, the terminal 140) can send data and / or information to other components via the network 160. In some embodiments, the network 160 can be any one or more of a wired network and a wireless network.

[0052] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this specification. For those skilled in the art, various changes and modifications can be made under the guidance of the contents of this specification. The features, structures, methods and other features of the exemplary embodiments described in this specification can be combined in various ways to obtain additional and / or alternative exemplary embodiments. For example, the processing device 130 can be based on a cloud computing platform, such as a public cloud, a private cloud, a community and a hybrid cloud. However, these changes and modifications do not deviate from the scope of this specification.

[0053] Figure 3 is an exemplary flow chart of a surgical path planning method according to some embodiments of this specification.

[0054] like Figure 3 As shown, process 300 includes the following steps. In some embodiments, process 300 can be executed by processing device 130. In some embodiments, the surgical path planning method shown in process 300 can be used for path planning of puncture surgery, where it can be used for single needle planning or multi-needle planning.

[0055] Step 310: Acquire first position information of the scanning device, second position information of the scanning bed, and surgical plan information, wherein the surgical plan information includes one or more operating positions and one or more operating channels corresponding to the operating positions.

[0056] The first position is the position of the scanning device. The first position information refers to information that can indicate the first position of the scanning device and can include various information, such as the horizontal and vertical positions of the scanning device gantry, the tilt angle of the scanning device gantry, etc. The second position refers to the position of the scanning bed. The second position information refers to information that can indicate the second position of the scanning bed and can include various information, such as the vertical position (bed height) and horizontal position of the scanning bed. In some embodiments, the first position can include the initial position and / or real-time position of the scanning device. In some embodiments, the second position can include the initial position and / or real-time position of the scanning bed. In some embodiments, the processing device 130 can determine the starting position of the scanning device and the scanning bed in the planned surgical path based on the first position of the scanning device and the second position of the scanning bed. For example, the first position of the scanning device and the second position of the scanning bed can be directly used as the above-mentioned starting position. In another example, the first position of the scanning device and the second position of the scanning bed can be added to a preset offset to form the above-mentioned starting position.

[0057] In some embodiments, the first position and the second position may be the same as or different from the initial surgical position in the surgical plan information. In some embodiments, the first position and the second position may include the initial surgical position of the scanning device and the scanning bed in the surgical plan information, and the processing device 130 may obtain the initial surgical position of the scanning device and the scanning bed from the surgical plan information. In some embodiments, the first position and the second position may be preset position information. For example, a fixed position may be set for each of the scanning device and the scanning bed as their respective initial positions, and the processing device 130 may obtain the preset position information by reading a configuration file, etc. In some embodiments, the first position and the second position may include the current position of the scanning device and the scanning bed, and the processing device 130 may obtain the current position of the scanning device and the scanning bed by various means, such as through sensors on the scanning device and the scanning bed, images captured by an external camera, etc. In some embodiments, the processing device 130 may obtain the current position of the scanning device and the scanning bed at any time as the first position and the second position.

[0058] A surgical plan refers to a plan for a surgical procedure requiring path planning. This includes various types of surgeries, such as vascular surgery and puncture surgery. Surgical plan information refers to information that can represent the surgical plan, such as the type of surgery, time of surgery, who will perform the surgery, the patient, and the location of the surgery. In some embodiments, this surgical plan information may include the initial surgical position of the scanning device and scanning table at the start of the surgery, and / or the positions that the scanning device and scanning table should reach during the surgery.

[0059] The operating position refers to the position on the body of the surgical object (e.g., the patient) where the surgical operation is performed during the execution of the operation, such as the needle entry point in a puncture operation, etc. In some embodiments, the operating position may include at least one. The operating channel refers to the surgical path of the surgical instrument corresponding to the operating position, such as the needle track including the needle entry point and the target point in a puncture operation, etc. In some embodiments, the number of operating channels may correspond to the number of operating positions, such as one-to-one, many-to-one, etc. In some embodiments, the surgical plan information may include one or more operating positions and one or more operating channels corresponding to these operating positions. For example, multiple needle entry points and multiple needle tracks corresponding thereto, etc. Unless otherwise specified, the operations in this specification refer to multi-needle puncture operations.

[0060] In some embodiments, the processing device 130 may obtain the surgical plan information in various ways, such as from a storage device (eg, storage device 150 ), etc. This specification does not limit this.

[0061] Step 320 : planning a movement path of the end effector based on the first position information, the second position information, and the surgical plan information.

[0062] In some embodiments, the surgical planning information may include one or more operating positions and one or more operating channels corresponding to these operating positions. The end effector in the surgical robot system (for example, the end effector 122 in the interventional surgical robot system 120) needs to perform corresponding operations at these operating positions along the corresponding operating channels. Therefore, it is necessary to determine the movement sequence of the end effector between these operating positions, that is, the movement path, so as to ensure the maximum movement space of the robotic arm. The movement path of the end effector determines the movement sequence of the surgical instruments connected to the end effector (for example, the puncture sequence of the puncture needle). Since the end effector is driven by the movement of the robotic arm, in some embodiments, the movement path of the end effector can be determined by the movement path of each joint angle of the robotic arm.

[0063] In some embodiments, the processing device 130 may sort the multiple operating positions and plan the movement path of the end effector based on the sorting result, which is also referred to as a first sorting result.

[0064] In some embodiments, the processing device 130 may determine a plurality of path planning sub-regions extending along a first direction, wherein the path planning sub-regions cover a plurality of operating positions, and the first direction is parallel to the short axis direction of the scanning bed (e.g., the scanning bed 112). For example only, Figure 5 and 6 The path planning sub-areas include multiple paths extending along the X-axis direction of the CT bed coordinate system, that is, the first direction is parallel to the short axis direction of the scanning bed. In some embodiments, the first direction can be parallel to the long axis direction of the scanning bed. For example only, Figure 7 It includes multiple path planning sub-areas extending along the Z-axis direction of the CT bed coordinate system, that is, the first direction is parallel to the long axis direction of the scanning bed.

[0065] In some embodiments, the processing device 130 may determine a first ranking result of the plurality of operation positions based on the path planning sub-areas.

[0066] In some embodiments, the processing device 130 may sort the obtained multiple path planning sub-areas along a preset direction to obtain a first intermediate sorting result, wherein the preset direction may be parallel to the horizontal plane and perpendicular to the first direction. In some embodiments, the processing device 130 may sort the operation positions in each path planning sub-area to obtain a second intermediate sorting result. The sorting of the multiple path planning sub-areas and the sorting of the operation positions in each path planning sub-area may be performed sequentially or simultaneously. In some embodiments, the processing device 130 may obtain a first sorting result based on the first intermediate sorting result and the multiple second intermediate sorting results.

[0067] In some embodiments, the processing device 130 can determine the ranking characteristic value of each path planning sub-area, and then sort these areas along a preset direction based on the ranking characteristic values ​​of all path planning sub-areas. In some embodiments, the ranking characteristic value of a path planning sub-area may include the extreme value of the operation position characteristic value in the area, such as the maximum or minimum value. In some embodiments, the ranking characteristic value of a path planning sub-area may be other values ​​related to the operation position characteristic value in the area, such as the average value of all characteristic values. In some embodiments, the processing device 130 can determine the ranking characteristic value of a path planning sub-area in various ways. For example, the operation position characteristic value ranked first / last in the second intermediate sorting result of the path planning sub-area is used as the ranking characteristic value. For another example, the extreme value of the operation position characteristic value in the path planning sub-area is directly obtained as its ranking characteristic value.

[0068] In some embodiments, the preset direction can be determined based on the position of the path planning sub-area relative to the robot base. Figure 7 As shown, the first direction is parallel to the long axis of the scanning bed (i.e., the Z-axis direction of the CT bed coordinate system), and the path planning sub-area is on the left side of the robotic arm base. The preset direction can be the direction of the X-coordinate value of the CT bed coordinate system from small to large (the positive direction of the X-axis). For another example, if the path planning sub-area is on the right side of the robotic arm base, the preset direction can be the direction of the X-coordinate value of the CT bed coordinate system from large to small (the negative direction of the X-axis). For another example, if the first direction is parallel to the long axis of the scanning bed (i.e., the Z-axis direction of the CT bed coordinate system), the path planning sub-areas can be sorted in the order of farthest from the robotic arm base to close to the robotic arm base, or the path planning sub-areas can be sorted in the order of close to the robotic arm base to farthest from the robotic arm base.

[0069] In some embodiments, the preset direction can be determined based on the position of the path planning sub-area relative to the scanning device. Figure 5 and 6As shown, the first direction is parallel to the short axis of the scanning bed (i.e., the X-axis direction of the CT bed coordinate system). The path planning sub-regions can be sorted in the order from inside the scanning cavity aperture of the scanning device to outside the scanning cavity aperture along the Z-axis direction of the CT bed coordinate system. That is, the preset direction can be the negative direction of the Z-axis of the CT bed coordinate system. For another example, the preset direction can be the positive direction of the Z-axis of the CT bed coordinate system. That is, the path planning sub-regions can be sorted in the order from outside the scanning cavity aperture of the scanning device to inside the scanning cavity aperture along the Z-axis direction.

[0070] In some embodiments, the end effector may include an end clamp (e.g., a clamping claw) for clamping the puncture needle, and the preset direction may be determined based on the opening direction of the end clamp. In some embodiments, the processing device may first determine the opening orientation of the end clamp at multiple needle entry points, and then determine the preset direction based on the opening orientation. In some embodiments, when the opening orientation of the end clamp at most needle entry points is toward the inside of the scanning cavity aperture of the scanning device, the preset direction may be away from the aperture (i.e., toward the outside of the aperture). In some embodiments, when the opening orientation of the end clamp at most needle entry points is toward the outside of the scanning cavity aperture of the scanning device, the preset direction may be toward the aperture (i.e., toward the inside of the aperture). In some embodiments, the preset direction may be a direction opposite to the opening orientation of the end clamp at most needle entry points.

[0071] The second intermediate sorting result is the sorting of all operating positions in a single area. In some embodiments, in each of the multiple path planning sub-areas, the processing device 130 can sort the operating positions according to a preset sorting rule to obtain a second intermediate sorting result. The preset sorting rules may include multiple ones. In some embodiments, the processing device 130 can sort based on the characteristic values ​​of the operating positions, wherein the characteristic values ​​of the operating positions may include various values ​​that can represent the operating positions, such as the coordinate values ​​of the operating positions (for example, the coordinate values ​​along the first direction or the second direction, etc.), the measurement value of the importance of the operating position, etc. In some embodiments, the processing device 130 can determine the preset sorting rules based on the shape of the end effector at the end of the robotic arm.

[0072] In some embodiments, the end effector may include an end gripper (e.g., a jaw), and the preset sorting rule may be determined based on the opening orientation of the end gripper. Specifically, the processing device 130 may determine the opening orientation of the end gripper at at least one operating position and sort at least one operating position within a path planning subregion based on the opening orientation. Taking the end gripper as an example, the processing device 130 may first determine the opening orientation of the jaw at multiple needle entry points and, based on the opening orientation, determine the sorting of the needle entry points covered by one or more path planning subregions. For example, a puncture region may include multiple needle entry points, each corresponding to a jaw opening orientation. Several needle entry points within each path planning subregion correspond to several jaw opening orientations. If the majority of jaw openings in the puncture region face inward from the scanning aperture, the processing device 130 may sort the points in order from inside the scanning cavity aperture to outside the scanning cavity aperture (e.g., when the positive Z-axis direction points into the scanning aperture, the points are sorted in descending order based on their Z coordinate values). If most of the jaw openings in the puncture area are facing outside the scanning aperture, the preset sorting rule is to sort in the order from outside the scanning cavity aperture of the scanning device to inside the scanning cavity aperture of the scanning device (for example, when the positive direction of the Z axis points into the scanning aperture, sort in ascending order according to the Z coordinate value of the needle entry point).

[0073] In some embodiments, the opening direction of the end gripper of the robot arm can be determined by the following method: according to the needle entry point and the target point, the position of the origin of the end effector coordinate system at the needle entry point and the end effector coordinate system (such as Figure 8 and 9 The coordinate system composed of X2, Y2, and Z2 is established with the end point of the end effector away from the robotic arm 590 as the origin. The orientation of the Z2 axis is determined by taking any direction in the plane passing through the origin of the end effector coordinate system and perpendicular to its Z2 axis as the orientation of its Y2 axis. The orientation of its X2 axis is determined based on the orientation of its Z2 axis and Y2 axis. The rationality of the end effector coordinate system (before the end effector is retracted) is tested using inverse kinematics. If it is not rational, the Z2 axis of the end effector coordinate system is rotated by the end of the robotic arm to continue looking for a new orientation of the end effector coordinate system. If it is rational, the position and posture of the end effector coordinate system are determined, and the X2 axis of the end effector coordinate system is the opening orientation of the end gripper assembly.

[0074] In some embodiments, the end effector may be an auxiliary locator that assists the puncture needle in puncturing, and the preset sorting rules may be determined based on the shape of the auxiliary locator. For example, when the shape of the end effector is L-shaped (e.g. Figure 8 auxiliary positioner shown in the figure), as shown in the figure Figure 7As shown, on each path planning sub-area in the Z-axis direction of the CT bed coordinate system, the needle entry points can be sorted in the order from outside the scanning cavity aperture to inside the aperture of the scanning device (for example, when the positive direction of the Z axis points to the inside of the aperture, the puncture positions are sorted in ascending order based on the Z coordinate value); at the same time, for the sorting of multiple path planning sub-areas, the puncture positions on adjacent path planning sub-areas are sorted in the order from far away from the robot arm base to close to the robot arm base (that is, the puncture positions are sorted in ascending order based on the X coordinate value). This sorting method can be called an N-shaped sorting method. For another example, when the shape of the end effector is I-shaped (such as Figure 9 auxiliary positioner shown in the figure), as shown in the figure Figure 5 and 6 As shown, in each path planning sub-area in the X-axis direction of the CT bed coordinate system, the needle entry points are sorted in the order from farthest from the robot arm base to close to the robot arm base (for example, when the positive direction of the X-axis points to the base, the needle entry points are sorted in the order from small to large in terms of the X-coordinate value); at the same time, for the sorting of multiple path planning sub-areas, the needle entry points on adjacent path planning sub-areas are sorted in the order from inside to outside the scanning cavity aperture of the scanning device (that is, the needle entry points are sorted in the order from large to small in terms of the Z-coordinate value). This sorting method can be called a quasi-Z-shaped sorting method.

[0075] In some embodiments, in each of the plurality of path planning sub-areas, the processing device 130 may sort the operation positions along the first direction to obtain a second intermediate sorting result. Figure 5 and 6 As shown, the first direction is parallel to the short axis of the scanning bed, that is, the X axis of the CT bed coordinate system. In each path planning sub-area, the X coordinate value is from small to large ( Figure 5 ) or the direction from large to small X coordinate values ​​( Figure 6 ) Sort the needle entry points.

[0076] In some embodiments, processing device 130 may sort the operation positions based on the first intermediate sorting result and multiple second intermediate sorting results to obtain a sorting result of all the operation positions, i.e., a first sorting result. In some embodiments, processing device 130 may arrange the second intermediate sorting results of each path planning sub-area along a preset direction according to the sorting results of the path planning sub-areas (the first intermediate sorting results) to obtain a first sorting result.

[0077] In some embodiments, the above sorting can use any sorting algorithm, for example, sorting by the coordinate value of the operation position along a certain direction. In some embodiments, the sorting method can also be user-selected, or can be selected using a sorting algorithm combined with user-selected sorting.

[0078] As an example, when planning the puncture sequence for a puncture needle, since the entry points for all needle tracts are known, assuming the coordinates of the entry points are located in the CT bed coordinate system, the processing device 130 can select the entry point with the smallest / largest Z-axis coordinate value. Based on the X-axis / Z-axis coordinate values ​​of this point, the space is divided into multiple parallel regions in the X-axis / Z-axis directions. The spacing between these regions can be equal or unequal, and all the entry points are located in these regions. The following uses the division of regions in the Z-axis direction as an example to illustrate. For different entry points in the same region, the needle tracts corresponding to the entry points farther from the robotic arm are prioritized for planning. That is, within each region, all entry points are sorted from smallest to largest X-axis coordinate values. All regions are sorted based on the entry point with the smallest / largest Z-axis coordinate value in each region, prioritizing the planning of needle tracts corresponding to entry points in regions with smaller / larger Z-axis coordinate values. Specifically, the minimum / maximum Z-axis coordinate value of the entry points in each region can be used as the Z-axis coordinate value of that region, and each region is planned sequentially based on the region coordinate value sorting results (small to large or large to small). In some embodiments, the processing device 130 can establish a CT bed coordinate system in the following manner: with a point on the CT bed as the origin, the short axis of the CT bed as the X-axis, with the positive direction of the X-axis pointing to the right; the direction perpendicular to the CT bed surface as the Y-axis, with the positive direction of the Y-axis pointing from the bottom surface of the CT bed to the top surface; and the long axis of the CT bed as the Z-axis, with the positive direction of the Z-axis pointing from the direction the CT bed enters the scanning bore. In some embodiments, the processing device 130 can establish the CT bed coordinate system in the following manner: the CT bed is moved into the scanning bore. When the bed code value displayed on the CT device operation interface becomes 0, the CT isocenter and the origin of the CT bed coordinate system coincide. The position of the CT isocenter relative to the CT bed at this time is the position of the origin of the CT bed coordinate system.

[0079] Figure 5 、 6 Schematic diagram of the arrangement of needle paths according to some embodiments of this specification. Figure 5 In the figure, the robotic arm 590 mounted on the robotic arm base is placed on the right side of the CT scanning hole. Figure 6 In the figure, the robotic arm 590 is placed on the left side of the CT scanning hole. Figure 5 and 6In the figure, the X-axis of the CT bed coordinate system is the short axis of the CT bed, and the positive direction of the X-axis is to the right. The Z-axis is the long axis of the CT bed, and the positive direction of the Z-axis is the direction in which the CT bed enters the scanning hole, that is, the direction in which the CT bed enters the aperture. 510, 520, 530, 540, 550, 550, 560, 570, and 580 are all needle entry points, that is, the operating positions in the surgical plan.

[0080] like Figure 5 As shown in the figure, assuming that the line spacing is set to a when dividing the area, first find the needle entry point 520 with the largest Z-axis coordinate value among all the needle entry points in the CT bed coordinate system. Then, divide the dividing plane of ±a / 2 through point 520, and the distance between the two dividing planes is a. Then, divide the areas with equal distance (line spacing of a) in the negative direction of the Z axis, and each needle entry point is divided into a certain area. In the same area, the needle entry points far away from the robot arm 590 are planned first, that is, the needle entry points are planned in order from far to near in terms of distance from the robot arm 590. In the area where 510 and 520 are located, 510 and 520 are planned in sequence; in the area where 530-550 are located, 530, 540, and 550 are planned in sequence; in the area where 560-580 are located, 560, 570, and 580 are planned in sequence. Sort all areas by the entry point with the smallest / largest Z-axis coordinate value in each area, prioritizing the areas with the largest Z-axis coordinate value. That is, from front to back, plan the following: areas 510 and 520, areas 530-550, and areas 560-580. The final order of planned entry points is, from first to last, 510, 520, 530, 540, 550, 550, 560, 570, and 580.

[0081] like Figure 6 As shown, it is Figure 5 The main difference is that the robot arm 590 is placed on the left side of the CT scanning hole. Figure 5 For the same area, the sorting results for each area are also the same. However, when sorting the needle entry points in each area, the points are planned in order of distance from the robot arm 590, from farthest to closest. For example, in the area where 510 and 520 are located, 520 and 510 are planned in that order; in the area where 530-550 are located, 550, 540, and 530 are planned in that order; and in the area where 560-580 are located, 580, 570, and 560 are planned in that order. Therefore, the final order of the planned needle entry points, from earliest to latest, is: 520, 510, 550, 540, 530, 550, 580, 570, and 560.

[0082] In some embodiments, when the scenario involves real-time image-guided online or offline puncture, the processing device 130 can sort the punctures in order from farther away from the robotic arm to closer to the robotic arm to obtain a first sorting result, thereby increasing the available space for planning. In some embodiments, when the scenario involves offline puncture, the processing device 130 can sort the punctures in order from closer to the robotic arm to farther away from the robotic arm to obtain a first sorting result, thereby facilitating the doctor's operation.

[0083] In some embodiments, processing device 130 can plan the movement path of the end effector based on the first sorting result. The order of the operating positions on the movement path corresponds to the order in which the operating positions are performed during surgery. By sorting the operating positions and planning the path based on the first sorting result, the success rate of path planning is improved.

[0084] In some embodiments, the processing device 130 may connect various operating positions (eg, needle entry points, etc.) with line segments according to the first sorting result, thereby forming a planned movement path of the end effector.

[0085] A patient model is a model used to simulate the patient entity and its motion trajectory. It can be implemented in various ways, such as a physical phantom or a computer-simulated patient body. In some embodiments, the patient model can be determined based on the patient's positional information and appearance profile at different times. A scanning device model is a model used to simulate the scanning device entity and its motion trajectory. It can be implemented in various ways, such as scanning device motion simulation or computer-simulated scanning device motion. In some embodiments, the scanning device model can be determined based on the positional information and appearance profile of the entire scanning device or a portion thereof (e.g., the scanning aperture) at different times. A robotic arm end model is a model used to simulate the robotic arm end, the surgical instrument connected thereto, and the motion trajectory of these objects. It can include the robotic arm end portion, the end effector, and the surgical instrument. The surgical instrument can be connected to the robotic arm via the end effector; therefore, the robotic arm end portion, the end effector, and the surgical instrument can be considered as a whole. A collision model is a model used to simulate whether multiple objects will collide during their relative motion. It can simulate various types of objects, such as the robotic arm, the end effector, the patient, the scanning device, and the surgical instrument connected to the robotic arm. In some embodiments, the collision model can include a patient model, a scanning device model, and a robotic arm end model.

[0086] In some embodiments, processing device 130 can determine a collision model based on the first position information and the second position information. Specifically, because the patient is placed on a scanning table and is fixed relative to the table, processing device 130 can determine the relative position of the scanning table and the table at any moment based on their real-time positions. This means that the relative position of the scanning table and the patient at any moment is also determined. Therefore, when a patient model is used to represent the patient and a scanning device model is used to represent the scanning device, processing device 130 can determine a collision model based on the relative position of the scanning device and the patient at any moment.

[0087] In some embodiments, based on the inclination of one or more operating channels (for example, the needle track of the puncture in an interventional puncture surgery) relative to a second direction, the processing device 130 can determine the rotation information of the scanning device in the second direction, wherein the second direction can be parallel to the long axis direction of the scanning bed. Since the head-to-foot direction of the patient on the scanning bed is parallel to the long axis direction of the scanning bed, the rotation of the scanning device in the second direction is equivalent to the tilting of the collimation center line of the scanning device in the head-to-foot direction, thereby facilitating the observation of the needle track tilted in the head-to-foot direction. In this specification, the head-to-foot tilt refers to the tilt relative to the head-to-foot direction of the patient. In some embodiments, the processing device 130 can obtain the inclination angle of one or more operating channels relative to the long axis direction of the scanning bed, which can be referred to as the inclination angle of the needle track head-to-foot direction in puncture surgery. In some embodiments, the processing device 130 can determine the rotation angle of the scanning device in the second direction based on the inclination angle of the needle track head-to-foot direction, that is, the angle between the collimation center line of the scanning device and the long axis direction of the scanning bed, wherein the angle can be referred to as the tilt angle of the scanning device. In some embodiments, the scanning device tilt angle and the needle track head-foot tilt angle can be equal, or the difference between the two can meet a preset condition, where the preset condition can be that the operating channel is within the scanning range of the scanning device. When the scanning device is a CT scan device, the rotation of the scanning device can be equivalent to the rotation of the CT gantry. Therefore, the scanning device tilt angle in this case can be referred to as the CT gantry tilt angle.

[0088] In some embodiments, the scanning bed can move (e.g., along the second and / or third directions), and the processing device 130 can determine the movement information of the scanning bed based on the rotation information of the scanning device in the second direction. For details on how to determine the movement information of the scanning bed, please refer to the description of step 450 and will not be repeated here.

[0089] In some embodiments, when planning a path between an operating position and an adjacent operating position for one or more operating channels, the processing device 130 may determine a scanning device model based on the first position information and the rotation information, and determine a patient model based on the second position information and the movement information. Since the position of the scanning device changes after the scanning device rotates in the second direction, the processing device 130 may determine the scanning device model after the rotation in the second direction based on the scanning device's original position information (i.e., the first position information) and its position change information (i.e., the rotation information). Since the patient's position changes after the scanning bed moves, the processing device 130 may determine the patient model after the movement based on the scanning bed's original position information (i.e., the second position information) and its position change information (i.e., the movement information).

[0090] In some embodiments of the present specification, by tilting the frame of the scanning equipment (e.g., a CT machine, etc.) toward the head and foot sides, it is easier to observe the entire needle track during surgery, especially the needle track tilted toward the head and foot sides, thereby reducing the difficulty of the surgery and shortening the surgery time.

[0091] In some embodiments, the processing device 130 may plan the movement path of the end effector based on the collision model and the first sorting result. Specifically, the processing device 130 may plan an initial movement path of the end effector based on the first sorting result and then verify the movement path through simulation based on the collision model. If, during the verification process, the end effector collides with another object (e.g., at least one of a scanning device, a patient, or a surgical instrument), the verification fails and the path planning fails; otherwise, the verification succeeds and the path planning succeeds.

[0092] In some embodiments of the present specification, the movement path of the terminal path is planned based on the collision model and the sorting results of all operation positions (for example, needle entry points, etc.), so that the path planning is more practical, the planned path is more reasonable, and the risk of path adjustment during surgery is avoided.

[0093] After the scanning bed moves, the various operating positions change relative to their positions before the scanning bed moved. In some embodiments, the processing device 130 can update one or more operating positions based on the scanning bed's movement information and then plan the movement path of the end effector based on the updated one or more operating positions.

[0094] In some embodiments, after planning the end effector's movement path based on the first sorting result, the processing device 130 may perform a path verification on the movement path. This involves simulating the movement path of the end effector based on the collision model and the planned movement path, and using the collision model to determine whether the end effector can reach the operation positions in the order of the first sorting result. If the path verification succeeds, the movement path planning is determined to be successful; if the path verification fails, the movement path planning is determined to be unsuccessful. The entire movement path must pass all verifications at once before proceeding to the subsequent intraoperative phase.

[0095] In some embodiments, the movement path may include a verification path, which may include one or more verification sub-paths. The processing device 130 may sequentially plan the verification sub-paths between each group of adjacent operation positions in the first sorting result.

[0096] In some embodiments, for each set of adjacent operating positions in the first sorting result, the processing device 130 may plan and verify the displacement path segment and the avoidance point based on the preceding operating position in the set of adjacent operating positions. The preceding operating position refers to the operating position that comes first in the sorting result (e.g., the first sorting result, the second sorting result) among the set of adjacent operating positions. Correspondingly, the following operating position refers to the operating position that comes later in the sorting result among the set of adjacent operating positions.

[0097] In order to prevent uncontrollable interference and collision between the end effector and the surgical instruments (for example, puncture needles) left on the patient's body surface during interventional surgery, the initial path from the front operating position to the rear operating position in the adjacent operating positions can allow the end effector to be horizontally retracted for a distance. This operation is called end retraction, and the retraction path can be called a verification shift path segment. Among them, one end of the verification shift path segment is the front operating position, and the other end is the verification avoidance point. In some embodiments, when the end effector moves in the verification shift path segment, the posture of the end effector at the end of the robotic arm remains unchanged, and the position of the end effector will move horizontally for a distance. For example, when the end effector is an auxiliary locator, in the verification shift path segment, the end effector will move horizontally for a distance in the opposite direction of the opening of the positioning slot. In some embodiments, the processing device 130 can pre-set the retraction distance of the end effector, and then based on the retraction distance and the end effector coordinate system before retraction (such as Figure 8 and 9 The coordinate system composed of X2, Y2 and Z2 in the figure) and the robot coordinate system (such as Figure 7-9 The transformation relationship between the end effector coordinate system and the robotic arm coordinate system after retraction is obtained by using the transformation relationship of the robotic arm coordinate system composed of X1, Y1 and Z1 in the figure. Then, the position of the verification avoidance point is obtained based on the transformation relationship, that is, the coordinates of the verification avoidance point in the robotic arm coordinate system.

[0098] In some embodiments of the present specification, the end effector is retracted so that the surgical instrument remaining on the patient's body and the jaws of the end effector can be separated by a certain distance first, thereby avoiding interference and collision between the end effector and the surgical instrument remaining on the patient's body, avoiding safety risks, and also being able to verify the surgical instrument model in the operating position before adding the avoidance point.

[0099] During an interventional multi-needle puncture procedure, the puncture needles from the previous punctures may remain on the patient's body surface. These puncture needles need to be taken into account when planning the movement path of the end effector to avoid collision and interference between the end effector and these puncture needles when the end effector moves according to the planned movement path. Therefore, the models corresponding to these puncture needles need to be added to the collision model at the appropriate time. In some embodiments, after the end effector moves along the verification shift path segment, the processing device 130 can add the surgical instrument model corresponding to the previous operating position to the collision model to update the collision model. The surgical instrument model is a model used to simulate the entity of the surgical instrument (for example, a puncture needle, etc.) and its motion trajectory that remains on the patient's body during interventional surgery. Since the positions of these surgical instruments and the patient are relatively fixed, their movement can be equated with the movement of the patient model.

[0100] In some embodiments, the processing device 130 may plan a verification avoidance path segment between the verification avoidance point and a subsequent operating position in the aforementioned adjacent operating positions based on the updated collision model, wherein the verification shift path segment and the verification avoidance path segment may constitute a verification sub-path.

[0101] In some embodiments, for any verification sub-path, the processing device 130 may verify through simulation whether the end effector in the updated collision model will collide with at least one of the surgical instrument model, the patient model, the scanning device model, etc. If the simulation result shows that the end effector will collide with any one or more of the three, the verification in the verification sub-path fails, and the end effector movement path planning is determined to have failed. If the simulation result shows that the end effector will not collide with any of the three, the verification of the verification sub-path is successful, and the next verification sub-path is verified. When all verification sub-paths are verified successfully, the entire end effector movement path planning is successful.

[0102] In some embodiments, the processing device 130 may also use any other path planning method to plan the movement path of the end effector. For example, a trained machine learning model may be used for path planning, where the model may be trained using a large amount of historical path planning sample data.

[0103] In some embodiments, in addition to verifying the planned end effector movement path, the processing device 130 may also perform other verifications related to the surgical operation, such as verifying at least one of whether the operating channel (e.g., the needle track, etc.) interferes with important tissue, whether the operating channel is drawn incorrectly, and whether the operating channel length meets the corresponding instrument requirements. In some embodiments, if at least one of these other verifications fails, the processing device 130 may also determine that the end effector movement path planning has failed. In some embodiments, if the planned end effector movement path verification succeeds and all other verifications are also successful, the processing device 130 may determine that the end effector movement path planning has succeeded.

[0104] In some embodiments, after the movement path planning of the end effector fails, the processing device 130 can execute steps 330-350 to issue a prompt to the user and re-plan the movement path; after the movement path planning of the end effector is successful, the processing device 130 can execute steps 360-370 to perform surgery according to the planned movement path.

[0105] Step 330 : In response to the end effector's movement path planning failing, output a prompt message.

[0106] In some embodiments, when the movement path planning of the end effector fails, the processing device 130 can output prompt information to the user (eg, a doctor, etc.) in various ways, wherein the prompt information can be determined according to the reason for the planning failure.

[0107] In some embodiments, the reason for planning failure may include at least one of the following situations: collision between the end effector and the patient model, collision between the end effector and the scanning device model, collision between the end effector and the instrument model, etc.

[0108] In some embodiments, the processing device 130 can adjust the operation position and the corresponding operation channel according to the prompt information, and use the adjusted operation position and the corresponding operation channel information to update the surgical plan information, so that after the adjustment, subsequent re-path planning can be successful. The adjustment can be performed in various ways, such as based on adjustment suggestions, based on experience, etc.

[0109] As shown in Table 1 below, in some embodiments, the prompt information may include the reason for the path planning failure, and the processing device 130 may make adjustments based on the reason.

[0110] As shown in Table 1 below, in some embodiments, the prompt information may include adjustment suggestions (eg, adjustment suggestions for the needle track, etc.), and the processing device 130 may make adjustments based on the adjustment suggestions.

[0111] In some embodiments, in response to the collision between the robot end model and the patient model causing the path planning to fail, the processing device 130 may output a first prompt message, wherein the first prompt message may at least instruct the user to adjust one or more operation positions in the surgical plan information. Figure 10 For example, when the planned end effector movement path is verified by the collision model, it is detected that at the needle entry point 1010, the manipulator end 1020 (i.e., the end effector) in the manipulator end model collides with the phantom 1030 (i.e., the patient model), and the verification fails. At this time, the processing device 130 can display a first prompt message to the user. For example, the prompt message corresponding to serial number 1 in Table 1 below contains the cause of the failure and the adjustment suggestions. In some embodiments, the processing device 130 can adjust the needle entry point (i.e., the operating position) and / or the corresponding needle track (i.e., the operating channel) at the point where the verification failed according to the suggestions in the first prompt message. For example, according to suggestion ② of the prompt message corresponding to serial number 1 in Table 1, the needle track can be tilted to the head and foot side. In some embodiments, the processing device 130 can change the relative position between the end of the manipulator and the patient by adjusting at least one of the manipulator and the scanning bed, thereby adjusting the operating position and / or the corresponding operating channel. For example, as Figure 11 As shown, the processing device 130 adjusts the needle entry point 1010 through the movement of the robotic arm according to the prompt information corresponding to the serial number 1 in the table, forming a needle track tilted toward the head and foot side, so that at the adjusted needle entry point 1010, the robotic arm end 1020 and the phantom 1030 will no longer collide.

[0112] In some embodiments, in response to the collision between the robot end model and the scanning device model causing the path planning to fail, the processing device 130 may output a second prompt message, wherein the second prompt message may instruct the user to adjust one or more operation positions in the surgical plan information or adjust the position of the scanning bed. Figure 12For example, when the planned end effector movement path is verified by the collision model, it is detected that at the needle entry point 1210, the manipulator end 1220 (i.e., the end effector) in the manipulator end model collides with the inner wall of the scanning cavity of the CT machine 1230 (i.e., the scanning device model), and the verification fails. At this time, the processing device 130 can display a second prompt message to the user. For example, the prompt message corresponding to serial number 2 in Table 1 below contains the cause of the failure and the adjustment suggestions. In some embodiments, the processing device 130 can adjust the needle entry point (i.e., the operating position) and / or the corresponding needle track (i.e., the operating channel) at the point where the verification failed according to the suggestions in the second prompt message. For example, according to suggestion ① of the prompt message corresponding to serial number 2 in Table 1, the needle entry point 1210 can be adjusted to a position close to the middle of the patient's body. In some embodiments, the processing device 130 can adjust the operating position or the position of the scanning bed by adjusting at least one of the manipulator and the scanning bed, thereby changing the relative position between the end of the manipulator and the patient. For example, as Figure 13 As shown, the processing device 130 moves the robotic arm end 1220 a distance toward the middle of the patient's body according to the prompt information corresponding to serial number 2 in the table, so that the robotic arm end 1220 and the CT machine 1230 will no longer collide at the adjusted needle insertion point 1210.

[0113] In some embodiments, in response to the collision between the robot arm end model and the surgical instrument model causing the path planning to fail, a third prompt message is output, wherein the third prompt message can at least instruct the user to adjust one or more operation positions in the surgical plan information along the long axis direction of the scanning table. Figure 14For example, when the planned end effector movement path is verified by the collision model, it is detected that at the needle entry point 1410, the robotic arm end 1420 (i.e., the end effector) in the robotic arm end model collides with the puncture needle 1430 (i.e., the surgical instrument model), and the verification fails. At this time, the processing device 130 can display a third prompt message to the user. For example, the prompt message corresponding to sequence number 3 in Table 1 below includes the reason for the failure and adjustment suggestions. In some embodiments, the processing device 130 can adjust the needle entry point (i.e., the operation position) and / or the corresponding needle track (i.e., the operation channel) at the point where the verification failed according to the suggestions in the third prompt message. For example, since the needle track (i.e., needle track X in the prompt information of Table 1 below) corresponding to the post-operation position (needle entry point 1410) and the needle track of the puncture needle 1430 (i.e., needle track A in the prompt information of Table 1 below, the operation order of needle track A is before needle track X) are in different scanning layers, the distance between the operation positions (needle entry points) of the two needle tracks can be increased according to suggestion ② of the prompt information corresponding to sequence number 3 in Table 1. In some embodiments, the processing device 130 can change the relative position between the end of the robotic arm and the surgical instrument on the patient's surface by adjusting at least one of the robotic arm and the scanning bed, thereby adjusting the operation position and / or the corresponding operation channel at least along the long axis direction of the scanning bed. For example, Figure 15 As shown, the processing device 130 adjusts the needle track (needle track X in the prompt information of Table 1 below) corresponding to the needle entry point 1410 along the long axis direction of the scanning bed through the movement of the robotic arm according to the prompt information corresponding to serial number 3 in the table, that is, increases the left and right side inclination angles of the needle track so that the inclination angles are greater than the left and right side inclination angles of the needle track of the puncture needle 1430, so that at the adjusted needle entry point 1410, the robotic arm end 1420 and the puncture needle 1430 will no longer collide.

[0114] In some embodiments, the reasons for planning failure may include other reasons, such as interference between the operating channel and important tissue, errors in drawing the operating channel, and the operating channel length not meeting the requirements of the corresponding instrument. The processing device 130 may make corresponding adjustments based on the prompt information corresponding to these reasons.

[0115] In some embodiments, the output mode of the prompt information may include multiple methods, such as text, sound, image, video, animation, etc., or any combination thereof. In some embodiments, the processing device 130 may display the text prompt information to the user through a display device (e.g., a display component of the terminal 140). For example, as shown in Table 1: Table 1 Sequence number 1 corresponds to the collision between the robot end model and the patient model, sequence number 2 corresponds to the collision between the robot end model and the scanning device model, and sequence number 3 corresponds to the collision between the robot end model and the surgical instrument model. Needle track A is operated before needle track X.

[0116] In some embodiments of this specification, by outputting a prompt message after path planning fails, it is convenient for the user to find the cause of the failure and to select an adjustment strategy more effectively.

[0117] Step 340: Reacquire the second position information of the scanning bed and / or the surgical plan information, wherein the reacquired surgical plan information includes at least one adjusted operating position and an adjusted operating channel corresponding to the adjusted operating position.

[0118] In some embodiments, after adjusting the operating position and corresponding operating channel in the planned movement path based on the prompt information, the processing device 130 may reacquire the second position information of the scanning table and / or the surgical plan information. The reacquired surgical plan information may include at least one adjusted operating position and an adjusted operating channel corresponding to the adjusted operating position. The acquisition method may be the same or similar to step 310 and will not be further described here.

[0119] Step 350 : replanning the movement path of the end effector based on the reacquired second position information and / or the reacquired surgical plan information.

[0120] In some embodiments, the processing device 130 may re-plan the movement path of the end effector based on the re-acquired second position information and / or the re-acquired surgical plan information. The path planning method may be the same or similar to step 320 and will not be repeated here.

[0121] In some embodiments, when the movement path of the end effector is successfully planned again, the processing device 130 can execute steps 360 and 3701 to perform surgery based on the successfully planned movement path; when the re-planning fails, the processing device 130 can return to execute steps 330-350.

[0122] Step 360 : In response to the successful planning of the movement path of the end effector, determine target surgical plan information and a second ranking result of the plurality of target operation positions in the target surgical plan information.

[0123] In some embodiments, when the movement path planning of the end effector is successful, the processing device 130 can determine the target surgical plan information, and the sorting results of the multiple target operation positions in the target surgical plan information, which is also called the second sorting result. After the planning is successful, the operation position and the corresponding operation channel in the surgical plan information have been verified and can meet the requirements of normal surgery. The processing device 130 can determine the surgical plan information after the planning is successful as the target surgical plan information, and the second sorting result of the multiple target operation positions in the target surgical plan information is the sorting result of the multiple operation positions in the surgical plan information (i.e., the first sorting result). The sorting result of the target operation position determines the execution order of these target operation positions during the operation. In some embodiments, the target surgical plan information may include surgical plan information to be executed. For example, surgical plan information executed in the actual operation or surgical simulation stage.

[0124] Step 370: sequentially plan and execute execution sub-paths between each group of adjacent target operation positions in the second sorting result, wherein at least a portion of each execution sub-path is executed within a scanning aperture of the scanning device.

[0125] In some embodiments, after the target surgical plan information is determined in step 360, in a preset environment, the processing device 130 can, based on the first position information, the second position information and the target surgical plan information obtained in step 310, sequentially plan and execute the execution sub-paths between each group of adjacent target operation positions in the second sorting result. An execution sub-path refers to the movement path of the end effector between two adjacent target operation positions in the second sorting result. The complete movement path of the end effector may include multiple continuous execution sub-paths. Specifically, the processing device 130 may control the movement of the robotic arm end effector along the execution sub-path through instructions, etc. In some embodiments, the preset environment may include at least one of an actual surgical stage, a surgical simulation stage, etc. In some embodiments, steps 310-360 may be performed in the preoperative stage of the surgery, and step 370 may be performed in the intraoperative stage of the surgery.

[0126] In some embodiments, path planning and path execution may be performed by the same or different processors. For example, the path planning operations in steps 310-370 may be performed by a first processor (e.g., a processor on a master terminal), and the path execution operations in step 370 may be performed by a second processor different from the first processor (e.g., a processor on a slave terminal), wherein the first processor and the second processor may be included in the processing device 130. For another example, the path planning and path execution operations in steps 310-370 may be performed by the same processor, which may be included in the processing device 130.

[0127] To monitor the accuracy of surgical procedures during surgery, real-time imaging can be used to observe operations related to the operating position and operating channel. For example, during an interventional puncture procedure, as the robotic arm grips the puncture needle and moves to the interventional puncture point, the doctor can monitor the entire needle insertion process through real-time exposure on a CT machine. In some embodiments, at least a portion of each execution subpath can be executed within the scanning aperture of the scanning device. This allows the scanning device to capture real-time images of the patient and monitor the surgical process, thereby ensuring the smooth progress of the surgery.

[0128] In some embodiments, the processing device 130 can tilt the scanning device frame, i.e., tilt the frame toward the head-foot side (i.e., the second direction), so that the target operating channel is located within the scanning range of the scanning device, so as to enable complete and accurate real-time monitoring of the surgical process through the scanning device.

[0129] In some embodiments, if the target operation channel corresponding to the target operation position included in the execution sub-path to be executed is tilted relative to the second direction, the processing device 130 can control the scanning device, the scanning bed and the robotic arm to execute the sub-path in a coordinated manner.

[0130] In some embodiments, the target operation position ranked first in the second sorting result is referred to as the first target operation position. The processing device 130 can tilt the scanning device gantry and perform surgery by the following steps: when the target operation channel corresponding to the first target operation position is tilted relative to the second direction, the processing device 130 can control the scanning device to rotate in the second direction so that the scanning aperture of the scanning device is tilted by a first angle relative to the second direction, wherein the first angle corresponds to the tilt angle of the target operation channel corresponding to the first target operation position relative to the second direction, and the second direction is parallel to the long axis of the scanning bed; control the scanning bed to move along the second direction and a third direction to enter the scanning aperture of the scanning device so that the target operation channel corresponding to the first target operation position is within the scanning range of the scanning device, and the third direction is a direction perpendicular to the scanning bed; and plan and control the robotic arm and end effector to execute an execution subpath from the initial safe position to the first target operation position. The safe position, also known as the external safety point, is a position where the robotic arm and end effector will not collide with the scanning device and scanning bed during tilt adjustment. The initial safe position is the safe position of the robotic arm and end effector at the beginning of the surgery. In some embodiments, the first angle is equal to or approximately equal to an inclination angle of the target operating channel corresponding to the first target operating position relative to the second direction. In some embodiments, controlling the scanning bed to move into the scanning bore may only include movement along the second direction.

[0131] In some embodiments, the processing device 130 can tilt the scanning device gantry and perform surgery by: based on the tilt of the rear target operating channel corresponding to the rear target operating position among the adjacent target operating positions relative to a second direction, the processing device 130 can control the robotic arm and end effector to withdraw from the scanning bore of the scanning device to a safe position from the front target operating position among the adjacent target operating positions, wherein the second direction can be parallel to the long axis of the scanning bed; control the scanning bed to withdraw from the scanning bore of the scanning device, and then control the scanning device to rotate in the second direction so that the scanning bore of the scanning device is tilted by a second angle relative to the second direction, wherein the second angle corresponds to the tilt angle of the rear target operating channel relative to the second direction; control the scanning bed to move along the second direction and a third direction to enter the scanning bore of the scanning device so that the rear target operating channel is within the scanning range of the scanning device, wherein the third direction can be a direction perpendicular to the scanning bed; and plan and control the robotic arm and end effector to execute an execution subpath from the safe position to the rear target operating position. The front target operating position is the target operating position that is ranked earlier among the adjacent target operating positions, and the rear target operating position is the target operating position that is ranked later among the adjacent target operating positions. In some embodiments, the second angle is equal to or approximately equal to the inclination angle of the rear target operating channel relative to the second direction. In some embodiments, when controlling the movement of the scanning bed into the scanning aperture, only movement along the second direction may be included. For example, if the front target operating position is the first target operating position, the rear target operating channel is inclined relative to the second direction, and the front target operating channel and the rear target operating channel are located on different scanning layers, then after tilting the scanning device frame and controlling the robotic arm and end effector to execute the execution subpath from the initial safe position to the front target operating position, the processing device 130 can adjust the position of the frame and / or the scanning bed in the above manner so that the rear target operating channel is within the scanning range of the scanning device, and then plan and control the robotic arm and end effector to execute the execution subpath from the front target operating position to the rear target operating position. For another example, if the front target operating position is not the first target operating position, and the rear target operating channel is tilted relative to the second direction, the processing device 130 may adjust the position of the gantry and / or the scanning bed in the above manner so that the rear target operating channel is within the scanning range of the scanning device, and then plan and control the robotic arm and the end effector to execute the execution sub-path from the front target operating position to the rear target operating position.

[0132] In some embodiments, if the rear target operating channel corresponding to the rear target operating position among adjacent target operating positions is tilted relative to a second direction, and the front target operating channel and the rear target operating channel are located on different scanning layers, the processing device 130 may also tilt the scanning device gantry and perform the surgery by: controlling the scanning device to rotate in a second direction so that the scanning aperture of the scanning device is tilted by a second angle relative to the second direction; controlling the scanning bed to move along the second and third directions so that the rear target operating channel is within the scanning range of the scanning device; and planning and controlling the robotic arm and end effector to execute an execution subpath from the front target operating position to the rear target operating position. For example, if the front target operating position is the first target operating position, after tilting the scanning device gantry and controlling the robotic arm and end effector to execute an execution subpath from the initial safe position to the front target operating position, if the front target operating channel and the rear target operating channel are located on different scanning layers, the processing device 130 may adjust the position of the gantry and / or scanning bed in the manner described above so that the rear target operating channel is within the scanning range of the scanning device, and then plan and control the robotic arm and end effector to execute an execution subpath from the front target operating position to the rear target operating position.

[0133] In some embodiments, if the rear target operation channel corresponding to the rear target operation position among the adjacent target operation positions is tilted relative to the second direction, and the front target operation channel and the rear target operation channel corresponding to the front target operation position among the adjacent target operation positions are located on the same scanning layer, the processing device 130 may plan and control the manipulator and end effector to execute an execution subpath from the front target operation position to the rear target operation position. For example, if the front target operation position is the first target operation position, then after tilting the scanning device gantry and controlling the manipulator and end effector to execute the execution subpath from the initial safety position to the front target operation position, the tilt angle of the front target operation channel corresponding to the front target position relative to the second direction is the first angle. If the first angle and the second angle are equal or unequal, and the front target operation channel and the rear target operation channel are located on the same scanning layer, the processing device 130 may directly plan and control the manipulator and end effector to execute the execution subpath from the front target operation position to the rear target operation position without adjusting the position of the gantry and the scanning bed.

[0134] In some embodiments, if the rear target operating channel corresponding to the rear target operating position among the adjacent target operating positions is tilted relative to a second direction, and the front target operating channel and the rear target operating channel corresponding to the front target operating position among the adjacent target operating positions are located on the same scanning layer, and the front target operating channel and the rear target operating channel are tilted at unequal angles relative to the second direction, the processing device 130 can tilt the scanning device gantry and perform the surgery by: controlling the scanning device to rotate in the second direction, tilting the scanning aperture of the scanning device at a second angle relative to the second direction, so that the rear target operating channel is within the scanning range of the scanning device; and planning and controlling the robotic arm and end effector to execute an execution subpath from the front target operating position to the rear target operating position. In some embodiments, before or after controlling the scanning device to rotate in the second direction, the processing device 130 can also control the scanning bed to move in the second and third directions. For example, if the front target operating position is the first target operating position, then after tilting the scanning device gantry and controlling the robotic arm and end effector to execute the execution sub-path from the initial safety position to the front target operating position, the tilt angle of the front target operating channel corresponding to the front target position relative to the second direction is the first angle. If the first angle and the second angle are not equal, and the front target operating channel and the rear target operating channel are located on the same scanning layer, the processing device 130 can adjust the position of the gantry and / or the scanning bed in the above manner to place the rear target operating channel within the scanning range of the scanning device, and then plan and control the robotic arm and end effector to execute the execution sub-path from the front target operating position to the rear target operating position. In some embodiments, controlling the movement of the scanning bed may only include movement along the second direction.

[0135] In some embodiments, if the scanning device gantry is tilted at the front target position among the adjacent target operating positions, the processing device 130 may control the scanning device to perform a gantry zeroing operation, and then control the scanning device to rotate so that the scanning device's scanning aperture is tilted by a second angle relative to a second direction. In some embodiments, the gantry zeroing operation may include controlling the scanning device to rotate in the second direction so that the scanning device's scanning aperture is parallel to the second direction. For example, if the front target operating position is the first target operating position, after tilting the scanning device gantry and controlling the robotic arm and end effector to execute the execution sub-path from the initial safety position to the front target operating position, the processing device 130 may control the scanning device to rotate by a first angle in the opposite direction of the second direction so that the scanning device's scanning aperture is parallel to the second direction, i.e., returning the scanning device gantry to its initial position at the start of the procedure.

[0136] In some embodiments, after controlling the rotation of the scanning device and / or the movement of the scanning bed, the processing device 130 may update the collision model and the positional information of the rear target operation position, and plan the execution sub-path based on the updated collision model and the positional information of the rear target operation position. For details on how to plan the execution sub-path based on the updated collision model and the positional information of the rear target operation position, please refer to the relevant description in step 320 and will not be repeated here.

[0137] For more details on how to tilt the scanning device frame and perform surgery through the above steps, please refer to the relevant description of steps 430-460, which will not be repeated here.

[0138] In some embodiments of the present specification, the accuracy of path planning is improved and guaranteed by planning and simulation verification of the movement path of the robot end effector (for example, a puncture needle, etc.) before surgery; by scanning the tilt of the equipment rack during surgery, the surgical process can be completely and accurately monitored in real time, reducing the difficulty of surgical operation; the above method saves the user's surgical time, reduces the difficulty of using the system, and improves the user experience.

[0139] Figure 4 is an exemplary flow chart of another surgical path planning method according to some embodiments of this specification.

[0140] like Figure 4 As shown, the process 400 includes the following steps: In some embodiments, the process 400 may be executed by the processing device 130 .

[0141] Step 410: Acquire first position information of the scanning device, second position information of the scanning bed, and target surgical plan information to be performed, wherein the target surgical plan information includes multiple target operating positions and multiple target operating channels corresponding to the target operating positions.

[0142] In some embodiments, the operation of step 410 may be similar to that of step 310, wherein the target surgical plan information to be executed may be equivalent to the surgical plan information in step 310. The difference is that the surgical plan corresponding to the target surgical plan information to be executed includes a directly executable end effector movement path, while the surgical plan information in step 310 requires path planning and verification. For more details on the operation of step 410, please refer to the description of step 310 and will not be repeated here.

[0143] Step 420 : sequentially plan and execute execution sub-paths between each group of adjacent target operation positions, wherein at least a portion of each execution sub-path is executed within a scanning aperture of the scanning device.

[0144] In some embodiments, the operation of step 420 may be similar to the relevant content in step 370 and will not be repeated here.

[0145] In step 430, the rear target operation channel corresponding to the rear target operation position among the adjacent target operation positions is tilted relative to a second direction, and the robotic arm and end effector are controlled to withdraw from the front target operation position among the adjacent target operation positions, from the scanning aperture of the scanning device to a safe position. The second direction is parallel to the long axis of the scanning bed.

[0146] When performing interventional punctures on the chest or abdomen, the needle path is tilted to the head and foot, and the puncture needle trajectory (i.e., the complete needle path) cannot be seen under real-time exposure, making the operation difficult. Figure 13 As shown, the CT gantry is not tilted, but Figure 13 The middle needle track is tilted toward the head and feet. When the CT machine is exposed in real time, the complete needle track cannot be seen because the exposure direction is not parallel to the layer where the needle track is located.

[0147] In some embodiments, the processing device 130 can tilt the scanning device (e.g., CT machine, MRI device, etc.) by tilting the scanning device frame, so that the puncture needle trajectory can be seen under real-time exposure, thereby reducing the difficulty of the operation to a certain extent. Figure 14-16 As shown, in Figure 13 On the basis of the CT gantry shown, the CT gantry is tilted toward the head-foot side (ie, the second direction) so that the exposure direction of the CT machine is parallel to the layer where the needle track is located, thereby allowing the complete head-foot side tilted needle track to be seen.

[0148] In some embodiments, the tilted scanning device may also be applicable to other surgeries, such as vascular surgery.

[0149] In the process of tilting the scanning device, the robot arm and the end effector need to be moved out of the scanning bore of the scanning device first. In some embodiments, based on the inclination of the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions relative to the second direction, the processing device 130 can control the robot arm and the end effector to withdraw from the scanning bore of the scanning device to a safe position from the front target operation position in the adjacent target operation positions. The second direction is parallel to the long axis direction of the scanning bed. Taking interventional multi-needle puncture surgery as an example, the front target position can correspond to the current needle entry point, the rear target position can correspond to the next needle entry point to be performed, and the target operation channel corresponding to the rear target operation position in the adjacent target operation positions is tilted toward the second direction, indicating that the target operation channel is a head-foot side tilted needle channel, that is, the next needle entry point corresponds to a head-foot side tilted needle channel. The processing device 130 can control the robot arm and the puncture needle to withdraw from the current needle entry point to a safe position from the scanning bore of the CT machine.

[0150] In some embodiments, processing device 130 can tilt the scanning device by executing step 440 and move the scanning bed by executing step 450 so that the target operating position is within the scanning range of the scanning device. For example, using a CT scanner, processing device 130 can rotate the CT gantry to tilt it by executing step 440 and move the scanning bed by executing step 450 so that the target point and the needle entry point are within the scanning range defined by the CT scanning boundary. Preferably, the target point and the needle entry point can be located on the collimation centerline of the CT scanner.

[0151] Step 440: Control the scanning bed to withdraw from the scanning aperture of the scanning device, and then control the scanning device to rotate in a second direction so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction. The second angle corresponds to the tilt angle of the rear target operating channel relative to the second direction.

[0152] In some embodiments, after controlling the robotic arm and the end effector to withdraw to a safe position, the processing device 130 may control the scanning bed to withdraw from the scanning aperture of the scanning device so that there is enough space for the scanning device to tilt.

[0153] In some embodiments, after the robotic arm, the end effector, and the scanning bed all exit the scanning aperture of the scanning device, the processing device 130 may control the tilt of the scanning device, i.e., control the scanning device to rotate in a second direction (i.e., the head-foot side direction) so that the scanning aperture of the scanning device is tilted relative to the second direction.

[0154] In some embodiments, the rotation angle (e.g., the first angle, the second angle) of the scanning device in the second direction can be equal to or similar to the tilt angle of the target operating channel in the second direction. The processing device 130 can determine the rotation angle in the following manner: determine the first vector of the target operating channel based on the target surgical plan information, and determine the rotation angle based on the first vector. The first vector is a vector representing the spatial direction of the target operating channel. For example, in a puncture surgery, the first vector can be a vector pointing from the needle entry point to the target point. In some embodiments, the tilt of the scanning device can be equivalent to the tilt of the gantry. Therefore, the tilt angle of the scanning device gantry can be used as the angle of rotation of the scanning device in the second direction.

[0155] The following Figure 14 The following describes how to calculate the tilt angle of the CT gantry by taking the scanning device as a CT gantry and the scanning bed as a CT bed as an example. Figure 14 As shown in the figure, during the preoperative scanning phase, when the CT machine locates the puncture point in the scanning bore, the target layer is positioned to the collimation center line by moving the bed. At this time, the needle point is recorded as E(x e ,y e , z e ), the target is T(x t ,y t , zt ), which are the needle entry point and target point values ​​in the target surgical plan information. Both the needle entry point and the target point are expressed in the CT bed coordinate system (also called the patient coordinate system).

[0156] In some embodiments, the vector from the needle entry point E to the target point T It can be expressed as the following formula:

[0157] In some embodiments, the X-axis direction vector of the CT bed coordinate system can be recorded as Then the vector and The vector after cross product can be expressed as follows:

[0158] In some embodiments, if the Z-axis reverse direction vector of the CT bed coordinate system is recorded as Then the vector Know The angle θ between them is the inclination angle of the needle track in the head-foot direction (equal to the inclination angle of the CT gantry), which can be expressed as follows:

[0159] Step 450: Control the scanning bed to move at least along the second direction and enter the scanning aperture of the scanning device so that the rear target operation channel is within the scanning range of the scanning device.

[0160] In some embodiments, after the scanning device completes the head-to-foot tilt, the processing device 130 can control the scanning bed to move along the second and third directions, enter the scanning aperture of the scanning device, and place the rear target operating channel within the scanning range of the scanning device. The third direction is perpendicular to the scanning bed, i.e., perpendicular to the plane containing the first and second directions. Therefore, the second direction is horizontal, and the third direction is vertical. Controlling the scanning bed to move along the second and third directions, i.e., controlling the scanning bed to move horizontally and vertically, allows the bed to re-enter the scanning aperture of the scanning device. In some embodiments, when the scanning device is tilted and there is no possibility of interference between the scanning bed and the scanning device, the processing device 130 can also control the scanning bed to move only along the second direction, i.e., only horizontally.

[0161] In some embodiments, after the scanning device is tilted, the tilt angle of the scanning device frame may correspond to a minimum scanning bed height. The processing device 130 may move the scanning bed up and down to avoid interference between the scanning bed and the scanning device, and move the scanning bed left and right to ensure that the target scanning position falls within the scanning range enclosed by the scanning boundary of the scanning device.

[0162] In some embodiments, the amount of movement of the scanning bed can be determined based on the following methods: obtaining the minimum scanning bed height of the scanning device at a first angle, and determining the vertical movement amount of the scanning bed based on the minimum scanning bed height; and, after the vertical movement, determining the relative position of the target point with respect to the isocenter of the scanning device, and determining the horizontal movement direction and amount of the scanning bed based on the relative position.

[0163] The following Figure 15 The following describes how to calculate the vertical movement amount of the CT bed by taking the scanning device as a CT gantry and the scanning bed as a CT bed as an example. Figure 15 Is in Figure 14 The CT bed is moved vertically based on the tilt of the CT gantry shown in the figure. For a specific model of CT, each tilt angle corresponds to a minimum scanning bed height, which can be obtained by querying the CT parameters. Figure 15 As shown, the rack tilt angle is (and Figure 15 The inclination angle θ of the middle needle channel is equal to the head and foot directions. Figure 15 The minimum scanning bed height of CT is h min .

[0164] In some embodiments, the bed height when the frame is not tilted is h0, and the vertical bed movement distance Δ h It can be expressed as the following formula: The bed height after the frame tilts Need to be set to

[0165] The following Figure 16 Take the example to explain how to calculate the horizontal movement amount of the CT bed. Figure 16 Is in Figure 15 The CT bed is moved horizontally after the vertical movement of the CT bed is completed. Figure 15 As shown, after the bed is moved vertically, the CT bed coordinate system moves upward by Δ h , let the needle insertion point after moving upward be E′(x e ,y e , z e ), the target is T′(x t ,y t , z t ),but Figure 16 In the figure, the needle insertion point after the bed is moved horizontally is E″, and the target point is T″. The coordinates of the isocenter point O in the target surgical plan information in the CT bed coordinate system during the preoperative scan are O(x o ,y o , z o), then the coordinate of the isocenter O in the CT bed coordinate system after the bed is moved in the vertical direction is O(x o ,y o +Δ h , z o The isocenter is the center of rotation of the CT gantry and does not change when the gantry is tilted or the scanning table moves.

[0166] In some embodiments, when the Y coordinate y of the target point T′ after the bed is moved vertically t Greater than the Y coordinate (y o +Δ h ), the bed moves horizontally toward the inside of the scanning hole; otherwise, the bed moves horizontally toward the outside of the scanning hole. Assuming the horizontal bed movement direction is horDir, the following formula is obtained:

[0167] In some embodiments, the horizontal bed movement amount Δ z It can be expressed as the following formula: Δ z =horDir*OT′*tanθ=horDir*abs(y t -(y o +Δ h ))*tanθ (6) Where OT′ represents the distance between the isocenter O and the target point T′; abs represents the absolute value operation; θ has the same meaning as in formula (3), which represents the inclination angle of the needle path in the head-foot direction, which is related to the gantry inclination angle equal.

[0168] like Figure 16 As shown, the Y coordinate y of the target point T′ after the bed is moved vertically t Greater than the Y coordinate (y o +Δ h ), therefore, the bed is moved horizontally toward the scanning hole. At this time, horDir=1, and horDir=1 can be substituted into formula (6) to obtain the horizontal bed movement amount Δz.

[0169] Step 460 : Plan and control the robotic arm and the end effector to execute an execution subpath from the safe position to the rear target operating position.

[0170] After the scanning bed moves vertically and horizontally, the position reached by the robotic arm also needs to change. In some embodiments, after step 450, the processing device 130 may plan and control the robotic arm and end effector to execute an execution subpath from the safe position to the subsequent target operating position. Specifically, the processing device 130 may reposition the robotic arm and plan its arrival position so that the robotic arm can continue to execute the execution subpath from the safe position to the subsequent target operating position (i.e., the next operating position).

[0171] In some embodiments, since the relative position of the scanning bed and the scanning device changes after the gantry tilt, the position information of the needle entry point and the target point will also change. In some embodiments, the position information of the needle entry point and the target point can be represented by coordinates in the robotic arm coordinate system. In some embodiments, the processing device 130 can determine the position information of the needle entry point and the target point after the gantry tilt by: determining a first transformation matrix based on the horizontal and vertical movement amounts of the scanning bed; obtaining the transformation relationship between the coordinate systems at the needle entry point and the target point in the robotic arm coordinate system; and determining the transformation relationship between the coordinate systems at the needle entry point and the target point in the robotic arm coordinate system based on the transformation relationship and the first transformation matrix.

[0172] In some embodiments, the positive direction of the Y coordinate axis of the CT bed is vertically downward, and the positive direction of the Z axis is from the foot side of the CT bed to the head side of the CT bed. After the gantry is tilted, the CT bed coordinate system moves along the Y coordinate axis relative to the preoperative scan by - Δ h , that is, move upward in the vertical direction Δ h , move -Δ along the Z coordinate axis z , that is, move from the CT bedside to the CT bedside z , then the transformation relationship of the CT bed coordinate system before and after movement (transformation matrix ) can be expressed as follows:

[0173] In some embodiments, during preoperative scanning, the transformation relationship between the needle insertion point and the target point coordinate system in the robotic arm coordinate system is: After the gantry is tilted, the positions of the needle entry point and target point in the robot arm coordinate system need to take into account the above transformation relationship. The needle entry point and target point can be converted to the robot arm coordinate system as shown in the following formula: Among them, E′ robot Indicates the coordinates of the needle insertion point in the robot arm coordinate system after the frame is tilted; T robot Indicates the coordinates of the target in the robot coordinate system after the frame is tilted.

[0174] In some embodiments, the processing device 130 may determine a collision model based on the first position information and the second position information acquired in step 410. For details on how to determine the collision model, please refer to the relevant description in step 320 and will not be repeated here.

[0175] After the scanning device is tilted, the scanning device model also needs to be updated. Due to vertical and horizontal bed movement, the patient model (including surgical instrument models already present on the patient's body) also needs to be updated in real time. In some embodiments, after step 450, the processing device 130 can update the collision model and the position information of the post-target operation position.

[0176] In some embodiments, the processing device 130 can update the scanning device model by the following method: obtaining the transformation relationship of the scanning bed coordinate system relative to the robotic arm coordinate system before the scanning device is tilted; determining a second transformation matrix of the scanning bed coordinate system based on the rotation angle of the scanning device in the second direction, and updating the transformation relationship of the scanning bed coordinate system relative to the robotic arm coordinate system based on the second transformation matrix to update the scanning device model.

[0177] In some embodiments, during preoperative scanning, the transformation relationship between the CT bed coordinate system and the robotic arm coordinate system is: Set the rack to tilt Afterwards, the CT bed coordinate system is transformed relative to the preoperative scan (transformation matrix ) can be expressed as follows:

[0178] In some embodiments, the position of the CT collision detection model (i.e., the scanning device model) after the CT gantry is tilted in the robotic arm coordinate system needs to consider the above transformation relationship and can be updated as

[0179] In some embodiments, the processing device 130 can update the patient model by the following method: obtaining the transformation relationship of the patient coordinate system relative to the robotic arm coordinate system before the scanning device is tilted; determining the third transformation matrix of the patient coordinate system based on the movement amount of the scanning bed, and updating the transformation relationship of the patient coordinate system relative to the robotic arm coordinate system based on the third transformation matrix to update the patient model.

[0180] In some embodiments, during preoperative scanning, the transformation relationship between the patient coordinate system and the robotic arm coordinate system is: Patient vertical movement Δ h , horizontal movement Δ z Afterwards, the patient collision detection model (i.e., the patient model) is transformed relative to the preoperative scan (transformation matrix ) can be expressed as follows:

[0181] In some embodiments, the position of the patient collision detection model (i.e., the scanning device model) after the CT gantry is tilted in the robotic arm coordinate system needs to consider the above transformation relationship and can be updated as

[0182] In some embodiments, the processing device 130 may plan and execute a sub-path based on the updated collision model and the position information of the rear target operation position. For details on how to plan and execute a sub-path based on the collision model and the position information of the rear target operation position, please refer to the relevant description in step 320 and will not be repeated here.

[0183] It should be noted that the above description of processes 300 and 400 is for illustration and explanation only and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to processes 300 and 400 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, the CT gantry tilt can be performed only when a tilted needle track on the head and foot side is detected during surgery, or the end effector movement path can be planned according to the CT gantry tilt at the beginning. In this way, adaptive planning can be made for situations such as tilted needle tracks on the hand and foot side at the beginning, avoiding adjustments during surgery and reducing surgical time.

[0184] The beneficial effects that may be brought about by the embodiments of this specification include but are not limited to: (1) by planning and simulating the movement path of the robot end effector (e.g., puncture needle, etc.) before surgery, the accuracy of path planning is improved and guaranteed; (2) the planned movement path is simulated and verified through a collision model, and when the planning fails, the user is prompted with prompt information to indicate the cause of failure and solution suggestions, so that the user can accurately and quickly determine the cause of failure and make corresponding adjustments to the plan in a timely manner to ensure the effectiveness of the planned path; (3) by scanning the tilt of the equipment rack during planning / surgery, the surgical process can be fully and accurately monitored in real time, reducing the difficulty of surgical operation; (4) by using a process including the above multiple methods, the user's surgical time is saved, the difficulty of using the system is reduced, and the user experience is improved. It should be noted that different embodiments may have different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other possible beneficial effects.

[0185] While the basic concepts have been described above, it will be apparent to those skilled in the art that the detailed disclosure is merely illustrative and does not limit this specification. Although not explicitly stated herein, various modifications, improvements, and revisions to this specification may be made by those skilled in the art. Such modifications, improvements, and revisions are suggested in this specification and remain within the spirit and scope of the exemplary embodiments of this specification.

[0186] This specification also uses specific terms to describe the embodiments of this specification. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "one embodiment," "an embodiment," or "an alternative embodiment" two or more times in different locations in this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of this specification may be appropriately combined.

[0187] In addition, unless expressly stated in the claims, the order of the processing elements and sequences, the use of alphanumeric characters, or the use of other names described in this specification are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the spirit and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0188] Similarly, it should be noted that, in order to simplify the presentation of this specification and thus facilitate understanding of one or more embodiments of the invention, the foregoing descriptions of the embodiments of this specification sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not imply that the subject matter of this specification requires more features than those recited in the claims. In fact, an embodiment may have fewer features than all of the features of a single disclosed embodiment.

[0189] In some embodiments, numbers are used to describe the quantity of components and attributes. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise stated, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, which may change according to the required characteristics of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the settings of such numerical values ​​are as accurate as possible within the feasible range.

[0190] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, and documents, cited in this specification is hereby incorporated by reference in its entirety. This includes application history documents that are inconsistent with or conflict with the content of this specification, as well as documents (currently or subsequently attached to this specification) that limit the broadest scope of the claims of this specification. It should be noted that if the descriptions, definitions, and / or terminology used in the accompanying materials are inconsistent or conflicting with the content of this specification, the descriptions, definitions, and / or terminology used in this specification will control.

[0191] Finally, it should be understood that the embodiments described in this specification are intended only to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly described and illustrated in this specification.

Claims

1. An interventional medical system, data-connected to a medical imaging system, wherein the medical imaging system includes a scanning device and a scanning bed, and the interventional medical system includes: An interventional surgical robot system and a processor, wherein the interventional surgical robot system includes a robotic arm and an end effector, and the processor is configured to: Acquiring first position information of the scanning device, second position information of the scanning bed, and target surgical plan information to be performed, wherein the target surgical plan information includes a plurality of target operating positions and a plurality of target operating channels corresponding to the plurality of target operating positions; Sequentially planning and executing an execution subpath between each group of adjacent target operation positions, wherein at least a portion of each execution subpath is executed within a scanning aperture of the scanning device, and sequentially planning and executing an execution subpath between each group of adjacent target operation positions includes: Based on the inclination of the target operation channel corresponding to the target operation position included in the execution sub-path to be executed relative to the second direction, the scanning device, the scanning bed and the robotic arm are controlled to be linked to execute the sub-path.

2. The system of claim 1, wherein controlling the scanning device, the scanning bed, and the robotic arm to execute the sub-path in a coordinated manner comprises: A collision model is determined based on the first position information and the second position information, where the collision model includes a patient model, a scanning device model, and a robotic arm end model.

3. The system of claim 2, wherein: The controlling the scanning device, the scanning bed, and the robotic arm to execute the sub-path in a coordinated manner includes: controlling the robotic arm and the end effector to withdraw from the scanning aperture of the scanning device to a safe position from the front target operating position among the adjacent target operating positions based on an inclination of a rear target operating channel corresponding to the rear target operating position among the adjacent target operating positions relative to a second direction, wherein the second direction is parallel to the long axis of the scanning bed; controlling the scanning bed to withdraw from the scanning aperture of the scanning device, and then controlling the scanning device to rotate in the second direction so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction, the second angle corresponding to the tilt angle of the rear target operating channel relative to the second direction; controlling the scanning bed to move at least along the second direction and enter the scanning aperture of the scanning device so that the rear target operation channel is within the scanning range of the scanning device; The robot arm and the end effector are planned and controlled to execute the execution subpath from the safety position to the rear target operation position.

4. The system of claim 2, wherein controlling the scanning device, the scanning bed, and the robotic arm to execute the sub-path in a coordinated manner comprises: Based on the inclination of the target operating channel corresponding to the first target operating position in the target surgical plan information relative to a second direction, controlling the scanning device to rotate toward the second direction so that the scanning aperture of the scanning device is inclined at a first angle relative to the second direction, the first angle corresponding to the inclination angle of the target operating channel corresponding to the first target operating position relative to the second direction, the second direction being parallel to the long axis of the scanning bed; Controlling the scanning bed to move at least along the second direction and enter the scanning aperture of the scanning device so that the target operation channel corresponding to the first target operation position is within the scanning range of the scanning device; The robot arm and the end effector are planned and controlled to execute the execution subpath from the initial safety position to the first target operation position.

5. The system according to claim 2, wherein controlling the scanning device, the scanning bed, and the robotic arm to execute the sub-path in a coordinated manner comprises: Based on the inclination of the rear target operation channel corresponding to the rear target operation position among the adjacent target operation positions relative to the second direction, the front target operation channel corresponding to the front target operation position among the adjacent target operation positions and the rear target operation channel are located in the same scanning layer, and the inclination angles of the front target operation channel and the rear target operation channel relative to the second direction are different, controlling the scanning device to rotate in the second direction so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction, so that the rear target operating channel is within a scanning range of the scanning device, wherein the second angle corresponds to an inclination angle of the rear target operating channel relative to the second direction, and the second direction is parallel to the long axis of the scanning bed; The robot arm and the end effector are planned and controlled to execute the execution subpath from the front target operation position to the rear target operation position.

6. The system according to any one of claims 3 and 5, wherein the controlling the scanning device, the scanning bed, and the robotic arm to execute the sub-path in a coordinated manner further comprises: Before controlling the scanning device to rotate in the second direction so that the scanning hole of the scanning device is tilted by a second angle relative to the second direction, the scanning device is controlled to perform a frame zeroing operation.

7. The system of claim 2, wherein controlling the scanning device, the scanning bed, and the robotic arm to execute the sub-path in a coordinated manner comprises: Based on the inclination of the rear target operation channel corresponding to the rear target operation position among the adjacent target operation positions relative to the second direction, and the front target operation channel corresponding to the front target operation position among the adjacent target operation positions and the rear target operation channel are located in the same scanning layer, The robot arm and the end effector are planned and controlled to execute the execution subpath from the front target operation position to the rear target operation position, wherein the second direction is parallel to the long axis direction of the scanning bed.

8. The system according to any one of claims 3 to 5, wherein controlling the scanning device, the scanning bed, and the robotic arm to execute the execution sub-path in a coordinated manner comprises: After controlling the scanning device to rotate and / or controlling the scanning bed to move, updating the collision model and position information of a subsequent target operating position in the adjacent target operating positions; The execution sub-path is planned according to the updated collision model and the position information of the rear target operation position.

9. The system of claim 1 , wherein the processor is further configured to: planning a movement path of the end effector based on the first position information, the second position information, and surgical plan information; In response to the movement path planning of the end effector being successful, the target surgical plan information is determined.

10. The system of claim 9, wherein planning the movement path of the end effector based on the first position information, the second position information, and surgical plan information comprises: A moving path of the end effector is planned based on the collision model and the sorting result of the operation positions in the surgical plan information.

Images