Interventional medical system

Abstract

The embodiment of the specification provides an interventional medical system, the system is connected with a medical image system, the medical image system comprises a scanning device and a scanning bed, the system comprises an interventional operation robot system and a processor, the interventional operation robot system comprises a mechanical arm and an end effector. The processor is configured to: acquire first position information of the scanning device, second position information of the scanning bed and target operation plan information to be executed, the target operation plan information comprising a plurality of target operation positions and a plurality of target operation channels corresponding thereto; sequentially plan and execute an execution sub-path between each group of adjacent target operation positions, comprising: based on the inclination of the target operation channel corresponding to the target operation position included in the execution sub-path to be executed relative to the second direction, controlling the scanning device, the scanning bed and the mechanical arm to be linked to execute the sub-path.

Description

Technical Field

[0001] This manual relates to the medical field, and in particular to an interventional medical system. Background Technology

[0002] In the current medical field, the use of robotic arms for interventional procedures is becoming increasingly common. In multi-needle interventional procedures, the procedure often needs to be performed within the scanning port (i.e., cavity) of scanning equipment, such as computed tomography (CT) and magnetic resonance imaging (MRI) devices. Taking CT as an example, after a CT scan of the patient's lesion, preoperative simulation is performed to determine the needle tract location. The robotic arm grips the puncture needle and moves it to the interventional puncture point, with the CT scan providing real-time exposure. The surgeon can then observe the entire needle insertion process from outside the CT room. However, because the effectiveness of preoperative simulation is difficult to guarantee, collisions between the robotic arm's end effector and other objects are common during actual surgery, such as with the patient, existing puncture needles, or scanning equipment. Furthermore, during real-time exposure, the puncture needle's trajectory cannot be observed in certain situations, such as during chest or abdominal punctures when facing a head-to-foot tilted needle tract. This increases the difficulty of the surgery and affects its outcome.

[0003] Therefore, it is desirable to provide an interventional medical system to ensure the effectiveness of interventional multi-needle puncture surgery. Summary of the Invention

[0004] One embodiment of this specification provides an interventional medical system that is data-connected to a medical imaging system. The medical imaging system includes a scanning device and a scanning bed. The interventional medical system includes an interventional surgical robot system and a processor. The interventional surgical robot system includes a robotic arm and an end effector. The processor is configured to: acquire first position information of the scanning device, second position information of the scanning bed, and target surgical plan information to be executed. The target surgical plan information includes multiple target operation positions and multiple target operation channels corresponding to the multiple target operation positions. The processor sequentially plans and executes execution sub-paths between each group of adjacent target operation positions. At least a portion of each execution sub-path is executed within the scanning aperture of the scanning device. The sequential planning and execution of execution sub-paths between each group of adjacent target operation positions includes: controlling the scanning device, the scanning bed, and the robotic arm to coordinate and execute the sub-path based on the tilt of the target operation channel corresponding to the target operation position included in the execution sub-path relative to a second direction.

[0005] In some embodiments, based on the inclination of the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions relative to the second direction, the robotic arm and the end effector can be controlled to withdraw from the scanning aperture of the scanning device to a safe position from the front target operation position in the adjacent target operation positions, where the second direction is parallel to the long axis of the scanning bed; the scanning bed is controlled to withdraw from the scanning aperture of the scanning device, and then the scanning device is controlled to rotate in the second direction, so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction, the second angle corresponding to the inclination angle of the rear target operation channel relative to the second direction; the scanning bed is controlled to move at least along the second direction and enter the scanning aperture of the scanning device, so that the rear target operation channel is within the scanning range of the scanning device; the robotic arm and the end effector are planned and controlled to execute the execution sub-path from the safe position to the rear target operation position.

[0006] In some embodiments, based on the tilt of the target operation channel corresponding to the first target operation position in the target surgical plan information relative to the second direction, the scanning device can be controlled to rotate in the second direction, so that the scanning aperture of the scanning device is tilted relative to the second direction by a first angle, the first angle corresponding to the tilt angle of the target operation channel corresponding to the first target operation position relative to the second direction, the second direction being parallel to the long axis direction of the scanning bed; the scanning bed is controlled to move at least along the second direction and enter the scanning aperture of the scanning device, so that the target operation channel corresponding to the first target operation position is within the scanning range of the scanning device; the robotic arm and the end effector are planned and controlled to execute the execution sub-path from the initial safe position to the first target operation position.

[0007] In some embodiments, based on the inclination of the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions relative to the second direction, and the front target operation channel corresponding to the front target operation position in the adjacent target operation positions and the rear target operation channel are located on the same scanning layer, and the inclination angles of the front target operation channel and the rear target operation channel relative to the second direction are different, the scanning device can be controlled to rotate in the second direction, so that the scanning aperture of the scanning device is tilted by a second angle relative to the second direction, so that the rear target operation channel is within the scanning range of the scanning device, the second angle corresponding to the inclination angle of the rear target operation channel relative to the second direction, and the second direction being parallel to the long axis direction of the scanning bed; the robotic arm and the end effector are planned and controlled to execute the execution sub-path from the front target operation position to the rear target operation position.

[0008] In some embodiments, before controlling the scanning device to rotate in the second direction so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction, the scanning device may be controlled to perform a rack zeroing operation.

[0009] In some embodiments, based on the inclination of the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions relative to the second direction, and the front target operation channel corresponding to the front target operation position in the adjacent target operation positions and the rear target operation channel being located in the same scanning layer, the robotic arm and the end effector can be planned and controlled to execute the execution sub-path from the front target operation position to the rear target operation position, wherein the second direction is parallel to the long axis direction of the scanning bed.

[0010] In some embodiments, a collision model can be determined based on the first location information and the second location information, the collision model including a patient model, a scanning device model, and a robotic arm end effector model.

[0011] In some embodiments, after controlling the scanning device to rotate and / or controlling the scanning bed to move, the collision model and the position information of the subsequent target operation position in the adjacent target operation positions can be updated; the execution sub-path can be planned according to the updated collision model and the position information of the subsequent target operation position.

[0012] In some embodiments, the processor may also be configured to: plan the movement path of the end effector based on the first location information, the second location information, and the surgical plan information; and determine the target surgical plan information in response to successful planning of the movement path of the end effector.

[0013] In some embodiments, the movement path of the end effector can be planned based on the collision model and the sorting results of the operation positions in the surgical plan information.

[0014] One embodiment of this specification provides another interventional medical system that is data-connected to a medical imaging system. The medical imaging system includes a scanning device and a scanning bed. The interventional medical system includes an interventional surgical robot system and a processor. The interventional surgical robot system includes a robotic arm, and the robotic arm includes an end effector. The processor is configured to: acquire first position information of the scanning device, second position information of the scanning bed, and surgical plan information. The surgical plan information includes at least one operating position and at least one operating channel corresponding to the at least one operating position. Based on the first position information, the second position information, and the surgical plan information, the processor plans a movement path for the end effector. In response to successful planning of the movement path of the end effector, the processor determines target surgical plan information. Attached Figure Description

[0015] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0016] Figure 1 These are schematic diagrams illustrating application scenarios of interventional medical systems based on some embodiments of this specification;

[0017] Figure 2 These are schematic diagrams of interventional medical systems illustrated in some embodiments of this specification;

[0018] Figure 3 This is an exemplary flowchart of a surgical path planning method according to some embodiments of this specification;

[0019] Figure 4 This is an exemplary flowchart of another surgical path planning method according to some embodiments of this specification;

[0020] Figure 5 This is a schematic diagram illustrating the needle track arrangement according to some embodiments of this specification;

[0021] Figure 6 This is another schematic diagram of needle arrangement according to some embodiments of this specification;

[0022] Figure 7 This is another schematic diagram of needle arrangement according to some embodiments of this specification;

[0023] Figure 8 This is a schematic diagram of the end effector of a surgical robot according to some embodiments of this specification;

[0024] Figure 9 This is a schematic diagram of the end effector of another surgical robot according to some embodiments of this specification;

[0025] Figure 10 This is a schematic diagram illustrating the path planning failure caused by a collision between the robotic arm end effector model and the patient model, as shown in some embodiments of this specification.

[0026] Figure 11 This is a schematic diagram illustrating the adjustment of the needle path after path planning fails, as shown in some embodiments of this specification.

[0027] Figure 12 This is a schematic diagram illustrating the path planning failure caused by a collision between the robotic arm end effector model and the scanning device model, as shown in some embodiments of this specification.

[0028] Figure 13This is a schematic diagram illustrating the adjustment of the needle path after another path planning failure, as shown in some embodiments of this specification;

[0029] Figure 14 This is a schematic diagram illustrating the path planning failure caused by a collision between the robotic arm end effector model and the surgical instrument model, as shown in some embodiments of this specification.

[0030] Figure 15 This is a schematic diagram illustrating the adjustment of the needle path after another path planning failure, as shown in some embodiments of this specification;

[0031] Figure 16 This is a schematic diagram of a needle path with cephalad tilt when the CT gantry is not tilted, as shown in some embodiments of this specification;

[0032] Figure 17 This is a schematic diagram showing the calculation of the CT gantry tilt angle based on some embodiments shown in this specification;

[0033] Figure 18 This is a schematic diagram illustrating the calculation of the vertical movement of the CT bed according to some embodiments shown in this specification;

[0034] Figure 19 This is a schematic diagram illustrating the calculation of the horizontal movement of the CT bed according to some embodiments of this specification. Detailed Implementation

[0035] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0036] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0037] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0038] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0039] Figure 1 These are schematic diagrams illustrating application scenarios of medical systems based on some embodiments of this specification.

[0040] In this manual, the interventional medical system 100 will be referred to simply as System 100. For example... Figure 1 As shown, in some embodiments, system 100 may include medical imaging system 110, interventional surgical robot system 120, processing device 130, terminal 140, storage device 150 and network 160.

[0041] Medical imaging system 110 refers to a system capable of reproducing images of the internal structures of the human body. In some embodiments, medical imaging system 110 may include a scanning device and a scanning bed. The scanning device may be any medical imaging device capable of imaging or treating designated body parts of a patient, such as a CT scanner, MR scanner, positron emission tomography (PET) scanner, single-photon emission computed tomography (SPECT) scanner, etc.; the scanning bed is used to place the patient. This specification will use a CT scanner as an example in the following description.

[0042] like Figure 2 As shown, in some embodiments, the medical imaging system 110 may include a scanning device 111 and a scanning bed 112. Figure 2 The scanning device 111 is a CT scanner, and the scanning bed 112 is an operating table (an operating table corresponding to a CT scanner can also be called a CT bed). The CT scanner performs scanning imaging and / or treatment on the patient placed on the operating table. In some embodiments, the scanning device 111 may include a gantry ( Figure 2 (Not shown) and scanning aperture 113, the scanning aperture being located in the middle of the gantry and being a perforated structure formed by the inner side of the gantry. When scanning a patient, the patient is placed on the scanning bed 112, and the scanning area is placed within the scanning aperture by the rotation of the gantry and the movement of the scanning bed 112. In some embodiments, the scanning aperture may include the cavity of a single medical imaging device or a cavity formed by combining multiple medical imaging devices.

[0043] In some embodiments, the medical imaging system 110 may exchange data and / or information with other components in the system 100 (e.g., interventional surgical robot system 120, processing device 130, storage device 150, terminal 140) via network 160. In some embodiments, the medical imaging system 110 may be directly connected to other components in the system 100. In some embodiments, one or more components in the system 100 (e.g., processing device 130, storage device 150) may be included within the medical imaging system 110.

[0044] Interventional surgical robot system 120 refers to a medical robot system capable of performing surgical treatment on a patient. Interventional surgical robot system 120 may include various types of surgical robots for performing interventional procedures, such as percutaneous puncture surgical robots. In some embodiments, interventional surgical robot system 120 may include a robotic arm for positioning surgical instruments between the patient and the robotic arm, which is connected to the surgical instruments via an end effector. An end effector is a device for positioning surgical instruments and / or connecting the robotic arm to the surgical instruments, such as an end gripper, end aid, etc. In some embodiments, the end effector can fix the surgical instruments to the end of the robotic arm in the surgical robot system, fixing their position relative to the robotic arm. Surgical instruments are instruments used to perform surgery on a patient and may include various types, such as puncture needles, scalpels, laser emitters, etc. In some cases, because the surgical instruments are connected to the end effector, in some embodiments, the end portion of the robotic arm, the end effector, and the surgical instruments connected to the robotic arm via the end effector can be considered as a single unit, collectively referred to as the end effector. In some embodiments, when a surgical instrument is attached to the end effector, the movement path of the end effector may also represent the movement path of the surgical instrument.

[0045] like Figure 2 As shown, in some embodiments, the interventional surgical robot system 120 may include a robotic arm 121 and an end effector 122. Figure 2 The robotic arm 121 is an interventional surgical robot, and the end effector 122 is connected to a puncture needle. The interventional surgical robot performs interventional puncture surgery on the patient on the operating table through the puncture needle fixed at its end. In some embodiments, the interventional surgical robot system 120 can perform surgery on the patient based on a surgical plan formulated by the processing device 130, such as interventional multi-needle puncture surgery.

[0046] As an example, the procedure for interventional multi-needle puncture surgery can be as follows: First, a CT scan is performed on the patient's surgical site on the CT table; after the scan, the user (e.g., the doctor) determines the positions of all puncture needle tracts on the interface and sorts the needle tracts; needle tract simulation verification is performed; after the needle tract simulation verification is passed, the operating table is moved to the planned position, and the CT gantry is tilted to the planned angle corresponding to that position; the end effector of the surgical robot's robotic arm grips the puncture needle and moves it to the corresponding puncture needle tract position according to the planned path, and the CT real-time exposure guides the user to perform the puncture surgery.

[0047] Processing device 130 can process data and / or information obtained from other devices or system components, and execute the surgical path planning method shown in some embodiments of this specification based on this data, information, and / or processing results to perform one or more functions described in some embodiments of this specification. For example, processing device 130 can plan the movement path of end effector 122 based on the position information of scanning device 111 and scanning bed 112 in medical imaging system 110 and surgical plan information to determine the target surgical plan. As another example, processing device 130 can prompt the user when path planning fails. For another example, during surgeries such as thoracic or abdominal punctures, for cephalad-foot-tilted needle tracts, processing device 130 can tilt the gantry of scanning device 111 so that the user can see the puncture needle trajectory during real-time exposure of scanning device 111. In some embodiments, processing device 130 can send the processed data, such as the target surgical plan and the sorting results of the target operation position, to storage device 150 for storage. In some embodiments, the processing device 130 may obtain pre-stored data and / or information, such as surgical plan information, component location information, etc., from the storage device 150 for use in executing the surgical path planning method shown in some embodiments of this specification.

[0048] In some embodiments, the processing device 130 may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-chip processing device). By way of example only, the processing device 130 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction processor (ASIP), a graphics processing unit (GPU), a physical processor (PPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, or any combination thereof.

[0049] Terminal 140 can control the operation of the surgery. The doctor can issue operation commands to the medical imaging system 110 through terminal 140 to cause the medical imaging system 110 to perform specified operations, such as imaging a specified body part of the patient. The doctor can also issue operation commands to the interventional surgical robot system 120 through terminal 140 to cause the interventional surgical robot system 120 to perform specified operations, such as executing a target surgical plan and performing surgery on the patient. In some embodiments, terminal 140 can instruct processing device 130 to execute surgical path planning methods as shown in some embodiments of this specification. In some embodiments, terminal 140 may include a display component through which processing device 130 can issue prompts to the user, such as information indicating surgical path planning failure and / or success. In some embodiments, terminal 140 may be one or any combination of devices with input and / or output functions, such as mobile device 140-1, tablet computer 140-2, laptop computer 140-3, and desktop computer 140-4.

[0050] Storage device 150 can store data or information generated by other devices. In some embodiments, storage device 150 can store data and / or information acquired by medical imaging system 110, such as scanned images of target objects like phantoms, patients, etc. In some embodiments, storage device 150 can store data and / or information processed by processing device 130, such as target surgical plans, parameters of scanning device gantry tilt, etc. Storage device 150 may include one or more storage components, each of which may be a separate device or part of other devices. Storage devices may be local or implemented via the cloud.

[0051] Network 160 can connect the various components of the system and / or connect the system to external resources. Network 160 enables communication between the components and with other parts outside the system, facilitating the exchange of data and / or information. In some embodiments, one or more components in system 100 (e.g., medical imaging system 110, interventional surgical robot system 120, processing device 130, storage device 150, terminal 140) can send data and / or information to other components via network 160. In some embodiments, network 160 can be any one or more of a wired network or a wireless network.

[0052] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this specification. Various changes and modifications can be made by those skilled in the art based on the content of this specification. Features, structures, methods, and other features of the exemplary embodiments described herein can be combined in various ways to obtain other and / or alternative exemplary embodiments. For example, the processing device 130 may be based on a cloud computing platform, such as a public cloud, private cloud, community cloud, and hybrid cloud. However, these changes and modifications will not depart from the scope of this specification.

[0053] Figure 3 This is an exemplary flowchart of a surgical path planning method according to some embodiments of this specification.

[0054] like Figure 3 As shown, process 300 includes the following steps. In some embodiments, process 300 may be executed by processing device 130. In some embodiments, the surgical path planning method shown in process 300 can be used for path planning in puncture surgery, wherein it can be used for single-needle planning or multi-needle planning.

[0055] Step 310: Obtain the first position information of the scanning device, the second position information of the scanning bed, and the surgical plan information. The surgical plan information includes one or more operating positions and one or more operating channels corresponding to these operating positions.

[0056] The first position refers to the position of the scanning device. First position information refers to information that indicates the first position of the scanning device, and may include various types of information, such as the horizontal and vertical positions of the scanning device gantry, the tilt angle of the scanning device gantry, etc. The second position refers to the position of the scanning bed. Second position information refers to information that indicates the second position of the scanning bed, and may include various types of information, such as the vertical position (bed height) and horizontal position of the scanning bed, etc. In some embodiments, the first position may include the initial position and / or real-time position of the scanning device. In some embodiments, the second position may include the initial position and / or real-time position of the scanning bed. In some embodiments, the processing device 130 can determine the starting position of the scanning device and the scanning bed in the planned surgical path based on the first position of the scanning device and the second position of the scanning bed. For example, the first position of the scanning device and the second position of the scanning bed can be directly used as the starting position; alternatively, a preset offset can be added to both the first position of the scanning device and the second position of the scanning bed to use as the starting position.

[0057] In some embodiments, the first position and the second position may be the same as or different from the initial surgical position in the surgical plan information. In some embodiments, the first position and the second position may include the initial surgical positions of the scanning device and the scanning bed in the surgical plan information, and the processing device 130 can obtain the initial surgical positions of the scanning device and the scanning bed from the surgical plan information. In some embodiments, the first position and the second position may be preset position information. For example, a fixed position may be set for each of the scanning device and the scanning bed as their respective initial positions, and the processing device 130 may obtain the preset position information by reading a configuration file or other means. In some embodiments, the first position and the second position may include the current position of the scanning device and the scanning bed, and the processing device 130 may obtain the current position of the scanning device and the scanning bed in various ways, such as through sensors on the scanning device and the scanning bed, or through images captured by an external camera. In some embodiments, the processing device 130 may obtain the current position of the scanning device and the scanning bed at any given time as the first position and the second position.

[0058] A surgical plan refers to the plan for a surgery that requires path planning. Surgeries can include various types, such as vascular surgery and puncture procedures. Surgical plan information refers to information that represents the surgical plan, such as the type of surgery, the timing of the surgery, the surgeon, the patient, and the surgical site. In some embodiments, surgical plan information may include the initial positions of the scanning equipment and scanning table at the start of the surgery, and / or the positions that the scanning equipment and scanning table should reach during the surgery.

[0059] The operating location refers to the position on the body of the surgical subject (e.g., the patient) where the surgical procedure is performed during the surgical process, such as the needle insertion point in a puncture procedure. In some embodiments, the operating location may include at least one. The operating channel refers to the surgical path of the surgical instrument corresponding to the operating location, such as the needle path including the needle insertion point and the target point in a puncture procedure. In some embodiments, the number of operating channels may correspond to the number of operating locations, such as one-to-one, many-to-one, etc. In some embodiments, the surgical planning information may include one or more operating locations and one or more operating channels corresponding to these operating locations. For example, multiple needle insertion points and multiple needle paths corresponding to them. Unless otherwise specified, the surgical procedures described in this specification refer to multi-needle puncture procedures.

[0060] In some embodiments, the processing device 130 may acquire surgical plan information in various ways, such as from a storage device (e.g., storage device 150). This specification does not limit this.

[0061] Step 320: Plan the movement path of the end effector based on the first position information, the second position information, and the surgical plan information.

[0062] In some embodiments, the surgical planning information may include one or more operating positions and one or more operating channels corresponding to these operating positions. The end effector in the surgical robot system (e.g., end effector 122 in interventional surgical robot system 120) needs to perform corresponding surgeries along the corresponding operating channels at these operating positions. Therefore, it is necessary to determine the movement sequence, i.e., the movement path, of the end effector between these operating positions to ensure maximum robotic arm movement space. The movement path of the end effector determines the movement sequence of surgical instruments connected to the end effector (e.g., the puncture sequence of puncture needles). Since the end effector is driven by the movement of the robotic arm, in some embodiments, the movement path of the end effector can be determined by the movement paths of the various joint angles of the robotic arm.

[0063] In some embodiments, the processing device 130 may sort multiple operating positions and plan the movement path of the end effector based on the sorting result, which is also referred to as the first sorting result.

[0064] In some embodiments, the processing device 130 may define a plurality of path planning sub-regions extending along a first direction, wherein these path planning sub-regions cover a plurality of operating positions, and the first direction is parallel to the minor axis direction of the scanning bed (e.g., scanning bed 112). This is merely an example. Figure 5 and 6 This includes multiple path planning sub-regions extending along the X-axis direction of the CT table coordinate system, i.e., the first direction is parallel to the minor axis of the scanning table. In some embodiments, the first direction may be parallel to the major axis of the scanning table. This is merely an example. Figure 7 It includes multiple path planning sub-regions extending along the Z-axis direction of the CT bed coordinate system, i.e., the first direction is parallel to the long axis direction of the scanning bed.

[0065] In some embodiments, the processing device 130 may determine a first sorting result of multiple operation locations based on a path planning sub-region.

[0066] In some embodiments, the processing device 130 can sort the acquired multiple path planning sub-regions along a preset direction to obtain a first intermediate sorting result, wherein the preset direction can be parallel to the horizontal plane and perpendicular to the first direction. In some embodiments, the processing device 130 can sort the operation positions in each path planning sub-region to obtain a second intermediate sorting result. The sorting of the multiple path planning sub-regions and the sorting of the operation positions in each path planning sub-region can be performed sequentially or simultaneously. In some embodiments, the processing device 130 can obtain a first sorting result based on the first intermediate sorting result and multiple second intermediate sorting results.

[0067] In some embodiments, the processing device 130 may determine the sorting characteristic value of each path planning sub-region, and then sort these regions along a preset direction based on the sorting characteristic values ​​of all path planning sub-regions. In some embodiments, the sorting characteristic value of a path planning sub-region may include the extreme value of the operation position characteristic value in that region, such as the maximum or minimum value. In some embodiments, the sorting characteristic value of a path planning sub-region may be other values ​​related to the operation position characteristic value in that region, such as the average of all characteristic values. In some embodiments, the processing device 130 may determine the sorting characteristic value of a path planning sub-region in various ways. For example, the operation position characteristic value ranked first / last in the second intermediate sorting result of the path planning sub-region may be used as the sorting characteristic value. Another example is to directly obtain the extreme value of the operation position characteristic value in the path planning sub-region as its sorting characteristic value.

[0068] In some embodiments, the preset direction can be determined based on the position of a path planning sub-region relative to the robot arm base. For example, as... Figure 7 As shown, the first direction is parallel to the major axis of the scanning bed (i.e., the Z-axis direction of the CT bed coordinate system). If the path planning sub-region is on the left side of the robotic arm base, the preset direction can be the direction of increasing X-coordinate values ​​in the CT bed coordinate system (positive X-axis direction). Alternatively, if the path planning sub-region is on the right side of the robotic arm base, the preset direction can be the direction of decreasing X-coordinate values ​​in the CT bed coordinate system (negative X-axis direction). Another example: if the first direction is parallel to the major axis of the scanning bed (i.e., the Z-axis direction of the CT bed coordinate system), the path planning sub-regions can be sorted in order from furthest from the robotic arm base to closest to the robotic arm base, or in order from closest to the robotic arm base to furthest from the robotic arm base.

[0069] In some embodiments, the preset direction can be determined based on the position of a path planning sub-region relative to the scanning device. For example, such as Figure 5 and 6As shown, the first direction is parallel to the minor axis of the scanning bed (i.e., the X-axis direction of the CT bed coordinate system). The path planning sub-regions can be sorted along the Z-axis direction of the CT bed coordinate system, from inside the scanning cavity aperture to outside the scanning cavity aperture. In other words, the preset direction can be the negative direction of the Z-axis of the CT bed coordinate system. Alternatively, the preset direction can be the positive direction of the Z-axis of the CT bed coordinate system, meaning the path planning sub-regions can be sorted along the Z-axis direction from outside the scanning cavity aperture to inside the scanning cavity aperture.

[0070] In some embodiments, the end effector may include an end gripper (e.g., a jaw) for gripping a puncture needle, and a preset orientation may be determined based on the opening orientation of the end gripper. In some embodiments, the processing device may first determine the opening orientation of the end gripper at multiple needle insertion points, and then determine the preset orientation based on the opening orientation. In some embodiments, when the end gripper's opening orientation at most needle insertion points faces into the scanning cavity aperture of the scanning device, the preset orientation may be away from the aperture (i.e., outward from the aperture). In some embodiments, when the end gripper's opening orientation at most needle insertion points faces outward from the scanning cavity aperture of the scanning device, the preset orientation may be towards the aperture (i.e., inward from the aperture). In some embodiments, the preset orientation may be the opposite direction to the opening orientation of the end gripper at most needle insertion points.

[0071] The second intermediate sorting result is the sorting of all operation positions in a single region. In some embodiments, in each of the multiple path planning sub-regions, the processing device 130 can sort the operation positions according to a preset sorting rule to obtain the second intermediate sorting result. The preset sorting rule can include various methods. In some embodiments, the processing device 130 can sort based on the feature values ​​of the operation positions, where the feature values ​​can include various values ​​that can represent the operation position, such as the coordinate values ​​of the operation position (e.g., coordinate values ​​along a first direction or a second direction), a measure of the importance of the operation position, etc. In some embodiments, the processing device 130 can determine the preset sorting rule based on the shape of the end effector at the end of the robotic arm.

[0072] In some embodiments, the end effector may include an end gripper (e.g., a gripper), and a preset sorting rule can be determined based on the opening orientation of the end gripper. Specifically, the processing device 130 can determine the opening orientation of the end gripper at at least one operating position, and sort at least one operating position in a path planning sub-region based on the opening orientation. Taking the end gripper as a gripper as an example, the processing device 130 can first determine the opening orientation of the gripper at multiple needle insertion points, and determine the sorting of the needle insertion points covered by one or more path planning sub-regions based on the opening orientation. For example, the puncture region includes multiple needle insertion points, and each needle insertion point corresponds to a gripper opening orientation. Several needle insertion points on each path planning sub-region correspond to several gripper opening orientations. If the majority of gripper opening orientations in the puncture region face the inside of the scanning aperture, the processing device 130 can sort them in order from inside the scanning cavity aperture to outside the scanning cavity aperture of the scanning device (e.g., when the positive Z-axis direction points into the scanning aperture, sorting is done in descending order based on the Z coordinate value of the needle insertion point). If most of the gripper openings in the puncture area face outwards from the scanning aperture, the default sorting rule is to sort them in order from outside the scanning cavity aperture to inside the scanning cavity aperture of the scanning device (for example, when the positive Z-axis direction points into the scanning aperture, they are sorted in ascending order according to the Z coordinate value of the needle insertion point).

[0073] In some embodiments, the opening orientation of the end effector gripper of the robotic arm can be determined by: determining the position of the origin of the end effector coordinate system at the needle insertion point and the end effector coordinate system (e.g., based on the needle insertion point and the target point). Figure 8 and 9 A coordinate system consisting of X2, Y2, and Z2 (established with the endpoint of the end effector furthest from the robotic arm 590 as the origin) is used. The orientation of the Z2 axis is determined by arbitrarily choosing a direction within a plane passing through the origin of the end effector coordinate system and perpendicular to its Z2 axis. The orientation of the X2 axis is then determined based on the Z2 and Y2 axis orientations. Inverse kinematics is used to verify the rationality of this end effector coordinate system (before end effector retraction). If it is not rational, the Z2 axis of the end effector coordinate system is rotated via the robotic arm end effector to continue searching for a new orientation. If it is rational, the position and orientation of the end effector coordinate system are determined, and the X2 axis of the end effector coordinate system is the opening orientation of the end effector gripper assembly.

[0074] In some embodiments, the end effector can be an auxiliary positioner that assists in needle puncture. A preset sorting rule can be determined based on the shape of the auxiliary positioner. For example, when the end effector is L-shaped (e.g., ...), Figure 8 When using the auxiliary locator shown, such as Figure 7As shown, in each path planning sub-region along the Z-axis of the CT bed coordinate system, the needle insertion points can be sorted according to the order from outside to inside the scanning cavity aperture of the scanning device (for example, when the positive Z-axis points into the aperture, the puncture positions are sorted in ascending order based on the Z-coordinate values). Simultaneously, for the sorting of multiple path planning sub-regions, the puncture positions in adjacent path planning sub-regions are sorted in the order from furthest from the robotic arm base to closest to the robotic arm base (i.e., sorted in ascending order based on the X-coordinate values). This sorting method can be called a quasi-N-shaped sorting method. For example, when the end effector is I-shaped (e.g., ... Figure 9 When using the auxiliary locator shown, such as Figure 5 and 6 As shown, in each path planning sub-region along the X-axis of the CT bed coordinate system, the needle entry points are sorted in order from furthest from the robotic arm base to closest to the robotic arm base (for example, when the positive X-axis points towards the base, the needle entry points are sorted in ascending order based on their X-coordinate values). Simultaneously, for the sorting of multiple path planning sub-regions, the needle entry points in adjacent path planning sub-regions are sorted in order from inside to outside the scanning cavity aperture of the scanning device (i.e., sorted in descending order based on their Z-coordinate values). This sorting method can be called a zigzag sorting method.

[0075] In some embodiments, in each of the plurality of path planning sub-regions, the processing device 130 may sort the operation positions along a first direction to obtain a second intermediate sorting result. For example, as Figure 5 and 6 As shown, the first direction is parallel to the minor axis of the scanning bed, i.e., the X-axis of the CT bed coordinate system. In each path planning sub-region, the direction is determined by the X-coordinate values ​​from smallest to largest. Figure 5 ) or the direction from largest to smallest X coordinate value ( Figure 6 Sort the needle entry points.

[0076] In some embodiments, the processing device 130 can sort the operation positions according to the first intermediate sorting result and multiple second intermediate sorting results to obtain a sorting result of all the operation positions, i.e., the first sorting result. In some embodiments, the processing device 130 can arrange the second intermediate sorting results of each path planning sub-region along a preset direction according to the sorting result of the path planning sub-region (the first intermediate sorting result) to obtain the first sorting result.

[0077] In some embodiments, the sorting described above can use any sorting algorithm. For example, sorting can be done according to the coordinate values ​​of the operation position along a certain direction. In some embodiments, the sorting method can also be user-initiated sorting, or a combination of sorting algorithm and user-initiated sorting.

[0078] As an example, when planning the puncture sequence of the puncture needles, since the entry points of all needle paths are known, assuming the coordinates of the entry points are located in the CT table coordinate system, the processing device 130 can select the entry point with the smallest / largest Z-axis coordinate value. Based on the X-axis / Z-axis coordinate values ​​of this point, the space is divided into multiple parallel regions along the X-axis / Z-axis direction. The spacing between these regions can be equal or unequal, and all entry points are located within these regions. The following explanation uses the division of regions along the Z-axis direction as an example. For different entry points within the same region, priority is given to planning the needle path corresponding to the entry point furthest from the robotic arm. That is, within each region, all entry points are sorted according to their X-axis coordinate values ​​from smallest to largest. Based on the entry point with the smallest / largest Z-axis coordinate value in each region, all regions are sorted, and priority is given to planning the needle path corresponding to the entry point in the region with the smallest / largest Z-axis coordinate value. Specifically, the smallest / largest Z-axis coordinate of the entry point in each region can be used as the Z-axis coordinate value of that region, and each region is planned sequentially according to the region coordinate value sorting result (from smallest to largest or from largest to smallest). In some embodiments, the processing device 130 may establish a CT bed coordinate system as follows: with a point on the CT bed as the origin, the minor axis of the CT bed is the X-axis, with the positive direction of the X-axis pointing to the right; the direction perpendicular to the surface of the CT bed is the Y-axis, with the positive direction of the Y-axis pointing from the lower surface of the CT bed to the upper surface; the major axis of the CT bed is the Z-axis, with the positive direction of the Z-axis pointing in the direction of the CT bed entering the scanning aperture. In some embodiments, the processing device 130 may establish a CT bed coordinate system as follows: the CT bed is moved into the scanning aperture, and when the bed code value displayed on the CT device operation interface becomes 0, the CT isocenter coincides with the origin of the CT bed coordinate system. At this time, the position of the CT isocenter relative to the CT bed is the position of the origin of the CT bed coordinate system.

[0079] Figure 5 , 6 This is a schematic diagram illustrating the needle track arrangement according to some embodiments of this specification. Wherein, Figure 5 In the image, the robotic arm 590, mounted on its base, is positioned to the right of the CT scan port. Figure 6 In the middle, the robotic arm 590 is positioned to the left of the CT scanning port. Figure 5 and 6In the CT bed coordinate system, the X-axis is the minor axis of the CT bed, with the positive direction pointing to the right. The Z-axis is the major axis of the CT bed, with the positive direction of the Z-axis indicating the direction of the CT bed entering the scanning aperture, i.e., the direction of the CT bed aperture diameter. 510, 520, 530, 540, 550, 550, 560, 570, and 580 are all needle entry points, i.e., the operation positions in the surgical plan.

[0080] like Figure 5 As shown, assuming the row spacing is set to 'a' when dividing the regions, first find the needle insertion point 520 with the largest Z-axis coordinate value among all needle insertion points in the CT table coordinate system. Then, divide the region into ±a / 2 division planes through point 520, with a distance of 'a' between the two division planes. Continue dividing the region into equally spaced regions (row spacing of 'a') in the negative Z-axis direction. Each needle insertion point is assigned to a region. Within the same region, prioritize planning needle insertion points farther from the robotic arm 590, i.e., plan each needle insertion point in order of distance from the robotic arm 590 from farthest to closest. In the region containing 510 and 520, plan 510 and 520 in sequence; in the region containing 530-550, plan 530, 540, and 550 in sequence; in the region containing 560-580, plan 560, 570, and 580 in sequence. Sort all regions according to the minimum / maximum Z-axis coordinate value of each region, and prioritize planning the region containing the needle entry point with the larger Z-axis coordinate value. That is, plan in the following order from front to back: the region containing 510 and 520, the region containing 530-550, and the region containing 560-580. The final planned order of the needle entry points is as follows: 510, 520, 530, 540, 550, 550, 560, 570, 580.

[0081] like Figure 6 As shown, it is similar to Figure 5 The main difference lies in the placement of the robotic arm 590, which is to the left of the CT scan aperture. Therefore, when divided into... Figure 5 After traversing the same regions, the sorting results for each region are also the same. However, when sorting the needle entry points within each region, the needle entry points are planned sequentially according to their distance from the robotic arm 590, from farthest to closest. Therefore, in the regions containing 510 and 520, 520 and 510 are planned sequentially; in the regions containing 530-550, 550, 540, and 530 are planned sequentially; and in the regions containing 560-580, 580, 570, and 560 are planned sequentially. Thus, the final planned order of the needle entry points, from first to last, is: 520, 510, 550, 540, 530, 550, 580, 570, and 560.

[0082] In some embodiments, when the scenario is image-guided online or offline puncture, the processing device 130 can sort the data in order from the furthest point from the robotic arm to the closest point to the robotic arm to obtain a first sorting result, thus increasing the available space for planning. In some embodiments, when the scenario is offline puncture, the processing device 130 can sort the data in order from the closest point to the robotic arm to the furthest point from the robotic arm to obtain a first sorting result, which facilitates the doctor's operation.

[0083] In some embodiments, the processing device 130 can plan the movement path of the end effector based on a first sorting result. The order of the various operating positions on the movement path corresponds to the execution order of each operating position during surgery. By sorting the operating positions and planning the path based on the first sorting result, the success rate of path planning is improved.

[0084] In some embodiments, the processing device 130 may connect the various operating positions (e.g., needle insertion points, etc.) with line segments according to a first sorting result, thereby forming a planned movement path for the end effector.

[0085] A patient model is a model used to simulate a physical patient and their motion trajectory. It can be implemented in various ways, such as a physical phantom or a computer-simulated patient body. In some embodiments, the patient model can be determined based on the patient's position and appearance contour information at different times. A scanning device model is a model used to simulate a physical scanning device and its motion trajectory. It can be implemented in various ways, such as scanning device motion simulation or computer-simulated scanning device motion. In some embodiments, the scanning device model can be determined based on the position and appearance contour information of the entire or part of the scanning device (e.g., the scanning aperture) at different times. A robotic arm end effector model is a model used to simulate the end effector of a robotic arm and the surgical instruments connected to it, as well as the motion trajectory of these objects. It can include the end effector portion of the robotic arm, the end effector, and the surgical instruments. The surgical instruments can be connected to the robotic arm via the end effector; therefore, the end effector portion of the robotic arm, the end effector, and the surgical instruments can be considered as a whole. A collision model is a model used to simulate whether multiple objects will collide during their mutual movement. It can simulate various types of objects, such as a robotic arm, an end effector, a patient, a scanning device, and surgical instruments connected to the robotic arm. In some embodiments, the collision model can include a patient model, a scanning device model, and a robotic arm end effector model.

[0086] In some embodiments, the processing device 130 can determine a collision model based on first and second position information. Specifically, since the patient is placed on the scanning bed and their position relative to the scanning bed is fixed, the processing device 130 can determine the relative position of the scanning device and the scanning bed at any given time based on their real-time positions; that is, the relative position of the scanning device and the patient at any given time is also determined. Therefore, when a patient model is used to represent the patient and a scanning device model is used to represent the scanning device, the processing device 130 can determine a collision model based on the relative position of the scanning device and the patient at any given time.

[0087] In some embodiments, based on the tilt of one or more operating channels (e.g., the needle track in an interventional puncture procedure) relative to a second direction, the processing device 130 can determine the rotation information of the scanning device in the second direction, wherein the second direction may be parallel to the long axis of the scanning bed. Since the cephalad direction of the patient on the scanning bed is parallel to the long axis of the scanning bed, the rotation of the scanning device in the second direction is equivalent to the collimation centerline of the scanning device tilting in the cephalad direction, thereby facilitating the observation of the cephalad tilted needle track. In this specification, cephalad tilt refers to tilting relative to the cephalad direction of the patient. In some embodiments, the processing device 130 can acquire the tilt angle of one or more operating channels relative to the long axis of the scanning bed, which in a puncture procedure may be referred to as the needle track cephalad tilt angle. In some embodiments, the processing device 130 can determine the rotation angle of the scanning device in the second direction based on the needle track cephalad tilt angle, i.e., the angle between the collimation centerline of the scanning device and the long axis of the scanning bed, wherein this angle may be referred to as the scanning device tilt angle. In some embodiments, the tilt angle of the scanning device and the tilt angle of the needle track in the direction of foot can be equal, or the difference between the two can meet a preset condition, wherein the preset condition can be that the operation channel is within the scanning range of the scanning device. When the scanning device is a CT scanner, the rotation of the scanning device can be equivalent to the rotation of the CT gantry; therefore, the tilt angle of the scanning device at this time can be called the CT gantry tilt angle.

[0088] In some embodiments, the scanning bed can be moved (e.g., moved along a second and / or third direction), and the processing device 130 can determine the movement information of the scanning bed based on the rotation information of the scanning device in the second direction. For details on how the movement information of the scanning bed is determined, please refer to the relevant description of step 450, which will not be repeated here.

[0089] In some embodiments, when planning the path between the operating positions and adjacent operating positions of one or more operating channels, the processing device 130 can determine a scanning device model based on first position information and rotation information, and determine a patient model based on second position information and movement information. Since the position of the scanning device changes after it rotates in the second direction, the processing device 130 can determine the scanning device model after rotating in the second direction based on the original position information of the scanning device (i.e., the first position information) and its position change information (i.e., rotation information). Since the patient's position changes after the scanning bed moves, the processing device 130 can determine the moved patient model based on the original position information of the scanning bed (i.e., the second position information) and its position change information (i.e., movement information).

[0090] In some embodiments of this specification, by tilting the gantry of the scanning device (e.g., a CT scanner) toward the cephalofoot side, it is easier to observe the complete needle path during surgery, especially the needle path tilted toward the cephalofoot side, thereby reducing the difficulty of the surgery and reducing the operation time.

[0091] In some embodiments, the processing device 130 can plan the movement path of the end effector based on a collision model and a first sorting result. Specifically, the processing device 130 can plan an initial movement path of the end effector based on the first sorting result, and then verify the movement path through simulation based on the collision model. If, during the verification process, the end effector collides with at least one of other objects (e.g., a scanning device, a patient, surgical instruments, etc.), the verification fails and the path planning fails; otherwise, the verification succeeds and the path planning succeeds.

[0092] In some embodiments of this specification, the movement path of the end path is planned based on the collision model and the sorting results of all operation positions (e.g., needle insertion point, etc.), making the path planning more realistic and the planned path more reasonable, thus avoiding the risk of path adjustment during surgery.

[0093] After the scanning bed moves, the positions of each operating position change relative to before the scanning bed moves. In some embodiments, the processing device 130 may update one or more operating positions based on the scanning bed movement information, and then plan the movement path of the end effector according to the updated one or more operating positions.

[0094] In some embodiments, after the end effector's movement path is planned based on the first sorting result, the processing device 130 can perform path verification, that is, simulate the operation based on the collision model and the planned end effector movement path, and determine whether the end effector can reach these operation positions sequentially according to the order of the first sorting result through the collision model. If the path verification is successful, the movement path planning is determined to be successful; if the path verification fails, the movement path planning is determined to be unsuccessful. The entire movement path must pass all verifications at once before proceeding to the subsequent intraoperative stage.

[0095] In some embodiments, the movement path may include a verification path, and the verification path may include one or more verification sub-paths. The processing device 130 may sequentially plan the verification sub-paths between each group of adjacent operation positions in the first sorting result.

[0096] In some embodiments, for each group of adjacent operation positions in the first sorting result, the processing device 130 can plan a verification shift path segment and a verification avoidance point based on the preceding operation position in that group of adjacent operation positions. The preceding operation position refers to the operation position that appears first in the sorting result (e.g., the first sorting result, the second sorting result) within a group of adjacent operation positions. Correspondingly, the following operation position refers to the operation position that appears last in the sorting result within a group of adjacent operation positions.

[0097] To prevent uncontrolled interference and collisions between the end effector and surgical instruments (e.g., puncture needles) left on the patient's body surface during interventional procedures, the initial path from the previous operating position to the subsequent operating position allows the end effector to retract horizontally a certain distance. This operation is called end effector retraction, and the retraction path can be called the verification shift path segment. One end of the verification shift path segment is the previous operating position, and the other end is the verification avoidance point. In some embodiments, when the end effector moves within the verification shift path segment, the attitude of the end effector at the end of the robotic arm remains unchanged, but the position of the end effector moves horizontally a certain distance. For example, when the end effector is an auxiliary locator, in the verification shift path segment, the end effector will move horizontally a certain distance in the opposite direction to the opening of the positioning slot. In some embodiments, the processing device 130 can preset the retraction distance of the end effector, and then, based on this retraction distance and the coordinate system of the end effector before retraction (e.g., ...), ... Figure 8 and 9 The coordinate system consisting of X2, Y2, and Z2) and the robot arm coordinate system (such as...) Figure 7-9 The transformation relationship of the robot arm coordinate system (composed of X1, Y1 and Z1) is obtained to obtain the transformation relationship between the end effector coordinate system and the robot arm coordinate system after retraction. Then, based on this transformation relationship, the position of the verification avoidance point is obtained, that is, the coordinates of the verification avoidance point in the robot arm coordinate system.

[0098] In some embodiments of this specification, by retracting the end effector, the surgical instruments remaining on the patient's body surface and the grippers of the end effector can be separated by a certain distance, thereby avoiding interference and collision between the end effector and the surgical instruments remaining on the patient's body surface, avoiding safety risks, and at the same time, the surgical instrument model of the previous operation position can be added at the calibration avoidance point.

[0099] During interventional multi-needle puncture surgery, the puncture needles from the first few punctures may remain on the patient's body surface. These needles need to be considered when planning the end effector's movement path to avoid collisions and interference between the end effector and these needles as it moves along the planned path. Therefore, the models corresponding to these needles need to be added to the collision model at appropriate times. In some embodiments, after the end effector moves along the verification displacement path segment, the processing device 130 can add the surgical instrument model corresponding to the previous operation position to the collision model to update the collision model. The surgical instrument model is a model used to simulate the physical surgical instruments (e.g., puncture needles) left on the patient's body surface during interventional surgery and their movement trajectories. Since the positions of these surgical instruments and the patient are relatively fixed, their movement can be equated to the movement of the patient model.

[0100] In some embodiments, the processing device 130 can plan a verification avoidance path segment between the verification avoidance point and the subsequent operation position among the aforementioned adjacent operation positions based on the updated collision model, wherein the verification shift path segment and the verification avoidance path segment can form a verification sub-path.

[0101] In some embodiments, for any verification sub-path, the processing device 130 can verify through simulation whether the end effector in the updated collision model will collide with at least one of the surgical instrument model, patient model, scanning device model, etc. If the simulation result shows that the end effector will collide with any one or more of the three, the verification in the verification sub-path fails, and the movement path planning of the end effector is determined to be unsuccessful; if the simulation result shows that the end effector will not collide with any of the three, the verification of the verification sub-path is successful, and the next verification sub-path is verified; when all verification sub-paths are verified successfully, the movement path planning of the entire end effector is successful.

[0102] In some embodiments, the processing device 130 may also use any other path planning method to plan the movement path of the end effector. For example, a trained machine learning model may be used for path planning, wherein the model can be trained on a large amount of historical path planning sample data.

[0103] In some embodiments, in addition to verifying the planned movement path of the end effector, the processing device 130 may also verify other aspects related to the surgical procedure, such as checking whether the operation channel (e.g., needle path) interferes with important tissues, whether the operation channel is drawn incorrectly, and whether the length of the operation channel meets the requirements of the corresponding instrument. In some embodiments, if at least one of these other verifications fails, the processing device 130 may also determine that the end effector movement path planning has failed. In some embodiments, if the planned end effector movement path verification is successful, and the other verifications are also successful, the processing device 130 may determine that the end effector movement path planning is successful.

[0104] In some embodiments, after the end effector's movement path planning fails, the processing device 130 may execute steps 330-350 to prompt the user and replan the movement path; after the end effector's movement path planning succeeds, the processing device 130 may execute steps 360-370 to perform surgery according to the planned movement path.

[0105] Step 330: In response to the failure of the end effector's movement path planning, an error message is output.

[0106] In some embodiments, when the movement path planning of the end effector fails, the processing device 130 may output a prompt message to the user (e.g., a doctor, etc.) in various ways, wherein the prompt message can be determined based on the reason for the planning failure.

[0107] In some embodiments, the reasons for planning failure may include at least one of the following: collision between the end effector and the patient model, collision between the end effector and the scanning device model, collision between the end effector and the instrument model, etc.

[0108] In some embodiments, the processing device 130 can adjust the operation position and corresponding operation channel according to the prompt information, and update the surgical plan information with the adjusted operation position and corresponding operation channel information, so that subsequent re-path planning can be successful after the adjustment. The adjustment can be performed in various ways, such as based on adjustment suggestions or based on experience.

[0109] As shown in Table 1 below, in some embodiments, the prompt message may include the reason for the path planning failure, and the processing device 130 may make adjustments based on the reason.

[0110] As shown in Table 1 below, in some embodiments, the prompt information may include adjustment suggestions (e.g., adjustment suggestions for needle channels, etc.), and the processing device 130 may make adjustments according to the adjustment suggestions.

[0111] In some embodiments, in response to a collision between the robotic arm end effector and the patient model causing path planning failure, the processing device 130 may output a first prompt message, wherein the first prompt message may at least instruct the user to adjust one or more operation positions in the surgical plan information. Figure 10 For example, when the planned end effector movement path is verified using a collision model, a collision is detected between the robotic arm end effector 1020 (i.e., the end effector) and the phantom 1030 (i.e., the patient model) at the needle insertion point 1010, resulting in verification failure. In this case, the processing device 130 can display a first prompt message to the user. For example, the prompt message corresponding to number 1 in Table 1 below includes the reason for the failure and adjustment suggestions. In some embodiments, the processing device 130 can adjust the needle insertion point (i.e., the operating position) and / or the corresponding needle path (i.e., the operating channel) at the verification failure location based on the suggestions in the first prompt message. For example, the needle path can be tilted cephalad / foot-to-foot according to suggestion ② in the prompt message corresponding to number 1 in Table 1. In some embodiments, the processing device 130 can adjust the relative position between the robotic arm end effector and the patient by adjusting at least one of the robotic arm and the scanning bed, thereby adjusting the operating position and / or the corresponding operating channel. For example, as... Figure 11 As shown, the processing device 130 adjusts the needle entry point 1010 by moving the robotic arm according to the prompt information corresponding to serial number 1 in the table, forming a needle path that is tilted towards the head and feet side, so that the end of the robotic arm 1020 and the model 1030 will no longer collide at the adjusted needle entry point 1010.

[0112] In some embodiments, in response to a path planning failure caused by a collision between the robotic arm end effector and the scanning device model, the processing device 130 may output a second prompt message, wherein the second prompt message may instruct the user to adjust one or more operation positions in the surgical plan information or to adjust the position of the scanning bed. Figure 12For example, when the planned end effector movement path is verified using a collision model, a collision is detected at needle entry point 1210 between the robotic arm end effector 1220 (i.e., the end effector) and the inner wall of the scanning cavity of the CT scanner 1230 (i.e., the scanning device model), resulting in verification failure. In this case, the processing device 130 can display a second prompt message to the user. For example, the prompt message corresponding to number 2 in Table 1 below includes the reason for the failure and adjustment suggestions. In some embodiments, the processing device 130 can adjust the needle entry point (i.e., the operating position) and / or the corresponding needle path (i.e., the operating channel) at the verification failure location according to the suggestions in the second prompt message. For example, according to suggestion ① of the prompt message corresponding to number 2 in Table 1, the needle entry point 1210 can be adjusted to a position closer to the center of the patient's body. In some embodiments, the processing device 130 can adjust the operating position or the position of the scanning bed by adjusting at least one of the robotic arm and the scanning bed, thereby changing the relative position between the robotic arm end effector and the patient. For example, as... Figure 13 As shown, according to the prompt information corresponding to serial number 2 in the table, the processing device 130 moves the robotic arm end 1220 a certain distance toward the middle of the patient's body, so that at the adjusted needle insertion point 1210, the robotic arm end 1220 and the CT machine 1230 will no longer collide.

[0113] In some embodiments, in response to a collision between the robotic arm end effector model and the surgical instrument model causing path planning failure, a third prompt message is output. This third prompt message can at least instruct the user to adjust one or more operational positions in the surgical plan information along the long axis of the scanning bed. Figure 14For example, when the planned end effector movement path is verified using a collision model, a collision is detected at needle insertion point 1410 between the robotic arm end effector 1420 (i.e., the end effector) and the puncture needle 1430 (i.e., the surgical instrument model), resulting in verification failure. In this case, the processing device 130 can display a third prompt message to the user. For example, the prompt message corresponding to number 3 in Table 1 below includes the reason for the failure and adjustment suggestions. In some embodiments, the processing device 130 can adjust the needle insertion point (i.e., the operation position) and / or the corresponding needle path (i.e., the operation channel) at the verification failure location based on the suggestions in the third prompt message. For example, since the needle path corresponding to the later operation position (needle insertion point 1410) (i.e., needle path X in the prompt information in Table 1 below) and the needle path of the puncture needle 1430 (i.e., needle path A in the prompt information in Table 1 below, where the operation sequence of needle path A precedes that of needle path X) are in different scanning layers, the distance between the operation positions (needle insertion points) of the two needle paths can be increased according to suggestion ② of the prompt information corresponding to serial number 3 in Table 1. In some embodiments, the processing device 130 can change the relative position between the surgical instrument at the end of the robotic arm and the patient surface by adjusting at least one of the robotic arm and the scanning bed, thereby adjusting the operation position and / or the corresponding operation channel at least along the long axis of the scanning bed. For example, as Figure 15 As shown, the processing device 130 adjusts the needle path corresponding to the needle insertion point 1410 (needle path X in the prompt information in Table 1 below) along the long axis of the scanning bed by moving the robotic arm according to the prompt information corresponding to serial number 3 in the table. That is, it increases the left and right tilt angle of the needle path so that the tilt angle is greater than the left and right tilt angle of the needle path of the puncture needle 1430, so that the end of the robotic arm 1420 and the puncture needle 1430 will no longer collide at the adjusted needle insertion point 1410.

[0114] In some embodiments, the reasons for planning failure may also include other factors, such as at least one of the following: interference between the operating channel and important tissues, incorrect drawing of the operating channel, or the operating channel length not meeting the requirements of the corresponding instrument. The processing device 130 can make corresponding adjustments based on the prompts corresponding to these reasons.

[0115] In some embodiments, the output method of the prompt information may include various methods, such as one or any combination of text, sound, image, video, animation, etc. In some embodiments, the processing device 130 may display the text prompt information to the user through a display device (e.g., the display component of the terminal 140). For example, as shown in Table 1: Table 1 In this sequence, number 1 corresponds to the collision between the robotic arm end effector model and the patient model, number 2 corresponds to the collision between the robotic arm end effector model and the scanning device model, and number 3 corresponds to the collision between the robotic arm end effector model and the surgical instrument model. The operation sequence for needle path A precedes that for needle path X.

[0116] In some embodiments of this specification, a prompt message is output after path planning fails, which helps users find the cause of the failure and select adjustment strategies more effectively.

[0117] Step 340: Reacquire the second position information of the scanning table and / or the surgical plan information. The reacquired surgical plan information includes at least one adjusted operating position and an adjusted operating channel corresponding to that adjusted operating position.

[0118] In some embodiments, after adjusting the operation position and corresponding operation channel in the planned movement path based on the prompting information, the processing device 130 can reacquire the second position information of the scanning bed and / or surgical plan information. The reacquired surgical plan information may include at least one adjusted operation position and an adjusted operation channel corresponding to the adjusted operation position. The acquisition method may be the same as or similar to step 310, and will not be described again here.

[0119] Step 350: Based on the reacquired second position information and / or the reacquired surgical plan information, replan the movement path of the end effector.

[0120] In some embodiments, the processing device 130 may replan the movement path of the end effector based on the reacquired second location information and / or the reacquired surgical plan information. The method of planning the path may be the same as or similar to step 320, and will not be described in detail here.

[0121] In some embodiments, when the movement path of the end effector is successfully planned again, the processing device 130 may execute steps 360 and 3701 to perform surgery based on the successfully planned movement path; when the planning fails again, the processing device 130 may return to executing steps 330-350.

[0122] Step 360: In response to the successful planning of the end effector's movement path, determine the target surgical plan information and the second sorting result of multiple target operation positions in the target surgical plan information.

[0123] In some embodiments, after the end effector's movement path is successfully planned, the processing device 130 can determine the target surgical plan information and the sorting result of multiple target operation positions in the target surgical plan information, which is also called the second sorting result. After successful planning, the operation positions and corresponding operation channels in the surgical plan information have been verified and can meet the requirements of normal surgery. The processing device 130 can determine the successfully planned surgical plan information as the target surgical plan information, and the second sorting result of multiple target operation positions in the target surgical plan information is the sorting result of multiple operation positions in the surgical plan information (i.e., the first sorting result). The sorting result of the target operation positions determines the execution order of these target operation positions in the surgery. In some embodiments, the target surgical plan information may include surgical plan information to be executed. For example, surgical plan information to be executed in the actual surgery or surgical simulation stage.

[0124] Step 370: Sequentially plan and execute the execution sub-paths between each group of adjacent target operation positions in the second sorting result. At least a portion of each execution sub-path is executed within the scanning aperture of the scanning device.

[0125] In some embodiments, after step 360 determines the target surgical plan information, in a preset environment, the processing device 130 can, based on the first position information, second position information, and target surgical plan information obtained in step 310, sequentially plan and execute execution sub-paths between each group of adjacent target operation positions in the second sorting result. An execution sub-path refers to the movement path of the end effector between two adjacent target operation positions in the second sorting result. A complete end effector movement path can include multiple consecutive execution sub-paths. Specifically, the processing device 130 can control the robotic arm end effector to move along the execution sub-paths via commands, etc. In some embodiments, the preset environment can include at least one of the actual surgical stage, surgical simulation stage, etc. In some embodiments, steps 310-360 can be executed in the preoperative stage of surgery, and step 370 can be executed in the intraoperative stage of surgery.

[0126] In some embodiments, path planning and path execution can be performed by the same or different processors. For example, the path planning operation in steps 310-370 can be performed by a first processor (e.g., a processor at the master end), and the path execution operation in step 370 can be performed by a second processor (e.g., a processor at the slave end), which is different from the first processor. The first and second processors can be included in the processing device 130. Alternatively, the path planning and path execution operations in steps 310-370 can be performed by the same processor, which can be included in the processing device 130.

[0127] To monitor the accuracy of procedures during surgery, real-time imaging with scanning equipment can be used to observe operations related to the procedure location and access route. For example, during interventional puncture surgery, as the robotic arm moves the puncture needle to the puncture point, the surgeon can monitor the entire needle insertion process in real-time using a CT scanner. In some embodiments, at least a portion of each execution sub-path can be performed within the scanning aperture of the scanning equipment, enabling real-time imaging of the patient and monitoring of the surgical process, thereby ensuring the smooth progress of the procedure.

[0128] In some embodiments, the processing device 130 can tilt the scanning device gantry, even if the gantry is tilted in the cephalopod direction (i.e., the second direction), so that the target operating channel is within the scanning range of the scanning device, so as to enable complete and accurate real-time monitoring of the surgical procedure through the scanning device.

[0129] In some embodiments, if the target operation channel corresponding to the target operation position included in the execution sub-path to be executed is tilted relative to the second direction, the processing device 130 can control the scanning device, the scanning bed and the robotic arm to work together to execute the sub-path.

[0130] In some embodiments, the target operation position ranked first in the second sorting result is referred to as the first target operation position. The processing device 130 can tilt the scanning device gantry and perform surgery through the following steps: When the target operation channel corresponding to the first target operation position is tilted relative to the second direction, the processing device 130 can control the scanning device to rotate in the second direction, so that the scanning aperture of the scanning device is tilted relative to the second direction by a first angle, wherein the first angle corresponds to the tilt angle of the target operation channel corresponding to the first target operation position relative to the second direction, and the second direction is parallel to the long axis direction of the scanning bed; control the scanning bed to move along the second direction and a third direction, entering the scanning aperture of the scanning device, so that the target operation channel corresponding to the first target operation position is within the scanning range of the scanning device, and the third direction is the direction perpendicular to the scanning bed; plan and control the robotic arm and end effector to execute an execution sub-path from the initial safe position to the first target operation position. The safe position, also known as the external safe point, is the position where the robotic arm and end effector will not collide with the scanning device and the scanning bed during tilt adjustment. The initial safe position is the safe position of the robotic arm and end effector at the start of the surgery. In some embodiments, the first angle and the tilt angle of the target operating channel corresponding to the first target operating position relative to the second direction are equal or approximately equal. In some embodiments, when controlling the scanning bed to move into the scanning aperture, it may only include movement along the second direction.

[0131] In some embodiments, the processing device 130 can tilt the scanning device gantry and perform surgery through the following steps: Based on the tilt of the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions relative to the second direction, the processing device 130 can control the robotic arm and end effector to withdraw from the scanning aperture of the scanning device to a safe position from the front target operation position in the adjacent target operation positions, wherein the second direction can be parallel to the long axis direction of the scanning bed; control the scanning bed to withdraw from the scanning aperture of the scanning device, and then control the scanning device to rotate in the second direction, so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction, wherein the second angle corresponds to the tilt angle of the rear target operation channel relative to the second direction; control the scanning bed to move along the second direction and a third direction, entering the scanning aperture of the scanning device, so that the rear target operation channel is within the scanning range of the scanning device, wherein the third direction can be a direction perpendicular to the scanning bed; plan and control the robotic arm and end effector to execute an execution sub-path from the safe position to the rear target operation position. The front target operation position is the target operation position that is ranked first among the adjacent target operation positions, and the rear target operation position is the target operation position that is ranked last among the adjacent target operation positions. In some embodiments, the second angle is equal to or approximately equal to the tilt angle of the rear target operation channel relative to the second direction. In some embodiments, when controlling the scanning bed to move into the scanning aperture, movement along the second direction may be the only component. For example, if the front target operation position is the first target operation position, the rear target operation channel is tilted relative to the second direction, and the front and rear target operation channels are located in different scanning layers, then after tilting the scanning device frame and controlling the robotic arm and end effector to execute the execution sub-path from the initial safe position to the front target operation position, the processing device 130 can adjust the position of the frame and / or the scanning bed in the manner described above so that the rear target operation channel is within the scanning range of the scanning device, and then plan and control the robotic arm and end effector to execute the execution sub-path from the front target operation position to the rear target operation position. For example, if the front target operation position is not the first target operation position and the rear target operation channel is tilted relative to the second direction, the processing device 130 can adjust the position of the frame and / or the scanning bed in the above manner so that the rear target operation channel is within the scanning range of the scanning device, and then plan and control the robotic arm and end effector to execute the execution sub-path from the front target operation position to the rear target operation position.

[0132] In some embodiments, if the rear target operation channel corresponding to the rear target operation position in an adjacent target operation position is tilted relative to the second direction, and the front target operation channel and the rear target operation channel are located in different scanning layers, the processing device 130 can also tilt the scanning device gantry and perform the operation through the following steps: controlling the scanning device to rotate in the second direction, so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction; controlling the scanning bed to move along the second direction and the third direction, so that the rear target operation channel is within the scanning range of the scanning device; planning and controlling the robotic arm and end effector to execute an execution sub-path from the front target operation position to the rear target operation position. For example, if the front target operation position is the first target operation position, after tilting the scanning device gantry and controlling the robotic arm and end effector to execute the execution sub-path from the initial safe position to the front target operation position, if the front target operation channel and the rear target operation channel are located in different scanning layers, the processing device 130 can adjust the position of the gantry and / or the scanning bed in the above manner so that the rear target operation channel is within the scanning range of the scanning device, and then plan and control the robotic arm and end effector to execute the execution sub-path from the front target operation position to the rear target operation position.

[0133] In some embodiments, if the rear target operation channel corresponding to the rear target operation position in an adjacent target operation position is tilted relative to the second direction, and the front target operation channel and the rear target operation channel corresponding to the front target operation position in an adjacent target operation position are located on the same scanning layer, the processing device 130 can plan and control the robotic arm and end effector to execute an execution sub-path from the front target operation position to the rear target operation position. For example, if the front target operation position is the first target operation position, after tilting the scanning device frame and controlling the robotic arm and end effector to execute the execution sub-path from the initial safe position to the front target operation position, the tilt angle of the front target operation channel corresponding to the front target position relative to the second direction is the first angle. If the first angle and the second angle are equal or unequal, and the front target operation channel and the rear target operation channel are located on the same scanning layer, the processing device 130 can directly plan and control the robotic arm and end effector to execute the execution sub-path from the front target operation position to the rear target operation position without adjusting the position of the frame and the scanning bed.

[0134] In some embodiments, if the rear target operation channel corresponding to the rear target operation position in an adjacent target operation position is tilted relative to the second direction, and the front target operation channel and the rear target operation channel corresponding to the front target operation position in an adjacent target operation position are located on the same scanning layer, and the tilt angles of the front target operation channel and the rear target operation channel relative to the second direction are not equal, the processing device 130 can tilt the scanning device gantry and perform surgery by the following steps: controlling the scanning device to rotate in the second direction, so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction, so that the rear target operation channel is within the scanning range of the scanning device; planning and controlling the robotic arm and end effector to execute an execution sub-path from the front target operation position to the rear target operation position. In some embodiments, before or after controlling the scanning device to rotate in the second direction, the processing device 130 can also control the scanning bed to move along the second direction and a third direction. For example, if the forward target operation position is the first target operation position, after tilting the scanning device gantry and controlling the robotic arm and end effector to execute the execution sub-path from the initial safe position to the forward target operation position, the tilt angle of the forward target operation channel corresponding to the forward target position relative to the second direction is the first angle. If the first angle and the second angle are not equal, and the forward target operation channel and the rear target operation channel are located on the same scanning layer, the processing device 130 can adjust the position of the gantry and / or scanning bed in the above manner so that the rear target operation channel is within the scanning range of the scanning device, and then plan and control the robotic arm and end effector to execute the execution sub-path from the forward target operation position to the rear target operation position. In some embodiments, when controlling the movement of the scanning bed, it may only include movement along the second direction.

[0135] In some embodiments, if the scanning device gantry has been tilted at the previous target position in an adjacent target operation position, the processing device 130 can control the scanning device to perform a gantry zeroing operation, and then control the scanning device to rotate so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction. In some embodiments, the gantry zeroing operation may include controlling the scanning device to rotate in the second direction so that the scanning aperture of the scanning device is parallel to the second direction. For example, if the previous target operation position is a first target operation position, after tilting the scanning device gantry and controlling the robotic arm and end effector to execute an execution sub-path from the initial safe position to the previous target operation position, the processing device 130 can control the scanning device to rotate in the opposite direction of the second direction by a first angle so that the scanning aperture of the scanning device is parallel to the second direction, thereby returning the scanning device gantry to its initial position at the start of the surgery.

[0136] In some embodiments, after controlling the rotation of the scanning device and / or the movement of the scanning bed, the processing device 130 may update the collision model and the position information of the subsequent target operation position, and plan an execution sub-path based on the updated collision model and the position information of the subsequent target operation position. For details on how to plan the execution sub-path based on the updated collision model and the position information of the subsequent target operation position, please refer to the relevant description in step 320, which will not be repeated here.

[0137] For more information on how to tilt the scanning equipment gantry and perform surgery using the above steps, please refer to the relevant descriptions in steps 430-460, which will not be repeated here.

[0138] In some embodiments of this specification, the accuracy of path planning is improved and guaranteed by planning and simulating the movement path of the robot's end effector (e.g., puncture needle) before surgery; by scanning the tilt of the equipment gantry during surgery, the surgical process can be monitored in real time with complete accuracy, reducing the difficulty of surgical operation; the above methods save the user's surgical time, reduce the difficulty of using the system, and improve the user experience.

[0139] Figure 4 This is an exemplary flowchart of another surgical path planning method according to some embodiments of this specification.

[0140] like Figure 4 As shown, process 400 includes the following steps. In some embodiments, process 400 may be executed by processing device 130.

[0141] Step 410: Obtain the first position information of the scanning device, the second position information of the scanning bed, and the target surgical plan information to be executed. The target surgical plan information includes multiple target operation positions and multiple target operation channels corresponding to these target operation positions.

[0142] In some embodiments, step 410 can be similar to step 310, wherein the target surgical plan information to be executed can be equivalent to the surgical plan information in step 310. The difference is that the surgical plan corresponding to the target surgical plan information to be executed contains the movement path of the end effector that can be directly executed, while the surgical plan information in step 310 requires path planning and verification. For more details on the operation of step 410, please refer to the description of step 310, which will not be repeated here.

[0143] Step 420: Sequentially plan and execute the execution sub-path between each group of adjacent target operation positions. At least a portion of each execution sub-path is executed within the scanning aperture of the scanning device.

[0144] In some embodiments, the operation of step 420 may be similar to the relevant content in step 370, and will not be described again here.

[0145] In step 430, the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions is tilted relative to the second direction. The robotic arm and end effector are controlled to withdraw from the scanning hole of the scanning device to a safe position from the front target operation position in the adjacent target operation positions. The second direction is parallel to the long axis of the scanning bed.

[0146] When performing interventional punctures in the chest or abdomen, the needle tract is tilted to the head and feet, making the puncture needle trajectory (i.e., the complete needle tract) invisible under real-time imaging, which increases the difficulty of the procedure. For example... Figure 13 As shown, the CT gantry is not tilted, while Figure 13 The needle tract is a cephalopod tilted needle tract. When the CT machine is exposed in real time, the complete needle tract cannot be seen because the exposure direction is not parallel to the layer where the needle tract is located.

[0147] In some embodiments, the processing device 130 can tilt the scanning device via a tilting scanning device gantry (e.g., a CT scanner, MRI machine, etc.), allowing the puncture needle trajectory to be seen under real-time exposure, thereby reducing the difficulty of the surgery to some extent. Figure 14-16 As shown, in Figure 13 Based on the CT gantry shown, the CT gantry is tilted towards the cephalofoot side (i.e., the second direction), so that the CT exposure direction is parallel to the layer where the needle track is located, thus allowing the complete cephalofoot tilted needle track to be seen.

[0148] In some embodiments, the tilt scanning device can also be used for other surgeries, such as vascular surgery.

[0149] During the process of using an oblique scanning device, the robotic arm and end effector must first be removed from the scanning aperture of the scanning device. In some embodiments, based on the tilt of the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions relative to the second direction, the processing device 130 can control the robotic arm and end effector to withdraw from the scanning aperture of the scanning device to a safe position from the front target operation position in the adjacent target operation positions. The second direction is parallel to the long axis of the scanning bed. Taking interventional multi-needle puncture surgery as an example, the front target position can correspond to the current needle insertion point, and the rear target position can correspond to the next needle insertion point to be performed. The target operation channel corresponding to the rear target operation position in the adjacent target operation positions is tilted towards the second direction, indicating that the target operation channel is a cephalopod tilted needle path, that is, the next needle insertion point corresponds to a cephalopod tilted needle path. The processing device 130 can control the robotic arm and puncture needle to withdraw from the scanning aperture of the CT scanner from the current needle insertion point to a safe position.

[0150] In some embodiments, the processing device 130 can tilt the scanning device by executing step 440 and move the scanning bed by executing step 450, so that the target operation position is within the scanning range of the scanning device. Taking a CT scanner as an example, the processing device 130 can rotate the CT gantry to tilt it by executing step 440 and move the scanning bed by executing step 450, so that the target point and the needle insertion point are within the scanning range enclosed by the CT scanning boundary. Preferably, the target point and the needle insertion point can be located on the collimation center line of the CT scanner.

[0151] Step 440: Control the scanning bed to retract from the scanning aperture of the scanning device, and then control the scanning device to rotate in the second direction, so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction. The second angle corresponds to the tilt angle of the rear target operation channel relative to the second direction.

[0152] In some embodiments, after the robotic arm and end effector are retracted to a safe position, the processing device 130 can control the scanning bed to retract from the scanning aperture of the scanning device so that the scanning device has sufficient space to tilt.

[0153] In some embodiments, after the robotic arm, end effector, and scanning bed have all exited the scanning aperture of the scanning device, the processing device 130 can control the scanning device to tilt, that is, control the scanning device to rotate in a second direction (i.e., the cephalopod direction) so that the scanning aperture of the scanning device is tilted relative to the second direction.

[0154] In some embodiments, the rotation angle (e.g., a first angle, a second angle) of the scanning device in the second direction may be equal to or similar to the tilt angle of the target operating channel in the second direction. The processing device 130 can determine this rotation angle by determining a first vector of the target operating channel based on target surgical plan information, and then determining the rotation angle based on the first vector. The first vector is a vector representing the spatial direction of the target operating channel; for example, in a puncture procedure, the first vector may be a vector pointing from the needle insertion point to the target point. In some embodiments, the tilt of the scanning device may be equivalent to the tilt of the gantry; therefore, the gantry tilt angle of the scanning device can be used as the angle of rotation of the scanning device in the second direction.

[0155] The following is Figure 14 The following example, using a CT gantry and a CT bed as an example, explains how to calculate the tilt angle of a CT gantry. Figure 14 As shown, during the preoperative scanning phase, when locating the puncture point within the CT scanner's scanning port, the target area is positioned at the collimation center line by moving the CT scanner. The needle insertion point at this point is recorded as E(x). e y e , z e The target point is T(x) t y t , zt This refers to the needle insertion point and target point values ​​in the target surgical plan information. Both the needle insertion point and target point are represented in the CT bed coordinate system (also known as the patient coordinate system).

[0156] In some embodiments, the vector pointing from the needle insertion point E to the target point T It can be represented by the following formula:

[0157] In some embodiments, the X-axis direction vector of the CT bed coordinate system can be denoted as... Then vector and The vector resulting from the cross product can be represented by the following formula:

[0158] In some embodiments, the reverse direction vector of the Z-axis in the CT bed coordinate system is denoted as... Then vector Know The included angle θ between them is the tip-to-foot tilt angle of the needle track (equal to the tilt angle of the CT gantry), which can be expressed by the following formula:

[0159] Step 450: After controlling the scanning bed to move at least along the second direction, it enters the scanning hole of the scanning device, so that the rear target operation channel is within the scanning range of the scanning device.

[0160] In some embodiments, after the scanning device completes head-to-foot tilting, the processing device 130 can control the scanning bed to move along the second direction and the third direction, entering the scanning aperture of the scanning device, and ensuring that the rear target operation channel is within the scanning range of the scanning device. The third direction is perpendicular to the scanning bed, i.e., perpendicular to the plane containing the first and second directions. Therefore, the second direction is horizontal, and the third direction is vertical. Controlling the scanning bed to move along the second and third directions means controlling the scanning bed to move horizontally and vertically, thereby re-entering the scanning aperture of the scanning device. In some embodiments, when there is no possibility of interference between the scanning bed and the scanning device after tilting, the processing device 130 can also control the scanning bed to move only along the second direction, i.e., only move horizontally.

[0161] In some embodiments, after the scanning device is tilted, the tilt angle of the scanning device frame can correspond to a minimum scanning bed height. The processing device 130 can avoid interference between the scanning bed and the scanning device by moving the scanning bed up and down, and make the target scanning position fall into the scanning range enclosed by the scanning boundary of the scanning device by moving the scanning bed left and right.

[0162] In some embodiments, the amount of movement of the scanning bed can be determined based on: obtaining the minimum scanning bed height of the scanning device at a first angle, and determining the vertical movement of the scanning bed based on the minimum scanning bed height; and, after vertical movement, determining the relative position of the target point with respect to the isocenter point of the scanning device, and determining the horizontal movement direction and the horizontal movement amount of the scanning bed based on the relative position.

[0163] The following is Figure 15 The following example illustrates how to calculate the vertical movement of a CT scan bed, using a CT gantry and a CT scan bed as examples. Figure 15 In such Figure 14 The CT gantry shown is tilted before the CT bed is moved vertically. For a specific CT model, each tilt angle corresponds to a minimum scanning bed height, which can be obtained by consulting the CT's parameters. Figure 15 As shown, let the tilt angle of the frame be... (and Figure 15 (The inclination angle θ in the head-to-foot direction of the middle needle is equal), and the corresponding inclination angle is... Figure 15 The minimum scanning bed height for CT scans is h. min .

[0164] In some embodiments, let the bed height be h0 when the machine frame is not tilted, then the vertical bed movement amount Δ h It can be represented by the following formula: The bed height after the frame is tilted It needs to be set to

[0165] The following is Figure 16 This example illustrates how to calculate the horizontal movement of the CT bed. Figure 16 In such Figure 15 The CT bed shown is moved horizontally after the vertical movement is completed. Figure 15 As shown, after the CT bed is moved vertically, the CT bed coordinate system shifts upward by Δ. h Let the needle insertion point after the upward movement be E′(x). e y e , z e The target point is T′(x). t y t , z t ),but Figure 16 In the horizontal direction, after moving the bed, the needle insertion point is E″, and the target point is T″. The isocenter point O in the target surgical plan information has the coordinates O(x) in the CT bed coordinate system during preoperative scanning. o y o , z oIf the center point O is in the CT bed coordinate system after the bed is moved vertically, then the coordinates of the center point O are O(x). o y o +Δ h , z o The isocenter is the rotation center of the CT gantry, which does not change when the gantry is tilted or the scanning bed is moved.

[0166] In some embodiments, when the target point T′ is moved vertically, the Y coordinate y t The Y-coordinate (y) of the isocenter point O in the CT bed coordinate system after vertical bed movement is greater than the vertical center point O. o +Δ h When the horizontal direction of the moving bed is horDir, the moving bed moves horizontally towards the inside of the scanning aperture; conversely, it moves horizontally towards the outside of the scanning aperture. Let the horizontal moving bed direction be horDir, then the following formula applies:

[0167] In some embodiments, the horizontal bed movement amount Δ z It can be represented by the following formula: Δ z =horDir*OT′*tanθ=horDir*abs(y t -(y o +Δ h ))*tanθ (6) Where OT′ represents the distance between the isocenter point O and the target point T′; abs represents the absolute value operation; θ has the same meaning as in formula (3), representing the tilt angle of the needle head and foot direction, which is related to the tilt angle of the frame. equal.

[0168] like Figure 16 As shown, the Y-coordinate of the target point T′ after the vertical movement of the bed is y t The Y-coordinate (y) of the isocenter point O in the CT bed coordinate system after vertical bed movement is greater than the vertical center point O. o +Δ h Therefore, the bed is moved horizontally into the scanning hole. At this time, horDir = 1. horDir = 1 can be substituted into formula (6) to obtain the horizontal bed movement amount Δz.

[0169] Step 460: Plan and control the robotic arm and end effector to execute the execution sub-path from the safe position to the subsequent target operation position.

[0170] After the scanning bed moves vertically and horizontally, the position reached by the robotic arm also needs to be changed. In some embodiments, after step 450, the processing device 130 can plan and control the robotic arm and end effector to execute an execution sub-path from the safe position to the next target operating position. Specifically, the processing device 130 can reposition the robotic arm and plan its arrival position so that the robotic arm can continue to execute the execution sub-path from the safe position to the next target operating position (i.e., the next operating position).

[0171] In some embodiments, the relative positions of the scanning bed and the scanning device change after the gantry tilts, thus changing the positional information of the needle insertion point and the target point. In some embodiments, the positional information of the needle insertion point and the target point can be represented by coordinates in the robotic arm coordinate system. In some embodiments, the processing device 130 can determine the positional information of the needle insertion point and the target point after the gantry tilts by: determining a first transformation matrix based on the horizontal and vertical movement of the scanning bed; obtaining the transformation relationship of the coordinate systems at the needle insertion point and the target point in the robotic arm coordinate system; and determining the transformation relationship of the coordinate systems at the needle insertion point and the target point in the robotic arm coordinate system based on the transformation relationship and the first transformation matrix.

[0172] In some embodiments, the positive direction of the Y-axis of the CT bed is vertically downward, and the positive direction of the Z-axis is from the foot side to the head side of the CT bed. After the gantry is tilted, the CT bed coordinate system moves along the Y-axis relative to the preoperative scan by Δ. h That is, moving vertically upwards by Δ h Move -Δ along the Z-axis z That is, moving Δ from the head side of the CT bed to the foot side of the CT bed. z The transformation relationship of the CT bed coordinate system before and after the movement (transformation matrix) It can be represented by the following formula:

[0173] In some embodiments, the transformation relationship between the coordinate systems of the needle insertion point and the target point in the robotic arm coordinate system during preoperative scanning is as follows: After the frame tilts, the positions of the needle entry point and target point in the robotic arm coordinate system need to take into account the aforementioned transformation relationships. The transformation of the needle entry point and target point to the robotic arm coordinate system can be represented by the following formula: Among them, E′ robot This indicates the coordinates of the needle entry point in the robotic arm coordinate system after the frame tilts; T robot This indicates the coordinates of the target point in the robotic arm coordinate system after the frame is tilted.

[0174] In some embodiments, the processing device 130 may determine a collision model based on the first and second position information obtained in step 410. For details on how the collision model is determined, please refer to the relevant description in step 320, which will not be repeated here.

[0175] After the scanning device tilts, the scanning device model also needs to be updated; due to the vertical and horizontal movement of the bed, the patient model (including the surgical instrument model already present on the patient's body surface) also needs to be updated in real time. In some embodiments, after step 450, the processing device 130 may update the collision model and the position information of the subsequent target operation position.

[0176] In some embodiments, the processing device 130 may update the scanning device model by: obtaining the transformation relationship between the scanning bed coordinate system and the robotic arm coordinate system before the scanning device is tilted; determining a second transformation matrix of the scanning bed coordinate system based on the rotation angle of the scanning device in the second direction; and updating the transformation relationship between the scanning bed coordinate system and the robotic arm coordinate system based on the second transformation matrix to update the scanning device model.

[0177] In some embodiments, the transformation relationship between the CT bed coordinate system and the robotic arm coordinate system during preoperative scanning is assumed to be: Set the frame to tilt Subsequently, the transformation relationship of the CT bed coordinate system relative to the preoperative scan (transformation matrix) It can be represented by the following formula:

[0178] In some embodiments, the pose of the CT collision detection model (i.e., the scanning device model) in the robotic arm coordinate system after the CT gantry is tilted needs to take into account the above transformation relationship, and can be updated as follows:

[0179] In some embodiments, the processing device 130 may update the patient model by: obtaining the transformation relationship between the patient coordinate system and the robotic arm coordinate system before the scanning device is tilted; determining a third transformation matrix of the patient coordinate system based on the amount of bed movement of the scanning bed; and updating the transformation relationship between the patient coordinate system and the robotic arm coordinate system based on the third transformation matrix to update the patient model.

[0180] In some embodiments, the transformation relationship between the patient coordinate system and the robotic arm coordinate system during preoperative scanning is assumed to be: The patient moves vertically by Δ h Horizontal movement Δ z Subsequently, the transformation relationship (transformation matrix) of the patient collision detection model (i.e., the patient model) relative to the preoperative scan is determined. It can be represented by the following formula:

[0181] In some embodiments, the pose of the patient collision detection model (i.e., the scanning equipment model) in the robotic arm coordinate system after the CT gantry is tilted needs to take into account the above transformation relationship, and can be updated as follows:

[0182] In some embodiments, the processing device 130 can plan an execution sub-path based on the updated collision model and the location information of the subsequent target operation position. For details on how to plan the execution sub-path based on the collision model and the location information of the subsequent target operation position, please refer to the relevant description in step 320, which will not be repeated here.

[0183] It should be noted that the above descriptions of procedures 300 and 400 are for illustrative purposes only and do not limit the scope of this specification. Those skilled in the art can make various modifications and changes to procedures 300 and 400 under the guidance of this specification. However, these modifications and changes remain within the scope of this specification. For example, CT gantry tilting can be performed only when cephalopelvic tilt needle tracts are detected during surgery, or the end effector movement path can be planned from the beginning according to the CT gantry tilt. This allows for adaptive planning for cephalopelvic tilt needle tracts and other conditions from the outset, avoiding adjustments during surgery and reducing surgical time.

[0184] The beneficial effects that the embodiments of this specification may bring include, but are not limited to: (1) improving and ensuring the accuracy of path planning by planning and simulating the movement path of the robot end effector (e.g., puncture needle, etc.) before surgery; (2) simulating and verifying the planned movement path through a collision model, and prompting the user with the reason for failure and solution suggestions when planning fails, so that the user can accurately and quickly judge the reason for failure and make corresponding adjustments to the planning in a timely manner, ensuring the effectiveness of the planned path; (3) ensuring that the surgical process can be monitored in real time with complete accuracy by scanning the tilt of the equipment gantry during planning / operation, reducing the difficulty of surgical operation; (4) saving the user's surgical time, reducing the difficulty of using the system, and improving the user experience by using a process that includes the above methods. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced can be any one or a combination of the above, or any other possible beneficial effects.

[0185] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0186] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0187] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0188] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0189] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0190] For each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, and documents, referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0191] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. An interventional medical system, data-connected to a medical imaging system, the medical imaging system including a scanning device and a scanning bed, the interventional medical system comprising: An interventional surgical robot system and a processor, the interventional surgical robot system including a robotic arm and an end effector, the processor being configured to: Acquire first position information of the scanning device, second position information of the scanning bed, and target surgical plan information to be executed. The target surgical plan information includes multiple target operation positions and multiple target operation channels corresponding to the multiple target operation positions. The execution sub-paths between each group of adjacent target operation positions are planned and executed sequentially, wherein at least a portion of each execution sub-path is executed within the scanning aperture of the scanning device, and the sequential planning and execution of the execution sub-paths between each group of adjacent target operation positions includes: Based on the inclination of the target operation channel corresponding to the target operation position included in the execution sub-path to be executed relative to the second direction, the scanning device, the scanning bed and the robotic arm are controlled to work together to execute the sub-path, wherein the second direction is parallel to the long axis direction of the scanning bed.

2. The system of claim 1, wherein controlling the scanning device, the scanning bed, and the robotic arm to coordinate and execute the sub-path includes: A collision model is determined based on the first location information and the second location information. The collision model includes a patient model, a scanning device model, and a robotic arm end effector model.

3. The system as described in claim 2, wherein, The control of the scanning device, the scanning bed, and the robotic arm to coordinate and execute the sub-path includes: Based on the inclination of the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions relative to the second direction, the robotic arm and the end effector are controlled to withdraw from the front target operation position in the adjacent target operation positions from the scanning hole of the scanning device to a safe position. Control the scanning bed to exit from the scanning hole of the scanning device, and then control the scanning device to rotate in the second direction, so that the scanning hole of the scanning device is tilted at a second angle relative to the second direction, the second angle corresponding to the tilt angle of the rear target operation channel relative to the second direction; The scanning bed is controlled to move at least along the second direction and enter the scanning aperture of the scanning device, so that the rear target operation channel is within the scanning range of the scanning device; Plan and control the robotic arm and the end effector to execute the execution sub-path from the safe position to the subsequent target operation position.

4. The system of claim 2, wherein controlling the scanning device, the scanning bed, and the robotic arm to coordinate and execute the sub-path includes: Based on the tilt of the target operation channel corresponding to the first target operation position in the target surgical plan information relative to the second direction, the scanning device is controlled to rotate in the second direction, so that the scanning aperture of the scanning device is tilted relative to the second direction by a first angle, the first angle corresponding to the tilt angle of the target operation channel corresponding to the first target operation position relative to the second direction; After controlling the scanning bed to move at least along the second direction, it enters the scanning hole of the scanning device, so that the target operation channel corresponding to the first target operation position is within the scanning range of the scanning device; Plan and control the robotic arm and the end effector to execute the execution sub-path from the initial safe position to the first target operating position.

5. The system of claim 2, wherein controlling the scanning device, the scanning bed, and the robotic arm to coordinate and execute the sub-path includes: Based on the tilt of the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions relative to the second direction, and the front target operation channel corresponding to the front target operation position in the adjacent target operation positions and the rear target operation channel being located in the same scanning layer, and the tilt angles of the front target operation channel and the rear target operation channel relative to the second direction being different. Control the scanning device to rotate in the second direction, so that the scanning aperture of the scanning device is tilted at a second angle relative to the second direction, so that the rear target operation channel is within the scanning range of the scanning device, and the second angle corresponds to the tilt angle of the rear target operation channel relative to the second direction; Plan and control the robotic arm and the end effector to execute the execution sub-path from the previous target operation position to the next target operation position.

6. The system of any one of claims 3 and 5, wherein controlling the scanning device, the scanning bed, and the robotic arm to coordinate and execute the sub-path further comprises: Before controlling the scanning device to rotate in the second direction, causing the scanning aperture of the scanning device to tilt at a second angle relative to the second direction, the scanning device is controlled to perform a rack zeroing operation.

7. The system of claim 2, wherein controlling the scanning device, the scanning bed, and the robotic arm to coordinate and execute the sub-path includes: Based on the tilt of the rear target operation channel corresponding to the rear target operation position in the adjacent target operation positions relative to the second direction, and the front target operation channel corresponding to the front target operation position in the adjacent target operation positions and the rear target operation channel being located in the same scanning layer, Plan and control the robotic arm and the end effector to execute the execution sub-path from the previous target operation position to the next target operation position.

8. The system according to any one of claims 3-5, wherein controlling the scanning device, the scanning bed, and the robotic arm to coordinate and execute the execution sub-path includes: After controlling the scanning device to rotate and / or controlling the scanning bed to move, update the collision model and the position information of the subsequent target operation position in the adjacent target operation positions; The execution sub-path is planned based on the updated collision model and the location information of the subsequent target operation position.

9. The system of claim 2, wherein the processor is further configured to: The movement path of the end effector is planned based on the first location information, the second location information, and the surgical plan information; In response to the successful planning of the movement path of the end effector, the target surgical plan information is determined.

10. The system of claim 9, wherein planning the movement path of the end effector based on the first location information, the second location information, and the surgical plan information comprises: Based on the collision model and the sorting results of the operation positions in the surgical plan information, the movement path of the end effector is planned.

Images