Method and system for robot-guided needle placement

Abstract

Methods, systems, media, and implementations for robotically guided instrument insertion. A pre-planned path between a three-dimensional (3D) entry pose on the skin of the patient and a 3D pose of the target is generated for surgery of the target within the patient. The robot uses the pre-planned path as guidance to insert the surgical instrument from the 3D entry pose to reach the 3D pose of the target. During the procedure, a next pose is determined based on a current pose of the surgical instrument and a pre-planned path and a spatial relationship with the surrounding anatomical structure, and used to move the surgical instrument thereon. When the robot advances the surgical instrument to the next pose, an updated current pose of the instrument is obtained via tracking. The process is repeated until the instrument reaches the target pose.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Patent Application No. 18 / 465,288, filed September 12, 2023, entitled “Method and System for Robot-Guided Needle Placement,” the entire contents of which are incorporated herein by reference. Background Technology 1. Technical Field

[0004] This teaching relates to computers. More specifically, this teaching relates to signal processing. 2. Background Technology

[0006] With advancements in various technologies, an increasing number of tasks are now performed with the help of computers. Different industries have benefited from this technological progress, including the medical field. Various medical data acquisition technologies, such as computed tomography (CT) or magnetic resonance imaging (MRI), can easily acquire vast amounts of image data capturing a patient's anatomical information. Running sophisticated data processing algorithms on fast computers with enormous storage capacity allows these massive amounts of medical data to be processed to identify anatomical structures of interest (e.g., organs, bones, blood vessels, or abnormal nodules), obtain measurements of each object of interest (e.g., the size of nodules growing in organs), quantify anatomical structures (e.g., the size and shape of abnormal nodules), and construct three-dimensional (3D) models of organs and related anatomical structures.

[0007] This type of information can be used for various medical purposes, including assisting in diagnosis, preoperative planning, and providing certain guidance during surgery. For example, preoperative planning can be based on a constructed 3D model of the target organ and surrounding anatomy to, for instance, plan the path of surgical instruments (such as biopsy needles) from the patient's skin to the target organ. This is as follows... Figure 1 As shown, there may be a target object 100 and other surrounding anatomical structures (such as 120-1 and 120-2) beneath the patient's skin 110. To reach the target 100 (e.g., a malignant tumor) to perform resection, biopsy, or ablation, the surgical instrument 140 may need to be inserted into the patient's skin 110 and travel toward the object 100 without contacting nearby critical anatomical structures 120-1 and 120-2. During preoperative planning, a path 130 from the insertion point on the skin 110 to the target object 100 can be planned such that the tip of the surgical instrument follows to reach the target object.

[0008] While such surgical paths can be accurately generated prior to surgery based on 3D models of different anatomical parts, during surgery, the features or spatial relationships of the anatomical parts often change due to the deformable nature of the anatomical parts, e.g., anatomical structures 120-1 and 120-2 can become closer or even overlap with the pre-planned path. Such deformations make it necessary to dynamically adjust the path in real-time based on the current situation to maneuver the surgical instrument to access the target organ without colliding with other anatomical parts. In some cases, the surgeon can not even see the target organ, which also makes it extremely challenging. For example, in laparoscopic surgery, the surgeon only sees what the laparoscope camera captures in a limited field of view. Without a wider peripheral view or what is under the surface of the visible anatomical structure, it can also be difficult to figure out what the next step is.

[0009] Therefore, there is a need for solutions that can address the above challenges. SUMMARY

[0010] The teachings disclosed herein relate to methods, systems, and programming for information management. More specifically, the teachings relate to methods, systems, and programming related to hash tables and storage management using hash tables.

[0011] In one example, a method implemented on a machine having at least one processor, storage, and a communication platform capable of connecting to a network for robotically guided instrument insertion. A pre-planned path between a three-dimensional (3D) entry pose on a patient's skin and a 3D pose of a target is generated for a surgery of the target in the patient's body. A robot uses the pre-planned path and spatial relationships with surrounding anatomical structures as guidance to advance a surgical instrument from the 3D entry pose to the 3D pose of the target. During the surgery, a next pose is determined based on a current pose of the surgical instrument and the pre-planned path, and the next pose is used to move the surgical instrument thereon. As the robot inserts the surgical instrument to the next pose, an updated current pose of the instrument is subsequently obtained via tracking. The process repeats until the instrument reaches the target pose.

[0012] In different examples, a system for robotically guided instrument insertion is disclosed that includes a next pose determiner, a robotically guided instrument insertion controller, and a current instrument pose determiner. The next pose determiner is provided to determine a next pose based on a pre-planned path that is generated for a surgery on a patient relative to a target within the patient. The pre-planned path is between a three-dimensional (3D) entry pose on the patient's skin and a 3D pose of the target and is used by a robot with detected spatial relationships to surrounding anatomy to advance a surgical instrument from the 3D entry pose to the 3D pose of the target. Each next pose is determined based on a current pose of the surgical instrument and the pre-planned path. The robotically guided instrument insertion controller is to control the robot to move the surgical instrument to the next pose. The current instrument pose determiner is provided to obtain an updated current pose of the surgical instrument via tracking. The steps of determining, controlling, and obtaining can be repeated until the surgical instrument reaches the 3D pose of the target.

[0013] Other concepts relate to software for implementing the present teachings. A software product according to this concept includes at least one machine-readable non-transitory medium and information carried by the medium. The information carried by the medium can be executable program code data, parameters associated with executable program code, and / or information related to a user, a request, content, or other additional information.

[0014] Another example is a machine-readable, non-transitory, and tangible medium having recorded thereon information for robotically guided instrument insertion. A pre-planned path is generated between a three-dimensional (3D) entry pose on a patient's skin and a 3D pose of a target for a surgery of the target within the patient. A robot uses the pre-planned path and spatial relationships to surrounding anatomy as guidance to advance a surgical instrument from the 3D entry pose to the 3D pose of the target. During the surgery, a next pose is determined based on a current pose of the surgical instrument and the pre-planned path, and the next pose is used to move the surgical instrument thereon. When the robot inserts the surgical instrument into the next pose, an updated current pose of the instrument is subsequently obtained via tracking. The process is repeated until the instrument reaches the target pose.

[0015] Some of the additional advantages and novel features of the present teachings will be set forth in the description that follows, and in part will be apparent to those skilled in the art upon examination of the following and the attached drawings, or can be learned by production or operation of the methods, instruments and compositions described in the detailed examples that follow. The advantages of the present teachings can be realized and obtained by means of the instrumentalities and combinations particularly pointed out in the appended claims. BRIEF DESCRIPTION OF DRAWINGS

[0016] The methods, systems, and / or programming described herein will be further described by way of example, with reference to the following illustrative embodiments. These embodiments are non-limiting examples, in which like numbers represent similar structures, and in which: Figure 1 A pre-planned surgical path is shown in which a surgical instrument reaches a target but avoids collision with other anatomical structures; Figure 2 A surgical setting is depicted in which a robot is deployed to manipulate a surgical instrument along a dynamically adjusted path to reach a target within a patient’s body, in accordance with embodiments of the present teachings; Figure 3A An exemplary high-level system diagram of a framework for robot-guided needle placement, in accordance with embodiments of the present teachings, is depicted; Figure 3B Exemplary types of operational parameters related to surgery are shown; Figure 3C A flowchart of an exemplary process for pre-operative planning of a surgical needle insertion path, in accordance with embodiments of the present teachings, is shown; Figure 3D A flowchart of an exemplary process for continuous robot-guided needle placement during surgery to reach a target based on a pre-planned surgical path, in accordance with embodiments of the present teachings, is shown; Figure 4 An exemplary high-level system diagram of a next target needle pose determiner, in accordance with embodiments of the present teachings, is depicted; Figure 5A A flowchart of an exemplary process for a next target needle pose determiner, in accordance with embodiments of the present teachings, is shown; Figure 5B A flowchart of an exemplary process for computing an adjusted needle pose under different circumstances, in accordance with embodiments of the present teachings, is shown; Figure 6A An exemplary scenario is shown in which a needle pose is dynamically adjusted to avoid an anticipated collision with another anatomical structure along a pre-planned path, in accordance with embodiments of the present teachings; Figure 6B An exemplary scenario is shown in which a needle pose is adjusted when there is no obstacle, in accordance with embodiments of the present teachings; Figure 6C An exemplary scenario is shown in which an adjustment is applied to a needle pose when the needle pose is close to an obstacle, in accordance with embodiments of the present teachings; Figure 7 A schematic diagram of an exemplary mobile device architecture that can be used to implement a special-purpose system to implement the present teachings in accordance with various embodiments is shown; and Figure 8is a schematic diagram of an exemplary computing device architecture that can be used to implement a specialized system for implementing the present teachings according to various embodiments. DETAILED DESCRIPTION

[0017] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the relevant teachings. However, it will be apparent to one skilled in the art that the present teachings can be practiced without these details. In other instances, well-known methods, procedures, components, and / or system have not been described in detail so as not to unnecessarily obscure aspects of the present teachings.

[0018] The present teachings disclose exemplary methods, systems, and implementations for a framework for robotically-guided surgical instrument insertion or placement. This is in the context of Figure 2 In this surgical setting, to perform a medical procedure on a patient 210 lying on a surgical table 200, a robot 240 can be deployed to manipulate the movement of a surgical instrument 240 having a tip 250, such as a needle, to reach a target pose inside the patient's body. The 3D pose of the tip of the surgical instrument 240 can be obtained via a tracking mechanism that includes a tracking camera 230 provided to monitor the 3D pose of a tracking element 220 within its field of view based on a calibration that maps what the camera 230 can observe to the 3D pose of the tracking element 220. In some embodiments, the 3D pose of the needle can be tracked by an electromagnetic tracking system. By a known spatial relationship between the tracking element 220 and the tip of the surgical instrument, the 3D pose of the tip of the surgical instrument can thus be determined. The robot 240 can operate to insert the surgical instrument 240 into the patient's body along a path that is determined based on a pre-planned insertion path (generated prior to the surgery) that is based on the observed pose of the surgical instrument and the known locations of the patient's anatomy.

[0019] The framework for robotically-guided surgical instrument insertion according to the present teachings includes two parts, a pre-operative planning part and an intra-operative part. The pre-operative planning part is used to plan a surgical instrument insertion path for reaching a target (e.g., a lesion) inside a patient based on a 3D model of the relevant anatomy of the patient. Such pre-planned insertion path generated prior to the surgery is then used by a robot in the intra-operative part of the framework to automatically insert a surgical instrument to reach the target while dynamically adjusting the pose of the surgical instrument relative to the pre-planned surgical path in different situations. The robot can adjust the surgical instrument pose to deviate from the pre-planned insertion path to avoid colliding with or to maintain a reasonable distance from the anatomy near the target, thereby ensuring that the surgical instrument can safely reach the target without a medical incident.

[0020] According to the present teachings, during the insertion process, situations associated with the current surgical instrument pose can be detected and the detected situations are used to determine the next target pose that the surgical instrument is to reach. In a first situation, the surgical instrument will collide with an anatomical structure if it follows the pre-planned insertion path. This can be due to deformation of the patient’s anatomy such that even though the pre-planned insertion path avoids such anatomical structures, during the surgery, this can change and the insertion path needs to be adjusted. In the first situation, a new next pose that the surgical instrument is to reach next can be calculated that deviates from the pre-planned insertion path to avoid the collision.

[0021] In a second situation, although the desired direction along the pre-planned insertion path does not collide with any anatomical structure, the distance between the pre-planned insertion path and certain anatomical structures (e.g. organs) can be too close. In this case, the robot can adjust the pose of the surgical instrument to deviate from the pre-planned insertion path to increase the distance between the surgical instrument and the nearby anatomical structures to ensure safety. In a third situation, when the surgical instrument approaches the target (e.g. a lesion) without colliding or being too close to other anatomical structures, the robot can still adjust the pre-planned insertion path to ensure that the surgical instrument can approach the target smoothly.

[0022] The present teachings can perform a step-wise adjustment process in which the current instrument pose is evaluated with respect to the pre-planned insertion path to determine the next target pose of the surgical instrument to gradually approach the target (e.g. a lesion). With this approach, the pre-planned insertion path can be used as a guide or baseline but can deviate based on necessary adjustments that can be made based on the actual situations observed during the surgery. This can effectively address safety issues that can arise due to deformation of the anatomical structures during the surgery.

[0023] Figure 3A An exemplary high-level system diagram of a framework 300 for robot-guided surgical instrument placement according to embodiments of the present teachings is depicted. As described herein, there are two parts, one corresponding to a pre-operative part for generating a pre-planned surgical instrument insertion path, and another corresponding to an intra-operative part in which the pre-planned surgical instrument insertion path is utilized by a robot to automatically insert a surgical instrument into a patient’s body to reach a target pose (e.g. a lesion to be removed). In the illustrated embodiment. The pre-operative part includes a patient surgery information determiner 310 and a surgical path pre-planning unit 340. The patient surgery information determiner 310 can be provided for obtaining information related to a patient (330) and an intended surgery (320). Such information can be related to the path to be pre-planned for insertion of a surgical instrument.

[0024] Figure 3BExemplary types of procedure-related operational parameters are shown, which can include, for example, the type of procedure to be performed on the patient (e.g., resection of a lesion or repair of a damaged organ), the target organ (e.g., a lesion in the liver), and information about the surgical tools used to perform the procedure (e.g., a cutter to cut the lesion and a hook to move blood vessels). Information related to the target organ can include both the specific target (e.g., lesion) to be operated on and the nearby anatomy (e.g., blood vessels and bones). Information about the surgical instruments to be inserted into the patient’s body in relation to the generation of a pre-planned insertion path can also be collected, including the type(s) of surgical instruments and the dimensions of each surgical instrument. Based on such related information, a surgical path pre-planning unit 340 can be provided to generate a pre-planned surgical instrument insertion path (360) for the patient’s medical procedure from the anatomical model (350) constructed for the patient.

[0025] The intraoperative portion of the framework 300 includes a current instrument pose determiner 370, a next target pose determiner 380, and a robot-guided instrument insertion controller 390. During the procedure, the next target pose determiner 380 is provided for accessing the pre-planned surgical instrument insertion path 360 (generated prior to the procedure) for the patient and using it to guide the determination of the next target pose that the surgical instrument is to reach. As discussed herein, the pre-planned insertion path can be used as a baseline, which can be adjusted according to real-time conditions observed during the procedure. In some cases, the next target pose can correspond to a point along the pre-planned insertion path. In some cases, the next target pose can correspond to an adjusted pose that deviates from the pre-planned insertion path for the safety of the patient during the procedure. Referring to Figure 4 - Figure 6C Detailed information is provided about the adjustment of the pose on the pre-planned insertion path.

[0026] The robot-guided instrument insertion controller 390 then uses the next target pose to calculate the configuration parameters of the robot needed to move the surgical instrument to reach the next target pose. The current instrument pose determiner 370 is provided to determine the current pose of the surgical instrument, which can be based on a tracking mechanism deployed during the procedure. As Figure 2 As shown in the middle, the pose of the tip of the surgical instrument can be tracked via a tracking mechanism that includes a tracking camera 230 and a tracking element 220. The tracked information can then be used by the current instrument pose determiner 370 to obtain the current pose of the tip of the surgical instrument. Based on the current pose of the surgical instrument, the next cycle starts again, i.e., the next target pose determiner 380 can determine the next target pose to which the surgical instrument is to be moved according to the present teachings.

[0027] Figure 3Cis a flowchart of an exemplary process of a first portion of a framework 300 for preoperative planning of surgical instrument insertion paths according to embodiments of the present teachings. At 305, a patient surgery information determiner 310 retrieves relevant surgery related operating parameters from a surgery related operating parameters storage 320 and at 315 accesses relevant patient specific information from a patient record 330, such as for example, the patient's age, diagnosis, surgical preparedness condition, etc. Such relevant information is then provided to a surgical path pre-planning unit 340 which then retrieves at 325 a three-dimensional model of the target organ and relevant anatomy in the vicinity of the target organ or between the target organ and the patient's skin accordingly. The retrieved 3D model can then be used by the surgical path pre-planning unit 340 to create at 335 a surgical instrument insertion path for the operation. The thus created surgical insertion path can then be archived at 345 in a pre-planned surgical path storage 360 which can be retrieved during the surgery.

[0028] Figure 3D is a flowchart of an exemplary process of a second portion of a framework 300 for intraoperative robot-guided continuous surgical instrument placement to reach a target based on a pre-planned surgical instrument insertion path according to embodiments of the present teachings. When the patient's surgery commences, a next target pose determiner 380 retrieves at 355 the pre-planned surgical instrument insertion path from 360 and then determines at 365 the next target instrument pose to which the surgical instrument is to be next brought. At 375, a robot-guided instrument insertion controller 390 then computes based on the next target pose the robot arm configuration parameters required to facilitate the robot 240 manipulating the surgical instrument to move to the next target instrument pose. The movement can then be tracked and at 385 a new current instrument pose is obtained by a current instrument pose determiner 370. If at 387 it is determined that the current instrument pose reached the target, the insertion of the surgical instrument is completed and thus at 395 the operation can be performed. If the current instrument pose has not yet reached the target, the process moves to step 365 to start the next cycle. The iterative process continues until the instrument reaches the target.

[0029] Figure 4 depicts an exemplary high-level system diagram of a next target pose determiner 380 according to embodiments of the present teachings. As discussed herein, the next target pose determiner 380 is invoked when the current instrument pose has not yet reached the specified target, thereby determining the next target pose to which the instrument is to be brought close. As discussed herein, the next target pose determiner 380 can be implemented in a variety of ways, such as for example, as a software module, a hardware module, a combination of software and hardware modules, etc. Figure 4As shown, the next target pose determiner 380 takes the current instrument pose, the pre-planned surgical instrument insertion path, and the ASM model 350 as inputs, and thus determines the next target pose. In the illustrated embodiment, the next target pose determiner 380 includes an insertion state evaluation unit 400, a target distance determiner 410, a collision evaluation unit 420, and a target pose determiner 430. The insertion state evaluation unit 400 is provided to evaluate the relationship between the current instrument pose and the pre-planned instrument insertion path, such as whether the current instrument pose is on or off the pre-planned insertion path.

[0030] A target distance determiner 410 can be provided to determine the current instrument pose relative to, for example, a target organ or other anatomical structure. For example, given a 3D pose of a surgical instrument, if the instrument continues in its current insertion direction, it can be assessed whether the instrument will collide with some anatomical structure (e.g., an organ). The distance between the current instrument pose and nearby anatomical structures can also be assessed. Such assessments can be performed against a 3D model of the anatomical structure; for example, the distance to nearby anatomical structures based on the current instrument pose can be represented by a 3D anatomical structure model. Furthermore, a collision assessment unit 420 can be provided to assess whether a collision with any anatomical structure is possible. Based on the assessment of what specific situation the current instrument pose is associated with, the target pose determiner 430 can then calculate the next target pose the surgical instrument should reach in the current iteration.

[0031] As discussed in this paper, although a pre-planned insertion path is provided, during surgery, the actual insertion path may deviate from the pre-planned path due to various reasons, such as displacement of different anatomical structures and / or deformation of certain anatomical structures during surgery. In each iteration, the conditions associated with the current surgical instrument pose can be determined, allowing the next target instrument pose to be determined accordingly. In some cases, the next target instrument pose (which corresponds to the next insertion direction from the current instrument pose) can be determined based on the pre-planned insertion path. In other cases, the next insertion pose specified in the pre-planned insertion path may not be adopted, and may deviate from the pre-planned path to avoid problems or improve safety.

[0032] Figure 6A - Figure 6C The illustrations show different calculations leading to the next target instrument pose, with the surgical needle as the instrument and the lesion as the target. In these illustrations, v d The desired orientation vector represents the next insertion direction from the current needle tip, based on the pre-planned insertion path. Figure 6A This illustrates a scenario where the desired insertion direction v extends from the current needle tip according to a pre-planned insertion path. dIf employed, would result in a collision with an obstructing organ. Thus, in this case, the desired direction from the pre-planned insertion path can not be employed as it would cause a collision, but rather an alternative next target instrument pose can be determined to deviate from the pre-planned insertion path to avoid the collision. Such collision situations can arise when some anatomy of the patient can shift during the procedure (e.g., due to the way the patient is lying on the surgical bed) or some anatomy of the patient can deform during the procedure.

[0033] Figure 6B Figure 6C shows an additional scenario where the desired insertion orientation v d is along the pre-planned insertion path extending from the needle tip Figure 6B involves a situation where the needle tip is sufficiently far away from any other anatomy, while Figure 6C shows a situation where the desired insertion direction v d is not obstructed by an organ, but the needle tip is sufficiently close to another organ. In Figure 6B the illustrated situation, the desired insertion direction v d according to the pre-planned insertion path can be employed to control the next needle movement according to the pre-planned insertion path as this does not cause any safety issues. However, in Figure 6C the illustrated situation, the distance between the needle tip and the other organ is very close, so to improve safety, the desired insertion direction v d may not be employed and the next target instrument pose can be computed to deviate from the pre-planned insertion path to minimize the risk.

[0034] In the following, detailed computations of the next target instrument pose determined based on different situations are discussed. In operation, when a surgical instrument is inserted into a patient through, e.g., a tool guide deployed on a robot, readings from a sensor on the tip of the surgical instrument (e.g., a needle) can be obtained via a tracking mechanism as discussed herein. Let the current instrument pose (3D position and 3D orientation) be represented as where is the current position of the instrument tip, is the current orientation unit vector of the instrument, where a, e, and r can represent pitch, roll, and yaw, respectively. Based on the current instrument position, further assume that the pre-planned position of the instrument tip is and the pre-planned orientation vector of the instrument is As discussed herein, the desired (pre-planned) instrument position and orientation may be a continuous function or a sequence of discrete samples. For example, to reach the desired instrument position ​The trajectory can be a list of discrete points starting from an entry point on the skin and extending to a target (e.g., the center of the lesion). In some embodiments, a series of desired, pre-planned instrument orientations. The trajectory can be represented as a set of vectors from the entry point to the target. In some embodiments, the pose trajectory of the pre-planned insertion path can be a smoothed version of the initially desired pose trajectory. Such smoothing can be introduced to reduce, for example, acute angles formed by adjacent poses, for purposes such as achieving more accurate tracking during surgery.

[0035] When determining the pose of the next target instrument, in each iteration after the instrument has been inserted into the skin, the detected conditions can be used to check whether the pose of the next target instrument needs to deviate from the expected trajectory. If the current instrument position deviates from the pre-planned insertion path, the position of the next target instrument can be calculated as follows:

[0036] Furthermore, the orientation vector of the next target instrument can be calculated as follows:

[0037] in It is the tip of the instrument. Distance between the insertion target (e.g., the center of the lesion) and the insertion target. The function. See Figure 6B .if If it is smaller, then the function can be Provided for values ​​with larger values. v a This corresponds to the current orientation of the surgical instrument.

[0038] In each iteration of the instrument pose, when the vector of the surgical instrument at step n-1... With any anatomical structure (such as organs, etc.) Figure 6A (As shown) during a collision, or if the vector Minimum distance between the nearest non-target anatomical structure Below a preset threshold (e.g.) Figure 6C (As shown), the orientation of the next target instrument can be calculated as follows:

[0039] Among them, Figure 6A In the case shown, In the expected needle vector The tangential vector along the surface of the anatomical structure at the intersection point where it collides with the structure. The vector at the instrument tip. When not passing through an organ ( Figure 6C (The situation in the middle) It is the closest point along the surface of the anatomical structure to the tip of the surgical instrument. The tangent vector in the direction of . In some embodiments, It can correspond to the position of the tip of the surgical instrument. Minimum distance between and anatomical structures The function. In some embodiments, it blocks the desired needle vector. The size of the anatomical structures and how much the surgical instruments need to bend to reach the next target instrument position can also be taken into account and incorporated into the equation to determine whether the needle path can avoid organ collision.

[0040] As shown above, at each current pose during the insertion process, if the surgical instrument has not yet reached the target, the next target instrument pose can be determined as the target position that the tip of the surgical instrument should reach in the next iteration. The next target instrument pose is calculated based on specific detected conditions to avoid potential collisions or enhance the operability of the insertion. This teaching allows for deviations from the pre-planned insertion path based on dynamically detected conditions when necessary, adjusting the next target instrument pose to a position safe for the patient. Once the next target instrument pose is determined, the robot joint positions can be calculated to move the surgical instrument toward the next target instrument pose. It must be understood that the pose of the surgical instrument relative to the robot base is known through certain calibrations. Iteration can continue until the tip of the surgical instrument is sufficiently close to the target.

[0041] Figure 5A This is a flowchart of an exemplary process of a next target pose determiner 380 according to an embodiment of this teaching. To calculate the next target instrument pose, the pose of the surgical instrument to be inserted is first checked at 500 to see if it is currently outside the patient's body. If it is outside the patient's body, the next target instrument pose is set at 510 based on the desired pose of the target, and then the set next target instrument pose is output at 590. When the surgical instrument is already inside the patient, to calculate the next target instrument pose, at 520, the next target pose determiner 380 receives various information to be used in determining the next target instrument pose, including the current pose of the surgical instrument, the pre-planned insertion path, and ASM 350. Based on the received information, at 530, the current pose of the surgical instrument is compared with the pose of the target (e.g., the lesion center) specified in the pre-planned insertion path.

[0042] If at 540 it is determined that the surgical instrument has reached the target, the insertion process is completed and a signal is output at 550 to indicate that the surgical instrument has reached the target. If the surgical instrument has not reached the target, the distance and spatial relationship between the current pose of the surgical instrument and the target and other nearby relevant anatomical structures are determined at 560. The situation in which the current instrument pose is at is analyzed at 570. Based on the thus determined distances, spatial relationships, and detected situation, the next target instrument pose is then calculated at 580 according to the present teachings (e.g., the formulas shown herein) for the detected situation. The determined next target instrument pose is then output at 590.

[0043] Figure 5B is a flowchart of an exemplary process of exemplary steps to determine the next target instrument pose in different situations according to embodiments of the present teachings. At 505, it is first assessed whether a collision will occur based on the current instrument pose, the pre-planned insertion path, and the 3D model of the anatomical structures in the vicinity of the current instrument pose. If an obstacle is detected at 515 (which is the case in Figure 6A shown above), the expected insertion pose from the pre-planned insertion path is not employed, and instead a target instrument pose is calculated at 525 according to the formulas described above to avoid the detected collision. If no expected collision is detected (which is the case in Figure 6B or Figure 6C ), the distance between the instrument tip and the nearby anatomical structures, if any, is calculated at 535, and this distance will be used to further assess whether the current situation is Figure 6B shown above or Figure 6C shown above.

[0044] To use the calculated distance to determine the specific situation, at 545 a predetermined criterion is accessed that defines what constitutes a close distance, and then at 555 this predetermined criterion is used to determine whether the calculated distance meets the criterion of being close to an anatomical structure. If the close condition is not met (which is the case in Figure 6B above), the next insertion pose specified in the pre-planned insertion path can be employed as the next target instrument pose. Otherwise, the tip of the instrument can be deemed too close to an obstacle (which is the case in Figure 6CAs discussed herein, the next target instrument pose that is adaptively computed is then used to adjust robot joint positions to facilitate robot arm motions to move the surgical instrument to the next target instrument pose. Since the relative spatial relationship between the pose of the instrument tip and the pose of the robot base is known through calibration, the required adjustment of robot joint positions is a matter of kinematic transformation.

[0045] Figure 7 is a schematic diagram of an exemplary mobile device architecture that can be used to implement a special-purpose system for implementing the present teachings in accordance with various embodiments. In this example, a user device on which the present teachings can be implemented corresponds to a mobile device 700, including but not limited to a smart phone, a tablet computer, a music player, a handheld game console, a global positioning system (GPS) receiver, and a wearable computing device, or in any other form factor. The mobile device 700 can include one or more central processing units ("CPUs") 740, one or more graphics processing units ("GPUs") 730, a display 720, a memory 760, a communication platform 710 such as a wireless communication module, a storage 790, and one or more input / output (I / O) devices 750. Any other suitable components, including but not limited to a system bus or controller (not shown), can also be included in the mobile device 700. As Figure 7 indicated, a mobile operating system 770 (e.g., iOS, Android, Windows Phone, etc.) and one or more applications 780 can be loaded from the storage 790 into the memory 760 for execution by the CPU(s) 740. The applications 780 can include, at least in part, a user interface or any other suitable mobile application for information analysis and management in accordance with the present teachings on the mobile device 700. User interaction, if any, can be implemented via the I / O devices 750 and provided to various components via network connection(s).

[0046] To implement the various modules, units, and their functions described in this disclosure, a computer hardware platform can be used as the hardware platform(s) of one or more elements described herein. The hardware elements, operating systems, and programming languages of such computers are conventional in nature and are assumed to be sufficiently familiar to those skilled in the art for adaptation to the appropriate settings described herein. A computer having user interface elements can be used to implement a personal computer (PC) or other type of work station or terminal device, but a computer also can act as a server if appropriately programmed. Those skilled in the art believe that those skilled in the art are familiar with the structure, programming, and general operation of such computer equipment, and therefore the drawings should be self-explanatory.

[0047] Figure 8 is a schematic diagram of an exemplary computing device architecture that can be used to implement a special-purpose system implementing the present teachings according to various embodiments. Such special-purpose systems incorporating the present teachings have a functional block diagram illustration of a hardware platform that includes user interface elements. The computer can be a general-purpose computer or a special-purpose computer. Both can be used to implement special-purpose systems for the present teachings. The computer 800 can be used to implement any component or aspect of the frameworks disclosed herein. For example, the information analysis and management methods and systems disclosed herein can be implemented on a computer such as the computer 800 via the computer's hardware, software programs, firmware, or a combination thereof. Although only one such computer is shown for convenience, the computer functions related to the present teachings described herein can be implemented in a distributed manner across several similar platforms to distribute processing loads.

[0048] The computer 800 includes, for example, a COM port 850 that is connected to and from a network connected to and from the COM port 850 to facilitate data communications. The computer 800 also includes a central processing unit (CPU) 820 in the form of one or more processors for executing program instructions. An exemplary computer platform includes an internal communication bus 810, a main memory 830, and a system memory 840, for storing various program instructions and data files used in connection with the functionality of the present teachings. The main memory 830 and system memory 840 can be used to load and store program instructions and data files for execution by the CPU 820. The computer 800 also includes an I / O component 860 that facilitates input / output operations and interactions with other devices or systems (such as user interface elements 880) that can be used in connection with the present teachings. The computer 800 can also receive programs and data via, for example, a network communication.

[0049] Accordingly, as indicated above, aspects of the method and / or other processes for information analysis and management can be embodied in programming. Program aspects of the technology can be thought of as "products" or "articles of manufacture" typically in the form of executable code and / or associated data that is carried on or in one type of machine-readable medium. Tangible non-transitory "storage" type media include any or all of the memory or storage of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which can provide storage at any time for the software programming.

[0050] All or portions of the software can at times be made available through a network (such as the Internet or various other telecommunication networks). Such communications, for example, can enable loading of the software from one computer or processor into another computer or processor, e.g., as associated with information analysis and management. Thus, another type of media that can bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces, through wired and optical fiber cables, through wires, printed material, and over the air, such as via radio and television. A physical element of a medium can be the circuitry that comprises the various components of the processor, such as various semiconductor circuits, which can carry or communicate the software. Thus, as used herein the term "non-transitory" means not having a period of existence. As used herein, the term "non-transitory" is not intended to encompass waves, signals, photons and other forms of electromagnetic radiation transitory signals.

[0051] Accordingly, a machine readable medium can take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, which ultimately store any or all of the programs, data structures, tables, or the like described herein. Volatile storage media include, for example, memory within any computer or the like, which provides temporary storage for any or all of the programs, data structures, tables, or the like described herein. Tangible transmission media include, for example, coaxial cables, copper wire and fiber optics, including the wires that comprise a bus within any computer system. Carrier-wave transmission media can take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, a hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards, paper tape, any other physical storage medium that can be used to store or transfer data or information, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such carrier waves, or any other medium applicable to the transport of data or instructions. Many of these forms of computer readable media can be involved in carrying one or more sequences of one or more instructions to a physical processor for execution.

[0052] Those skilled in the art will realize that the teachings of the present document are consonant with a variety of modifications and / or enhancements. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a software only solution, e.g., installed on an existing server. In addition, the technology as disclosed herein can be implemented as firmware, firmware / software combination, firmware / hardware combination, or hardware / firmware / software combination.

[0053] While the foregoing has described what are considered the principles and examples of the present teachings, it is understood that various modifications can be made thereto and that the subject disclosed herein can be implemented in various forms and examples and that the teachings can be applied in numerous applications, only some of which have been described herein. The claims are intended to embrace any and all applications, modifications and variations of the technology disclosed herein.

Claims

1. A method implemented on at least one processor, memory, and communication platform, the method comprising: retrieving a path generated for a pre-planning of a surgery on a patient relative to a target in the patient, wherein the pre-planned path is between a three-dimensional (3D) entry pose on the patient's skin and a 3D pose of the target and is provided to a robot to insert a surgical instrument from the 3D entry pose to reach the 3D pose of the target; determining a next pose of the surgical instrument based on a current pose of the surgical instrument and the pre-planned path and spatial relationships with surrounding anatomy; controlling the robot to move the surgical instrument to reach the next pose; obtaining an updated current pose of the surgical instrument via tracking as the robot advances the surgical instrument to the next pose; repeating the steps of determining, controlling, and obtaining if the updated current pose is not the 3D pose of the target determined based on predetermined criteria; and outputting a signal indicating that the surgical instrument reached the 3D pose of the target when the updated current pose reaches the 3D pose of the target based on the predetermined criteria.

2. The method of claim 1, wherein, The step of determining the next pose comprises: retrieving a 3D model structured to characterize anatomy of the patient; deriving spatial relationships between the current pose of the surgical instrument and at least some of the anatomy based on the 3D model; and computing the next pose based on the spatial relationships and the 3D pose of the target in the pre-planned path, wherein the spatial relationships include distances and spatial configurations relative to each of the at least some of the anatomy.

3. The method of claim 2, wherein, The step of computing the next pose comprises: retrieving a subsequent pose on the pre-planned path as a candidate next pose; adopting the candidate next pose as the next pose if the candidate next pose does not cause a collision with or risk of collision with any of the anatomy; and modifying the candidate next pose to derive the next pose to avoid or prevent a collision with any of the anatomy.

4. The method of claim 3, wherein the collision is determined based on an orientation associated with the candidate next pose; and the risk of the collision relative to the anatomy is determined based on a distance between the candidate next pose and the anatomy. The step of controlling the robot comprises:

5. The method of claim 1, wherein, determining a difference in a first set of parameters for configuring the robot to move the surgical instrument to the current pose and a second set of parameters required to configure the robot to move the surgical instrument to the next pose; and configuring the robot based on the difference to enable the robot to control movement of the surgical instrument from the current pose to the next pose. ​ 6. The method of claim 1, wherein, The predetermined criteria are defined in terms of a distance between the current pose and the 3D pose of the target.

7. The method of claim 1, wherein, The pre-planned path is generated by: obtaining information related to the patient, and operational parameters associated with the surgery; retrieving a 3D model structured to characterize the anatomy of the patient; estimating a start 3D pose of an entry on the skin of the patient and an end 3D pose of the target; determining, from the 3D model, a plurality of 3D poses between the start 3D pose and the end 3D pose that do not collide with the anatomy; and creating the pre-planned path based on the plurality of 3D poses.

8. A machine-readable and non-transitory medium having information recorded thereon, wherein, The information, when read by the machine, causes the machine to perform the steps of: retrieving a pre-planned path relative to a target in a patient's body for a pre-planning of a surgery on the patient, wherein the pre-planned path is between a three-dimensional (3D) entry pose on the patient's skin and a 3D pose of the target and is provided to a robot to insert a surgical instrument from the 3D entry pose to reach the 3D pose of the target; determining a next pose of the surgical instrument based on a current pose of the surgical instrument and the pre-planned path and spatial relationships with surrounding anatomy; controlling the robot to move the surgical instrument to reach the next pose; obtaining an updated current pose of the surgical instrument via tracking as the robot advances the surgical instrument to the next pose; repeating the steps of determining, controlling, and obtaining if the updated current pose is not the 3D pose of the target determined based on predetermined criteria; and outputting a signal indicating that the surgical instrument reached the 3D pose of the target when the updated current pose reaches the 3D pose of the target based on the predetermined criteria.

9. The medium of claim 8, wherein, The step of determining the next pose includes: retrieving a 3D model structured to characterize the anatomy of the patient; deriving, based on the 3D model, spatial relationships between the current pose of the surgical instrument and at least some of the anatomy; and computing the next pose based on the spatial relationships and the 3D pose of the target in the pre-planned path, wherein the spatial relationships include distances and spatial configurations relative to each of the at least some of the anatomy.

10. The medium of claim 9, wherein, The step of computing the next pose includes: retrieving a subsequent pose on the pre-planned path as a candidate next pose; adopting the candidate next pose as the next pose if the candidate next pose does not cause or risk causing a collision with any of the anatomy; and modifying the candidate next pose to derive the next pose to avoid or prevent a collision with any of the anatomy.

11. The medium of claim 10, wherein the collision is determined based on an orientation associated with the candidate next pose; and determining a risk of collision with the anatomical structure based on a distance between the candidate next pose and the anatomical structure.

12. The medium of claim 8, wherein, The step of controlling the robot comprises: determining a difference between a first set of parameters for configuring the robot to move the surgical instrument to the current pose and a second set of parameters required to configure the robot to move the surgical instrument to the next pose; and configuring the robot based on the difference to enable the robot to control movement of the surgical instrument from the current pose to the next pose.

13. The medium of claim 8, wherein, The predetermined criteria is defined in terms of a distance between the current pose and the 3D pose of the target.

14. The medium of claim 8, wherein, The pre-planned path is generated by: obtaining information related to the patient, and operational parameters associated with the surgery; retrieving the 3D model configured to characterize the anatomical structure of the patient; estimating a start 3D pose of an entry on the skin of the patient and an end 3D pose of the target; determining, from the 3D model, a plurality of 3D poses between the start 3D pose and the end 3D pose that do not collide with the anatomical structure; and creating the pre-planned path based on the plurality of 3D poses.

15. A system comprising: a next pose determiner implemented by a processor and configured to retrieve a pre-planned path relative to a target within a patient generated for a pre-planning of a surgery of the patient, wherein the pre-planned path is between a three-dimensional (3D) entry pose on the skin of the patient and a 3D pose of the target and is provided to a robot to insert a surgical instrument from the 3D entry pose to reach the 3D pose of the target, and determine a next pose of the surgical instrument based on a current pose of the surgical instrument and the pre-planned path and a spatial relationship with surrounding anatomical structure; a robot-guided instrument insertion controller implemented by a processor and configured to control the robot to move the surgical instrument to reach the next pose; and a current instrument pose determiner implemented by a processor and configured to obtain, via tracking, an updated current pose of the surgical instrument as the robot advances the surgical instrument to the next pose, wherein the next pose determiner, the robot-guided instrument insertion controller, and the current instrument pose determiner are configured to repeat the steps of determining, controlling, and obtaining if the updated current pose is not the 3D pose of the target determined based on predetermined criteria, and output a signal indicating that the surgical instrument reached the 3D pose of the target when the updated current pose reaches the 3D pose of the target based on the predetermined criteria.

16. The system of claim 15, wherein, The next pose determiner is configured to determine the next pose by: retrieving a 3D model configured to characterize an anatomical structure of the patient; deriving a spatial relationship between the current pose of the surgical instrument and at least some of the anatomical structures based on the 3D model; and computing the next pose based on the spatial relationship and the 3D pose of the target in the pre-planned path, wherein the spatial relationship comprises a distance and a spatial configuration with respect to each of the at least some of the anatomical structures.

17. The system of claim 16, wherein, The step of computing the next pose comprises: retrieving a subsequent pose on the pre-planned path as a candidate next pose; adopting the candidate next pose as the next pose if the candidate next pose does not cause or is not at risk of causing a collision with any of the anatomical structures; and modifying the candidate next pose to derive the next pose to avoid or prevent a collision with any of the anatomical structures.

18. The system of claim 17, wherein the collision is determined based on an orientation associated with the candidate next pose; and the risk of the collision with respect to an anatomical structure is determined based on a distance between the candidate next pose and the anatomical structure.

19. The system of claim 15, wherein, The robot-guided instrument insertion controller is configured to control the robot by: determining a difference between a first set of parameters for configuring the robot to move the surgical instrument to the current pose and a second set of parameters required to configure the robot to move the surgical instrument to the next pose; and configuring the robot based on the difference to enable the robot to control movement of the surgical instrument from the current pose to the next pose.

20. The system of claim 15, wherein, The predetermined criterion is defined in terms of a distance between the current pose and the 3D pose of the target.

21. The system of claim 15, further comprising a surgical path pre-planning unit implemented by a processor and configured to generate the pre-planned path by: obtaining information related to the patient, and operational parameters associated with the surgery; retrieving the 3D model structured to characterize the anatomical structure of the patient; estimating a start 3D pose of an entry on the skin of the patient and an end 3D pose of the target; determining a plurality of 3D poses between the start 3D pose and the end 3D pose that do not collide with the anatomical structure from the 3D model; and creating the pre-planned path based on the plurality of 3D poses.

Images