System and method for navigation

Abstract

A system for assisting in guiding and performing a procedure on a subject is disclosed. The subject may be any suitable subject, such as a non-living subject and / or a living subject. The system may include one or more tracking systems. In various embodiments, two tracking systems may operate cooperatively to ensure selected accuracy and / or precision.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 515,888, filed July 27, 2023. This application includes subject matter relating to the subject matter disclosed in U.S. Patent Application No. 63 / 516,006 (Attorney's File No. A0011164US01 / 5074A-000314-US-PS1), also filed July 27, 2023. The entire disclosure of the above application is incorporated herein by reference. Technical Field

[0003] This topic discloses information in general about a tracking and navigation system, and more specifically about using one or more tracking systems to track and determine the relationships between these tracking systems. Background Technology

[0004] This section provides background information related to this disclosure, which is not necessarily prior art.

[0005] The instrument can be navigated relative to a subject to perform various surgical procedures. For example, the subject may include a patient on whom surgery is being performed. During surgery, the instrument can be tracked in a physical space, which may also be referred to as the object space or subject space. In various embodiments, the subject space may be a patient space defined by the patient. The position of the tracked instrument can be displayed on a display device relative to an image or view of the patient.

[0006] The patient's position can be determined using a tracking system. Typically, the patient is registered to the image by tracking the device relative to the patient to generate a transformation (e.g., translation and rotation) mapping between subject space or object space (e.g., patient space) and image space. This usually requires a user (e.g., a surgeon) to spend time during surgery identifying one or more points in subject space and associating them with the same points in image space.

[0007] After registration, the device's position can be appropriately displayed on the display device while tracking the device. The device's position relative to the subject can be displayed graphically, sometimes referred to as an icon on the display device. Summary of the Invention

[0008] This section provides a general overview of this disclosure and is not a complete disclosure of the full scope or all features of this disclosure.

[0009] According to various embodiments, the imaging system can be used to acquire image data of a subject. The imaging system may include an ultrasound (US) imaging system comprising a US probe, which typically includes an ultrasound transducer for transmitting and receiving ultrasound frequencies. However, it should be understood that the imaging system may include separate components for transmitting and receiving ultrasound frequencies.

[0010] According to various embodiments, the US probe can be moved relative to the subject, such as by a user and / or by a robotic system. However, the US probe can be moved relative to the subject in any suitable manner. Furthermore, the US probe can be held relative to the subject by suitable holders or mounts. In any case, various objects can be placed relative to the subject, such as in or near the subject's space. These objects can be formed of various materials, such as conductive materials including metals or metal alloys.

[0011] Objects made of conductive materials (also known as interfering objects) may induce currents within themselves due to the field generated near the conductive object. Conductive materials can also be referred to as interfering materials. For example, an electromagnetic field from a tracking system can induce currents, such as eddy currents, within a conductive material. The conductive material or object can then emit fields that the tracking system does not emit. Fields emitted by conductive objects or materials can interfere with fields emitted by the electromagnetic tracking system. Therefore, various objects made of conductive materials can create interfering fields. These interfering fields can interfere with tracking by the tracking device.

[0012] A tracking or navigation system may include at least two tracking systems. These two tracking systems may operate independently or collaboratively to ensure that the navigation of the selected tracking device achieves a selected level of accuracy. The tracking systems may operate independently within a selected time or volume to achieve local navigation. The two tracking systems may work collaboratively to ensure that the navigation of the tracking device achieves global accuracy. These two tracking systems may include, for example, optical tracking systems, visual tracking systems, electromagnetic tracking systems, inertial tracking systems, or combinations thereof.

[0013] Two or more tracking systems can be used to ensure global accuracy and precision over a given time period and / or volume. Furthermore, the two or more tracking systems can be registered with each other and / or with the subject. Therefore, two or more tracking systems can be operable to narrow the tracking area over a large volume and / or time period.

[0014] Further applicable areas will become clear from the description provided herein. The descriptions and specific examples in this invention are intended for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0015] The accompanying drawings described herein are for illustrative purposes only, and not all possible embodiments, and are not intended to limit the scope of this disclosure.

[0016] Figure 1 It is a diagrammatic view showing an overview of robot systems and navigation systems according to various embodiments; Figure 2 It is a representation of two or more tracking systems for tracking one or more objects, according to various embodiments; Figure 3 It is a representation of two or more tracking systems for tracking one or more objects, according to various embodiments; Figure 4 It is a representation of an operating room having two or more tracking systems for tracking one or more objects, according to various embodiments; Figure 5 It is a representation of two or more tracking systems for tracking one or more objects, according to various embodiments; Figure 6 It is a representation of two or more tracking systems for tracking one or more objects, according to various embodiments; Figure 7 It is a representation of two or more tracking systems for tracking one or more objects, according to various embodiments; Figure 8 These are flowcharts of methods for tracking objects according to various embodiments; and Figure 9 This is a flowchart of a method for tracking an object according to various embodiments.

[0017] In several views of the accompanying drawings, corresponding reference numerals indicate the corresponding parts. Detailed Implementation

[0018] Example embodiments will now be described more fully with reference to the accompanying drawings.

[0019] This subject matter disclosure relates to exemplary embodiments of surgical procedures performed on subjects, such as human patients. However, it should be understood that the systems and methods described herein are merely exemplary and are not intended to limit the scope of the claims included herein. In various embodiments, it should be understood that these systems and methods can be incorporated into and / or used on inanimate objects. These systems can be used, for example, to register two or more coordinate systems for use by manufacturing systems, maintenance systems, etc. For example, automotive assembly may use one or more robotic systems comprising separate coordinate systems that can be registered together to achieve coordinated or cooperative action. Therefore, the exemplary description of surgical procedures herein is not intended to limit the scope of the appended claims.

[0020] As discussed herein, according to various embodiments, one or more tracking systems can be used to track selected tracking devices. According to various embodiments, at least one tracking system can be operated by emitting an EM field from a locator (also referred to as an electromagnetic (EM) locator). The EM field can be emitted from one or more coils that can be oriented relative to the origin. The field may primarily be a magnetic field. The field may be constant or may vary in time and / or frequency. In an EM tracking system, the tracking device may include one or more coils that operate as sensors to sense the field. The field can generate a current within the coils of the tracking device. The position and orientation (also collectively referred to as "pose") of the tracking device can be determined.

[0021] According to various embodiments, at least one tracking system can operate via an optical system that can emit or receive selected radiation, such as using an optical sensor. The optical sensor can sense visible light or other types of light, such as infrared (IR). For example, two or more sensors can be positioned to perform triangulation on an optical or visual sensor. As discussed herein, a visual sensor can sense visible light wavelengths (e.g., from about 350 nm to about 800 nm), while an optical sensor can sense non-visible wavelengths (e.g., IR from about 700 nm to about 1 mm). The tracking device can emit and / or reflect such wavelengths.

[0022] According to various embodiments, at least one tracking system can operate by sensing motion, such as by measuring component or vector acceleration via an accelerometer, or by measuring component or vector orientation via a gyroscope. According to various embodiments, these sensors may be referred to as inertial sensors. An exemplary sensor may be the MPU-9250 nine-axis (gyroscope + accelerometer + compass) MEMS MotionTracking™ device sold by InvenSense. This sensor can sense motion in at least three degrees of freedom. Therefore, two or more sensors can be used to sense motion in at least six degrees of freedom, including translation and rotation. Further, various sensors may include sensors for sensing selected degrees of freedom.

[0023] Various conditions can adversely affect the accuracy and / or precision of one or more of these tracking systems. For example, various materials are conductive, such as conductive polymers, metals or metal alloys, or other materials. Objects or articles can be formed from these materials. If an article formed from these materials is also in or near the field generated by the EM locator, a current may be formed or induced in the object. In this case, the object can be referred to as an interfering object. When a current is induced in the interfering object, a field is also generated. The field generated due to the induced current in the interfering object can also be referred to as an interfering field. These interfering fields can alter the field sensed by the tracking device, so that the tracking device does not always sense the EM field generated by the EM locator. Inertial sensors may drift over time. Furthermore, optical or vision sensors may require unobstructed lines of sight. Therefore, at any given moment, one or more tracking systems may be adversely affected by given conditions.

[0024] Regardless, individual parts can be tracked relative to a subject. For example, a tracking system can be incorporated into a navigation system that includes one or more instruments capable of tracking relative to a subject. The navigation system may include one or more tracking systems that track individual parts (e.g., tracking devices) associated with the instruments. The tracking system may include a locator configured to determine the pose of the tracking device in the navigation system coordinate system, either individually or in combination with a processor. The determination of the navigation system coordinate system may include those described in various references, including: U.S. Patent Nos. 8,737,708; 9,737,235; 8,503,745; and 8,175,681, all of which are incorporated herein by reference. In particular, the locator may be able to track an object within a volume relative to a subject. The navigation volume in which the tracking device may be tracked may include, or is referred to as, a navigation coordinate system or navigation space. The determination or association between two coordinate systems may allow for registration between the two coordinate systems, or is also referred to as registration between two coordinate systems.

[0025] In addition, images of selected portions of the subject can be acquired. These images can be displayed for viewing by a user (e.g., a surgeon). These images may have a graphical representation (e.g., an icon) of the tracked portion or component (e.g., an instrument) superimposed on at least a portion of the image. These images may have a coordinate system and define an image space. According to various embodiments, the graphical representation can be superimposed at an appropriate location on the image due to registration between the image space (also referred to as the image coordinate system) and the subject space (also referred to as the subject coordinate system or physical coordinate system). A method for registering a subject space defined by the subject (including the associated and / or included physical space) to an image space may include those disclosed in the following references: U.S. Patent Nos. 8,737,708; 9,737,235; 8,503,745; and 8,175,681; all of which are incorporated herein by reference.

[0026] As discussed herein, registration between image space and subject space may not be necessary. The imaging system (e.g., a US probe) can be tracked, as can an instrument that is separate from or secondary to the imaging system. Therefore, the image coordinate space is tracked using the imaging system, and the tracked pose of the instrument can be tracked using the same or associated tracking system. Thus, the pose of the instrument within the image can be determined based on the associated tracking system.

[0027] Nevertheless, image-subject registration can be performed. For example, during a selected procedure, the coordinate system can be registered to the subject space or subject coordinate system due to the selected procedure (e.g., imaging of the subject). In various embodiments, the first coordinate system can be registered to the subject by imaging the subject using a reference portion fixed relative to a first component or system (e.g., a robotic system). Since the subject's image includes the reference portion, the known position of the reference relative to the robotic system can be used to register the subject space relative to the robotic system. Therefore, the position of the robotic system or a portion thereof (e.g., an end effector) relative to the subject can be known or determined. Due to the registration of the second coordinate system with the robot coordinate system, it is possible to track additional elements not fixed to the robot relative to a position determined or tracked by the robot.

[0028] Tracking of instruments during surgery (such as surgical procedures or procedures) allows for instrument navigation during surgery and may enable guided surgery. When image data is used to define an image space, the image space can be associated with or registered to a physical space defined by a subject (such as the patient discussed herein). Thus, according to various embodiments, the patient defines a patient space in which instruments can be tracked and navigated. The image space defined by the image data can be registered to the patient space defined by the patient. Registration can be performed using reference elements that can be identified in both the image data and the patient space.

[0029] Figure 1 This is a schematic view showing an overview of an operating room or surgical area. In various embodiments, the operating room may include a surgical suite in which a robotic system 20 and a navigation system 26, which can be used for various surgical procedures, may be housed. The robotic system 20 may include the Mazor X™ robotic guidance system sold by Medtronic, Inc. The robotic system 20 may be used to assist in guiding selected instruments, such as drills, screws, etc., relative to the subject 30. Additionally or alternatively, the robotic system 20 may hold and / or move an imaging system, such as an ultrasound (US) probe 33. The robotic system 20 may include a mounting 34 that secures a portion, such as a robot base 38, relative to the subject 30. The robotic system 20 may include one or more arms 40 that are movable or pivotable relative to the subject 30, such as including an end effector 44. The robotic system 20 may further include a controller 22, which may include one or more processors and / or memory systems. The controller 22 may be present within and / or communicate with the robotic system 20. The controller 22 can access and / or receive instructions that are executed to move the end effector 44 and / or other parts of the arm 40. Therefore, the robot system 20 can be manually controlled by the user 72 and / or automatically controlled, for example, by executing instructions using the controller 22, and / or a combination of both.

[0030] The end effector 44 can be any suitable component, such as a tube, guide, or channel member. An imaging system, such as a US probe 33, can be attached to and / or replace the end effector. The end effector 44 can be moved relative to the base 38 by one or more motors. The position of the end effector 44 relative to the base 38 can be known or determined by one or more encoders at one or more joints of the robot system 20, such as the wrist joint 48 and / or the elbow joint 52. One or more parts of the robot system 20 can be formed of a conductive material.

[0031] Navigation system 26 can be used to track the position of one or more tracking devices and / or determine and / or display the pose of the tracking devices. Tracking devices may include robotic tracking device 54, subject tracking device 58, imaging system tracking device 62, imaging system or second imaging system tracking device 81, and / or instrument or tool tracking device 66. Tools or instruments 68 can be any suitable movable component, such as a drill, forceps, catheter, tube, scalpel, or other tool moved or controlled by user 72. Tools 68 may also or alternatively include implants, such as spinal implants or orthopedic implants. It should be further noted that navigation system 26 can be used to navigate any type of instrument, implant, or delivery system, including: guidewires, arthroscopic systems, orthopedic implants, spinal implants, deep brain stimulation (DBS) probes, etc. Furthermore, instruments can be used to navigate or map any area of ​​the body. Navigation system 26 and various instruments can be used for any suitable surgical procedure, such as procedures that are typically minimally invasive or open.

[0032] Additional or alternative imaging systems 80 may be used to acquire preoperative, intraoperative, or postoperative or real-time image data of a subject (e.g., subject 30). However, it will be understood that imaging can be performed on any suitable subject, and any suitable surgical procedure can be performed relative to that subject. In the example shown, imaging system 80 includes an O-arm sold by Medtronic Navigation, Inc., which has a business location in Colorado, USA. ® Imaging apparatus. Imaging system 80 may have a generally annular rack housing 82 in which image capture portions are movably positioned and / or enclosed. Imaging system 80 may include those imaging apparatuses disclosed in the following U.S. Patent Numbers: 7,188,998; 7,108,421; 7,106,825; 7,001,045; and 6,940,941; all of which are incorporated herein by reference, or any appropriate portion thereof. It should be further understood that imaging system 80 may additionally or alternatively include a fluoroscopic C-arm. Other exemplary imaging apparatus may include fluoroscopy systems such as biplane fluoroscopy systems, ceiling-mounted fluoroscopy systems, catheterization lab fluoroscopy systems, fixed C-arm fluoroscopy systems, isocentric C-arm fluoroscopy systems, 3D fluoroscopy systems, etc. Other suitable imaging apparatus may also include MRI, CT, ultrasound, etc.

[0033] The position of the imaging systems 33, 80 and / or portions thereof (such as the image capture portion) relative to any other portion of the imaging devices 33, 80 and / or within the navigable domain may be known (e.g., with an accuracy of approximately 0.1 mm to approximately 5 mm). The navigable domain or navigation space can be the physical space in which any one or more tracking systems can track the tracking devices. According to various embodiments, the imaging devices 33, 80 may know and / or recall precise coordinates relative to a fixed or selected coordinate system. For example, the robotic system 20 may know or determine its position and position the US probe 33 in a selected pose. Similarly, the imaging system 80 may also position the imaging portion in a selected pose. This may allow the imaging system 80 to know its position relative to the patient 30 or other reference objects. Additionally, as discussed herein, precise knowledge of the position of the image capture portion can be used in conjunction with the tracking system to determine the position of the image capture portion and image data relative to the tracked subject (e.g., patient 30). In other words, the imaging system tracking devices 62 and 81 can be used and / or are operable to determine the pose of the imaging systems 33 and 80 at a selected time (e.g., during image data acquisition).

[0034] Unless otherwise stated herein, references to imaging system 33 may refer to any suitable imaging system. Therefore, the US probe 33 as an imaging system is merely exemplary in relation to the disclosure of this subject matter. As those skilled in the art will understand, typically, the US probe 33 can emit a US wave in a plane and receive an echo relative to any part of the area on which the wave acts. The echo received at the US probe 33 or other appropriately received echoes can be used to generate image data and can be used to generate a US image (also known as an ultrasound map).

[0035] Imaging device 80 can be tracked using tracking device 62. Furthermore, tracking device 81 can be directly associated with US probe 33. Therefore, US probe 33 can be directly tracked using navigation system 26 as discussed herein. Alternatively or concurrently, US probe 33 can be located and tracked using robotic system 20. In any case, according to various embodiments, image data defining the acquired image space of patient 30 can be registered relative to object space (e.g., manually, inherently, or automatically). Object space can be the space defined by patient 30 in navigation system 26.

[0036] The patient 30 can also be tracked via a patient tracking device, DRF, or tracker 58 as the patient moves. Alternatively or otherwise, the patient 30 can be fixed within a navigation space defined by navigation system 26 to allow and / or maintain registration, such as registration with the image space of image 108. According to various embodiments, registration of the image space with the patient space or the subject space can allow navigation of device 68 using image data. According to various embodiments, navigation of the device does not require image-subject registration, as discussed herein. When navigating device 68, the position of device 68 relative to the acquired image data of patient 30 can be displayed on display device 84 (e.g., graphically represented as 68i, 68i'). Additional and / or alternative display devices 84' may also be present to display images. Various tracking systems (e.g., tracking systems including optical locator 88, visual locator 91, or electromagnetic (EM) locator 94, or other suitable tracking systems (e.g., inertial tracking systems)) can be used to track device 68.

[0037] More than one tracking system may be used to track instrument 68 or other parts (such as the US probe 33 with tracking device 81 in navigation system 26). According to various embodiments, these tracking systems may include an EM system with an electromagnetic tracking (EM) locator 94 and / or an optical tracking system with an optical locator 88. Any or both of these tracking systems may be used to track a selected tracking device, as discussed herein. It will be understood that, unless otherwise discussed, a tracking device may be a part that can be tracked by a selected tracking system. A tracking device does not necessarily refer to the entire component or structure to which it is attached or associated.

[0038] The position of patient 30 relative to imaging device 33 can be determined by navigation system 26. The position of imaging system 33 can be determined as discussed herein. Patient 30 can be tracked using a dynamic reference frame (or DRF) 58, as further discussed herein. Therefore, the position of patient 30 relative to imaging device 33 can be determined.

[0039] Image data acquired from imaging system 33 or any suitable imaging system may be acquired at and / or forwarded from image device controller 96 to navigation computer and / or processor module (also referred to as processor) 102, which may include the processor module as part of a controller or workstation 98 having display 84 and user interface 106. Further, processor 102 may access any suitable type of memory 103. It will also be understood that image data is not necessarily initially held in controller 96, but may also be directly transferred to workstation 98. Workstation 98 (which may be any suitable computing system) may provide facilities for displaying image data as image 108 on display 84, saving, digitally manipulating, or printing hard copies of the received image data. User interface 106 (which may be a keyboard, mouse, stylus, touchscreen, or other suitable device) allows user 72 to provide input to control imaging devices 80, 33 via image device controller 96, or to adjust display settings of display 84. Workstation 98 can also instruct image device controller 96 to adjust the image capture section of imaging device 80 to obtain various two-dimensional images along different planes in order to generate representative two-dimensional and three-dimensional image data.

[0040] Continue to refer to Figure 1The navigation system 26 may further include one or more tracking systems, including either or both of an electromagnetic (EM) locator 94 and / or an optical locator 88. The tracking system may include a controller and an interface portion 110. The controller 110 may be connected to a processor portion 102, which may include a processor contained within a computer. EM tracking systems may include the STEALTHSTATION® AXIEM™ navigation system sold by Medtronic Navigation, Inc., which has a business office in Louisville, Colorado; or may be the EM tracking system described in the following documents: U.S. Patent Application Serial No. 10 / 941,782, entitled “METHOD AND APPARATUS FOR SURGICALNAVIGATION”, filed September 15, 2004; U.S. Patent No. 5,913,820, entitled “Position Location System”, issued June 22, 1999; and U.S. Patent No. 5,592,939, entitled “Method and System for Navigating a Catheter Probe”, issued January 14, 1997; all of which are incorporated herein by reference. It will be understood that navigation system 26 may also be or include any suitable tracking system, including the STEALTHSTATION® TREON® or S7™ tracking system with an optical locator (which can be used as optical locator 88) sold by Medtronic Navigation, Inc. of Colorado. Other tracking systems include acoustic tracking systems, radiation tracking systems, radar tracking systems, etc. Tracking systems may be used in accordance with techniques generally known or described in the above-incorporated references. Details are not included herein unless for the purpose of clarifying selected operations disclosed in this subject matter.

[0041] Wired or physical connections can interconnect the tracking system, imaging device 80, etc. Alternatively, instead of direct coupling to the controller 110, various components, such as instrument 68, can utilize wireless communication channels, such as those disclosed in U.S. Patent No. 6,474,341, entitled "Surgical Communication Power System," issued November 5, 2002, which is incorporated herein by reference. Furthermore, the tracking devices (e.g., 62, 66, 54) can generate fields and / or signals sensed by the positioners 88, 94.

[0042] Various parts of the navigation system 26 (such as device 68 and other parts described in detail below) may be equipped with at least one tracking device of tracking devices 66, and typically multiple tracking devices. The device may also include more than one type or modality of tracking devices 66, such as EM tracking devices and / or optical tracking devices. Device 68 may include a graspable or manipulable portion at its proximal end, and the tracking device may be fixed near the manipulable portion of device 68.

[0043] Another representative or alternative positioning and tracking system is described in U.S. Patent No. 5,983,126, issued November 9, 1999, entitled "Catheter Location System and Method," which is incorporated herein by reference. Navigation system 26 may be a hybrid system comprising components from various tracking systems.

[0044] According to various embodiments, navigation system 26 can be used to track any appropriate portion, such as US probe 33 and / or device 68, relative to patient 30. As discussed above, a tracking system can be used to track device 68. Image data of patient 30 (or appropriate subject) can be used to assist user 72 in guiding device 68. Image data may or may not be registered to patient 30. For example, as discussed herein, US probe 33 is tracked and generates image data. Therefore, it is not necessary to register the image data to the subject to display the pose of the tracked device 68 relative to an image data generator having the tracked US probe 33. The image data defines an image space to be registered to the patient space defined by patient 30. Registration can be performed automatically, manually, or in combination thereof, as discussed herein. Registration may include procedural and final transformation (including translation and rotation) mappings. Typically, registration includes determining points in image data and subject space, and determining a transformation mapping between the two. Once completed, the image space is registered to subject space or any two or more coordinate spaces.

[0045] Typically, registration also allows for the generation of a transformation mapping of the tracked physical pose of the instrument 68 in the image space relative to the image data. This transformation mapping allows the tracked position of the instrument 68 to be displayed on the display device 84 relative to the image data 108. A graphical representation 68i (also referred to as an icon) can be used to show the position of the instrument 68 relative to the image data 108.

[0046] Continue to refer to Figure 1The subject registration system or method may use tracking device 58. Tracking device 58 may include a portion or component 120 that can be trackable, but may also act as a reference component or operate as a reference component. Reference component 120 may include a clamp or other fixation portion 124 and an imageable reference body 120. However, it should be understood that component 120 may be detachable from tracking device 58. Fixation portion 124 may be provided for fixing any suitable portion, such as a part of an anatomical structure. Figure 1 As shown, the reference component 120 can be interconnected with a portion of the spine 126 (e.g., the spinous process 130). The fixation portion 124 can be interconnected with the spinous process 130 in any suitable manner. For example, a pin or screw can be screwed into the spinous process 130. Furthermore, the tracking device 58 can be operable for tracking using one or more tracking systems or modalities (e.g., an EM tracking system or an optical tracking system).

[0047] like Figure 1 As shown, the imaging device 33 may include a US probe 33, which can be positioned relative to the subject 30, for example, via a robotic system 20 and / or a surgeon 72. In various embodiments, the surgeon 72 may manipulate the robotic arm 20 and / or keep the US probe 33 detached from the robotic arm. Thus, as discussed herein, the robotic system 20 may move the US probe 33 to a selected position relative to the subject 30. According to various embodiments, the imaging system may be positioned relative to the subject in any suitable manner.

[0048] Furthermore, as those skilled in the art will understand, image data acquired using one or more ultrasound arrays 125 of the US probe 33 can be registered in a navigation system, such as the navigation systems disclosed in U.S. Patent Nos. 7,085,400 and 9,138,204, both of which are incorporated herein by reference. Image data acquired within the respective ultrasound array 125 can be image data of the subject 30. When the ultrasound array 125 is registered to the subject 30 using the navigation system 26, the pose of the required image data for the subject 30 (e.g., heart 127 and / or other subject portions (e.g., vertebrae)) can also be determined in the navigation space within the navigation system 26. The image data can be used to generate an image of a specific portion, such as heart 127i. Image 127i can be a reconstruction result based on image data from the US probe 33. Graphical representation 68i can be represented relative to image 108 and / or portions thereof (e.g., image of heart 127i). Furthermore, graphic representation 68i can be superimposed on the reconstruction result and / or image. The reconstruction result may include additional data (e.g., atlas or population data) and may also be referred to as a model. Graphical representations can be overlaid on the model.

[0049] The pose of the imaging plane 129 can be determined by tracking the US probe 33 in the patient space using the tracking device 81. Therefore, due to the pose of the US probe 33, multiple discrete images collected at each pose of the imaging plane 129 can be combined in a selected manner to achieve a three-dimensional image. The pose of the imaging plane 129 relative to the tracking device 81 can be determined in an appropriate manner, such as via mechanical and electromechanical nominal dimensions, or using a calibration system that may include calibration fixtures or other suitable calibration systems. As discussed above, the imaging system 33 can be tracked. Various tracking systems may include and / or require calibration of the imaging system. Therefore, the pose of the tracking device 22 relative to the plane of the US imaging system can be determined and / or known. Various systems and methods are disclosed in the following U.S. patent numbers: 6,379,302; 6,669,635; 6,968,224; 7,085,400; 7,831,082; 8,320,653; 8,811,662; and 9,138,204, all of which are incorporated herein by reference.

[0050] According to various embodiments, an ultrasonic probe can emit or transmit ultrasonic waves in a selected mode or plane. The plane can be of a shape as understood by those skilled in the art. The plane is typically capable of acquiring data in the field of view to generate images, which are also referred to as ultrasound maps when images are generated based on ultrasonic data.

[0051] As described above, navigation system 26 may include multiple tracking systems. For example, an optical tracking system may include an optical locator 88 to track one or more optical tracking devices. An EM locator 94 may be used to track one or more EM tracking devices. In various embodiments, the tracking devices may include multiple parts that can be tracked by both tracking systems, such as a patient tracking device or a DRF 58, including an optical part 120 and an EM part 121. Alternatively or additionally, the corresponding tracking devices may track corresponding or different tracking devices on more than one part, such as a patient tracking device 58, an ultrasound tracking device 81, or an imager tracking device 62.

[0052] In various embodiments, as those skilled in the art will understand, the corresponding tracking systems can operate most efficiently under different conditions. For example, an EM tracking device including an EM locator 94 can emit fields that may be distorted by various objects, such as the housing of the US probe 33, the housing of the imaging system 80, or other systems. Furthermore, the line of sight of the optical locator 88 may be obstructed by opaque parts, such as the frame 82 of the imaging system 80, a portion of the user 72, or even a portion of the subject 30. Therefore, it is possible to select operating only a single tracking system within the tracking system at a time or receiving tracking information only from a single tracking system within the tracking system to track individual parts. Therefore, it is possible to select combining tracking information received from more than one tracking system at a time and weighting the tracking information according to determined and / or predetermined tracking quality indicators for each system to track individual parts. Nevertheless, the coordinate systems of the multiple tracking devices can be registered (also referred to as associated) with each other, such that a position in one tracking device can be transformed into a position in another tracking device, for example, through registration. Furthermore, the coordinate system of the tracking system can be registered to the coordinate system of image 108 and / or the imaging system can be tracked using at least one tracking imaging system from the corresponding tracking system. Therefore, regardless of the tracking system used for tracking, the tracked position of any part can be displayed. Furthermore, the determined position of any instrument can be translated to any other tracking system.

[0053] Various tracking systems can be used to register or confirm the pose of an instrument between different tracking systems. For example, if an EM tracking system with an EM locator 94 generates a distorted field, an optical tracking system 88 can be used to confirm the position of the tracked instrument by confirming the position of the EM locator 94 in the coordinate space of the optical locator 88 or other suitable locators. Furthermore, it is understood that the distortion is local, such that even if the global position of the tracked device is unknown, local changes in pose can be determined using the EM tracking system, as further discussed herein with reference to various embodiments. Therefore, even in the event of distortion or interference, multiple tracking systems can be used to confirm the pose of a tracked device tracked by another or different tracking system and / or to confirm the pose in a small volume or for a short time.

[0054] Furthermore, it should be understood that various other tracking systems can be used. For example, a visual tracking system may include one or more optical sensors capable of identifying the shape or portion of the tracked device, including the contours of the instrument itself. Further, the tracking system may include an inertial tracking system. In an inertial tracking system, drift may occur over time, and therefore a secondary tracking system can confirm or reconfirm the pose of the tracked device with inertial sensors. Therefore, according to various embodiments, various tracking systems, as further discussed herein, can be used simultaneously and / or individually.

[0055] The system (including the navigation system 26 discussed above) comprises various components, including an EM locator 94 and / or an optical or visual locator, which includes a visual camera 93 or an optical locator camera 88. As discussed above, the visual and / or optical cameras may be included in spaced-out components (e.g., Figure 1 The optical locator shown herein (and / or included in a user wearable system such as augmented reality system 91) is used in, and / or incorporated into, a user wearable system. It should be understood that the discussion herein regarding EM locator 94 and EM tracking systems, as well as optical or visual locator 93 and optical or positional tracking tracking system 88, can be associated with any suitable and relevant tracking system. Therefore, the discussion herein regarding optical tracking system 93 and EM tracking systems with EM locator 94 is merely exemplary, and various features associated with it can also be associated with or related to other systems. For example, as disclosed above, inertial tracking systems can be used to track instruments by sensing movement. However, drift may occur in inertial trackers over time, and the use of visual tracking system 93 may be advantageous and / or useful for confirming or resetting the pose of the tracking device. Therefore, the exemplary systems discussed herein are to be understood to relate to other systems unless otherwise specifically stated.

[0056] As disclosed above, one or more tracking systems can be used to track one or more objects, including at least their various parts. For example, such as Figure 2As shown, optical tracking system 88 and / or visual tracking system 93 can track various tracking devices, including those discussed above and herein. Optical or visual tracking device 170 can be associated with an EM locator 94 tracked by at least one of optical tracking system 88 and / or visual tracking system 93. For example, optical tracking device 170 can be attached to the outer housing or enclosure of EM locator 94 so that it is visible to or within the line of sight of either or both of optical locator 88 and / or visual locator 93. Unless otherwise explicitly stated, discussions herein referring only to one of optical locator 88 or visual locator 93 will be understood to refer to both. Thus, optical tracking device 170 can also be referred to as a visual tracking device tracked by visual locator 93. EM tracking device 174 can also be associated with optical locator 88, and / or EM tracking device 176 can be associated with visual locator 93. Therefore, each of optical locator 88 and visual locator 93 can be tracked using EM locator 94 in an EM tracking system. Furthermore, various instruments, such as the US probe 33, can have an associated visual tracking device 180. The visual tracking device 180 can be tracked using a visual locator 93 or any suitable locator as discussed above.

[0057] Visual tracking devices 170 and 180 can be configured in any suitable shape, configuration, geometry, or have markings, etc., features that can be visually identified or optically identified by the corresponding tracking system (including visual locator 93). For example, visual tracking device 170 may include one or more icons or features 184a, 184b, 184c, and 184d. Visual tracking device 180 may also include one or more markers 188a, 188b, 188c, and 188d. Markers 184 and 188 may be identical to each other except for size, shape, and / or configuration. Furthermore, the markers can be any suitable type of marker, such as an alternating color checkerboard, geometric shape, etc. Visual tracking devices 170 and 180 can be distinguished from each other by various features, such as the configuration of the corresponding markers 184 and 188 and / or their position relative to each other. For example, visual tracking device 170 may include marker 184b as an arrow. In visual tracking device 170, arrow 184b may point to marker 184c. In visual tracking device 180, marker 188b can also be an arrow, but it points to marker 188a. Therefore, navigation system 26 can distinguish between the two visual tracking devices 170 and 180.

[0058] Furthermore, various other visual differences between the visual tracking devices (such as 170, 180) may include the geometry of markers 184, 188. For example, each of markers 184, 188 may include a corresponding center 184a' and its subsequent sequence, as well as 188a' and its subsequent sequence. Centers 184a' and 188a' may be positioned at various distances relative to other markers. For example, center 184a' may be at a distance of 192 from the center 184d' of marker 184d. Center 188a' may be at a second distance of 196 from the center 188d' of marker 188d. Thus, these unique or different configurations can also be used to individually identify these various visual tracking devices 170, 180.

[0059] The field of view 200 of the visual locator 93 can simultaneously include both visual tracking devices 170 and 180. Therefore, the field of view 200 of the visual locator 93 can simultaneously observe both visual tracking devices 170 and 180, and allows determination of the pose of each visual tracking device within at least the navigation space of the visual locator 93. Furthermore, the corresponding or relative poses between the two visual tracking devices 170 and 180 can be determined, for example, using the navigation system 26.

[0060] For example, such as Figure 1 As shown, the visual tracking device 180 associated with the US probe 33 is within the field of view of the visual locator 93, and the visual tracking device 170 associated with the EM locator 94 is also within that field of view. Therefore, the navigation system 26 can determine the pose of each of the US probe 33 and the EM locator 94, and / or the pose between the two. Due to the visual locators 170 and 180 associated with the corresponding parts, the pose of either or both of the EM locator 94 and the US probe 33 can also be determined in the coordinate space or navigation space of the visual locator 93.

[0061] It should be understood that the visual tracking device may also be associated with any other component, such as the robotic system 20, the optical locator 88, the imaging system 80, and / or the tool 68. For example, the tool 68 may include a visual tracking device 204 associated with it. However, as discussed herein, the tool 68 may include only a single tracking device 66, and the tracking device 66 may be an EM tracking device. As further discussed herein, the instrument 68 may be within the image data or imaging plane of the US probe 33. Since the instrument 68 has an associated tracking device 66, which may be an EM tracking device, the pose of the imaging plane emitted or generated by the US probe 33 may be correlated with the pose of the instrument 68 tracked by the EM tracking device 66, at least because the visual locator 93 is able to view the visual tracking device 170 associated with the EM locator 94 and / or the visual tracking device 180 associated with the US probe 33.

[0062] Steering Reference Figure 3 The visual locator 93 can be used by the user 72 to view one or more visual trackers, such as visual tracker 170 associated with EM locator 94 and visual locator 180 associated with US probe 33. US probe 33 can be used to generate image data in image plane 129 of subject 30. Figure 3 As shown, the visual tracker 180 can be within the field of view of the visual locator 93. As discussed above, the visual locator 93 can be associated with a corresponding system, such as an augmented reality or virtual reality headset 91 worn by the user 72. However, it should also be understood that the optical locator 88 can be provided for tracking selected elements, and may even include visual tracking devices 170, 180. Additionally, the optical locator 88 can have an associated visual tracking device 95. The visual tracking device 95 may include features and markers similar to those discussed above, but allows the visual locator 93 to track the optical locator 88.

[0063] The visual locator 93 can define a coordinate space within its field of view. The coordinate space of the visual locator 93 can use the camera of the visual locator 93 as the origin and / or any other suitable portion. Therefore, various visual tracking devices 170, 180, 95 can be tracked within a single coordinate space of the visual locator 93.

[0064] like Figure 3 As shown, the EM locator 94 can be positioned relative to the subject 30 and its various parts (e.g., device 68). Device 68 may include a tracking device 66, which can be an EM tracking device. Device 68 may have a first portion 220 outside the subject 30 and a second portion 224 inside the subject 30 (e.g., extending through an incision 226). The incision 226 can be formed within the subject in any suitable manner. Device 68 can be a selected device, such as an ablation device, catheter, etc. Nevertheless, the internal portion 224 can be positioned within the subject 30, and the EM tracking device 66 can be associated with this internal portion. Therefore, an EM tracking system including the EM locator 94 can be used to track the pose of the EM tracking device 66 and the associated device 68 (including the internal portion 234). The US probe 33 can generate an imaging plane 129. However, as... Figure 3As shown, neither the US probe 33 nor the subject 30 is tracked using the EM locator 94 in the EM tracking system. Therefore, the tracked pose of the device 68 may not be directly determined by the visual tracking device 93 and / or directly displayed by the display 84. Nevertheless, the EM locator 94 can be tracked using the visual tracking device 170. The visual tracking device 170, tracking in the coordinate space of the visual locator 93, allows for registration or association between the coordinate spaces of the EM locator 94 and the visual locator 93. In other words, the coordinate space or navigation space of the EM locator 94 can be transformed to the coordinate space or navigation space of the visual locator 93. The navigation system 26 can know the position of the visual locator 170 on the EM locator 94. Therefore, the field emitted from the EM locator 94 and its corresponding points can be transformed to the coordinate space of the visual locator 93. This transformation is based on the known pose of the visual local tracking device 170 associated with the EM locator 94 and the visual locator 93. Therefore, the tracked pose of the EM tracking device 66 associated with the instrument 68 can also be transformed through the coordinate space or navigation space of the visual locator 93. The visual locator 93 is also capable of tracking the visual tracking device 180 associated with the US probe 93. Therefore, the pose of the image plane 129 can also be known in the coordinate space of the visual locator 93. Therefore, the navigation system 26 can be able to execute instructions to transform the pose of the EM tracking device 66 into the coordinate space of the visual locator 93. Therefore, the pose of the image plane 129 in the visual positioning coordinate space can be transformed or registered to the pose of the EM tracking device 66 on the instrument 68. Therefore, an image generated or reconstructed from image data from the image plane 129 can be registered to the pose of the instrument 6. This can be particularly useful when the instrument 68 is not in the image plane of the US probe 33. For example, the instrument 68 can be at a distance from the image plane 129, such as the instrument 68 shown as a dashed line. When a variant of the device 68 is located outside or far from the image plane 129, the display device 84 can still display the image 108 and the relative pose of the device even if the device 68 is not in the image plane 129 due to the registration of the coordinate space of the EM locator with the coordinate space of the visual locator 93.

[0065] Continue to refer to Figure 3 And other references Figure 4 The visual tracking system 93 has both a visual tracking device 170 associated with the EM locator 94 and a visual tracking device 180 associated with the US probe 33 within its field of view 200. The US probe 33 emits or generates an image plane 190, which allows image data to be collected on the display device 84 and an image 127 to be generated or reconstructed. However, as Figure 4As shown, device 68 is not within image plane 129. Nevertheless, EM tracking device 66 is able to track device 68 within subject 30. Therefore, the tracked pose and determined pose of the device can be displayed as a graphical representation 68i. As discussed above, EM locator 94 can generate a field, thereby allowing tracking of EM tracking device 66. The shared coordinate space generated by visual locator 93 for tracking US probe 33 with visual tracking device 180 and EM locator 94 with visual tracking device 170 allows the correlation between the tracked pose of device 68 and image 127i generated using image data from US probe 33 to be displayed on display device 84.

[0066] As disclosed above, navigation system 26 (e.g., via tracking of a tracking device) can track or determine the pose of individual parts (e.g., the US probe 33 generating image plane 129, and the instrument 68) to allow the generation of a graphic representation 68i associated with the displayed image (e.g., image 127i). The graphic representation can be superimposed on the image when the instrument is substantially above or in the same area as the image portion. However, as... Figure 4 As shown, the graphic representation 68i can be displayed at a distance from the image, which represents the distance of the instrument 68 from the portion imaged by the US probe 33 in the image plane 129. Therefore, even if the graphic representation and the image 127i are not in the same space, the graphic representation can be displayed relative to the image and on the display device 84 in combination with the determined pose, so that the graphic representation will be superimposed on the image 127i.

[0067] Steering Reference Figure 5 The EM locator 94 may optionally exclude the visual tracking device 170. However, as disclosed above, the EM locator 94 can typically emit an EM field to allow tracking and navigation of various tracking devices. For example, the US probe 33 may include a tracking device 81, which may be an EM tracking device. The EM tracking device 81 may be associated with the US probe 33 in a suitable manner, such as being fixed to its surface, enclosed within its housing, embedded in its housing or structure, or in any other suitable manner. Nevertheless, the EM tracking device 81 may be provided for tracking the pose of the US probe 33, and thus also for tracking the pose of the imaging plane 129. The US probe 33 may also include a visual tracking device 180, as discussed above. Thus, at least two tracking devices that can be tracked in two different tracking modalities (e.g., EM and visual) may be associated with the US probe 33. Therefore, both tracking systems can be used to track and determine the pose of the US probe 33 and its associated imaging plane 129.

[0068] Device 68 may also include an EM tracking device 66, as disclosed above. The EM tracking device 66 may be associated with the insert portion 234, which is part of device 68 within subject 30. Therefore, visual tracking system 93 and / or user 72 may not be able to see the insert portion 234 of device 68. Thus, EM tracking system 94 may be used to track both US probe 33 and device 68 via corresponding tracking devices 81, 66.

[0069] However, as disclosed above, various items or parts may generate distorted fields. For example, the US probe 33 may include a housing 250 or any structure (such as an internal structure or support) formed of a conductive material, and the conductive material may have a field induced therein. Therefore, the induced field in the US probe 33 may generate distorted fields (also referred to as interference fields), which may, for example globally (e.g., throughout the entire navigation space), interfere with the accuracy of tracking various EM tracking devices. For example, the visual tracking system 93 may be able to track various visual tracking devices 170, 180 relative to a subject in all areas within the field of view of the visual locator 93. However, due to local distortion, the registration of the EM locator 94 relative to the visual locator 93 may not be accurately used to determine the pose of the US probe 33 in all areas of the field of view of the visual tracking device 93.

[0070] As further discussed herein, when local distortion occurs, it is determined whether the local accuracy and precision of the EM tracking device 81 can be used for local tracking of various components, such as the tracking device 81 associated with the US probe 33 and / or the tracking device 66 associated with the instrument 68. This distortion can be local, allowing for accurate tracking of the tracking device within the local distortion, even if it is not globally accurate (e.g., within the navigation space of the visual tracking device 93).

[0071] Furthermore, the distortion can be determined, and the visual tracking device 180 can be tracked during inspection or reverification using the visual tracking device locator 93. If the visual tracking device 180 is tracked with the visual locator 93, the global pose of the US probe 33 can be reconfirmed and / or verified using at least a second tracking system. Thus, once the visual locator 93 tracks the visual tracking device 180, the registration of the EM locator 94 relative to the visual locator 93 can be reconfirmed. In this case, any distortion or interference field distortion can be compensated for, and the locally tracked pose of the EM tracking device 81 can again be globally accurate. Therefore, including both tracking devices 81 and 180 with the US probe 33 can help overcome or compensate for distortions that may cause distortions in the field sensed by the EM tracking device 81 and / or the EM tracking device 66.

[0072] Various distortions in the EM tracking field can be determined based on various measurement results. For example, the phase shift of the sensed EM field can be used to determine whether a distortion or interference field is occurring. For example, a time-varying field emitted by the EM locator 94 may have a known phase shift when sensed. If the phase shift changes relative to a known or predetermined phase shift, distortion can be determined. Further, the EM tracking device 81 may include multiple tracking parts, such as multiple conductive coils. The conductive coils have a known and predetermined geometry relative to each other. If the sensed relative geometry of the coils within the EM tracking device 81 changes, distortion can be determined. In this case, the determined distortion can be used to estimate navigation errors or error limits, and to assess whether it is useful or necessary to reconfirm the global tracking pose (e.g., using the visual tracking device 93), and / or to perform precise local tracking of the EM tracking devices (e.g., tracking device 81 and / or tracking device 66) for selected or known local areas or volumes, which may also be referred to as tracking constraints.

[0073] Steering Reference Figure 6 The US probe 33 can be associated with both the visual tracking device 180 and the EM tracking device 81. Therefore, both the EM locator 94 and the visual locator 93 can be used to track the US probe 33. Further, the instrument 68 may include an EM tracking device 66, particularly at the insertion portion 234 of the instrument 68. Therefore, as discussed above, the EM locator 94 can be used to track the instrument 68. Also as discussed above, the visual locator 93 can be used to confirm or reconfirm the pose of the US probe 33 in the global space of the navigation space.

[0074] However, in various embodiments, the image plane 129 can also be used to reconfirm or calibrate the tracked pose of the tracking device 66 and / or instrument 68. For example... Figure 6 As shown, display device 84 can display image 127i generated from image data from image plane 129. As discussed above, image 127i can be a reconstruction result, a model, direct image data, etc. In various embodiments, image 127i can be a reconstruction result based on collected image data, such as a 3D model. In any case, image 127i can be generated based on image plane 129.

[0075] As discussed above, the graphic representation 68i of the instrument 68 can also be displayed on the display device 84. When the instrument 68 is in the image plane or adjacent to or superimposed on the imaged portion, the graphic representation 68i can be displayed and / or superimposed on a portion of the image 127i.

[0076] As shown on display device 84, if instrument 68 is within image plane 129, an image 68x of instrument 68 can also be displayed. If image 68x of instrument is displayed, the position of instrument 68 relative to US probe 33 can be determined based on image data collected in image plane 129 using US probe 33.

[0077] According to various embodiments, the tracked pose of the US probe 33 may be undistorted, while the tracked pose of the instrument 68 may be distorted, especially if distorted relative to the EM tracker 66. However, due to the image plane 129, the image portion 68x of the instrument can also be displayed together with the image 127i. As discussed above, the image plane 129 can be registered or calibrated relative to the US probe 33. Therefore, the pose of the instrument 68 can be determined based on the image 68x of the instrument 68. For example, the distance 260 can be a value, such as a transformation value including translation, rotation, or other differences. The transformation value is the value between the image position 68x and the tracked pose of the instrument 68 (shown by the graphical representation position 68i of the instrument 68). The position of the graphical representation 68i can be based on tracking by the tracking device 66. However, if the two positions do not overlap, an error can be determined. The pose of the instrument in the image at image 68x can be determined according to an appropriate system based on image segmentation to identify the image portion 68x of the instrument 68. The transformation value 260 can be used to correct or update the global pose of the tracking device 66 associated with the instrument 68. As disclosed above, the transformation distance 260 can be a translation, rotation, or other value.

[0078] Therefore, the image pose of the instrument 68 collected in the image plane 129 can be determined and viewed using the display device 84. The image pose of the image portion 68x can be based on the segmentation disclosed above, such as identifying portions of the image that match the known shape of the instrument 68. The selected processor system can execute instructions to determine the transformation value 260. After determining this value, the navigation system 26 can update the EM locator 94 and the associated tracking system to correct or determine the global pose of the EM tracking device 66.

[0079] According to various embodiments, the tracked pose of the device 68 may be undistorted, while the tracked pose of the US probe 33 may be distorted, especially when distorted relative to the EM tracker 81. However, due to the image plane 129, the imaging portion 68x of the device can also be displayed together with the image 127i. Here, the system can determine the transformation value 260 and can correct or determine the global pose of the EM tracking device 81.

[0080] Therefore, the image data collected with the US probe 33 can also be used to update or confirm the global pose of one or more tracking devices in the tracking apparatus. For example, as discussed above, the instrument 68 can be imaged using the US probe 33 and image plane 129. The US probe 33 can be tracked using a visual tracking device 180 and / or an EM tracking device 81. Therefore, the pose of an image 68x of the instrument 68 within image plane 129 can be displayed, which is the actual image data collected with the US probe 33. The image 68x of the instrument can be determined, for example, by segmenting the image data acquired with the US probe 33 or any suitable imaging system. Thus, the pose of the image portion 68x can be determined within the image data and relative to the imaging system (such as the US probe 33), as discussed above (e.g., by calibrating image plane 129 relative to the US probe 33 and / or the tracking device associated therewith). This can be displayed relative to the tracked pose of the instrument as displayed or represented by graphical representation 68i. The difference can be a transformation value 260.

[0081] According to various embodiments, the transformation value is the pose difference between the image pose 68x of the instrument and the tracked pose 68i of the instrument. The navigation system can compare the transformation value 260 with an estimated navigation error or error limit and associated navigation constraints. The system can then notify the user of the difference and / or a registration correction request. Further, the value 260 can be used to check whether the image plane is placed in the correct position. Alternatively or additionally, the value 260 can be used to add to the transformation of the image plane to correct its pose. A maximum value or threshold can be set for the transformation 260 to improve accuracy.

[0082] start

[0083] Steering Reference Figure 7 As discussed above, one or more tracking systems (including EM locator 94 and visual locator 93) may be located in or within a surgical room. Further, the US probe 33 may include one or more tracking devices, such as visual tracking device 180 and EM tracking device 81. Further, the instrument 68 may be used to assist in performing surgery on the subject 30. Instrument 68 may include an EM tracking device 66 located on the insertion portion 234 inserted into the patient 30. Therefore, precise tracking of the instrument can be performed using the EM locator 94. However, as disclosed above, various distortions may occur.

[0084] In addition to the tracking devices associated with various instruments (such as US probe 33 and / or instrument 68), a tracking component 262 (also referred to as DRF) can be associated with subject 30. Similar to DRF 58 discussed above, DRF 262 can be associated with subject 30 in a selected manner. For example, DRF 262 can be attached to subject 30 and a selected location, such as near the heart 127. Thus, portions of subject 30 can be tracked by one or more tracking systems in a tracking system that are close to or adjacent to the location where image data is collected using US probe 33.

[0085] DRF 262 may include one or more tracking devices. For example, DRF 262 may include a visual tracking device 264 and an EM tracking device 268. It should be understood that only one of these two tracking devices, or any suitable number of tracking devices, may be associated with DRF 262. However, including two or more tracking devices allows DRF 262 to be tracked by two or more tracking systems. According to various embodiments, when at least two tracking devices 264, 268 are associated with a single DRF 262, tracking by two or more tracking systems can be performed substantially simultaneously.

[0086] Furthermore, including two or more tracking devices 264, 268 together with a single DRF 262 can bring various advantages and efficiency improvements. The two tracking devices 264, 268 may be in a known geometry relative to each other and / or relative to a point defined by the DRF 262. For example, the DRF 262 may include an indentation or registration mark or point 272. In various embodiments, the device 68 can be registered within the navigation space of the EM locator 94 and the visual locator 93 by associating or touching a portion of the device 68 to the registration point 272, which is in a known geometry relative to the two tracking devices 264, 268.

[0087] Furthermore, a known or selected single point relative to DRF 262 can be used as a registration point to associate or register the navigation space of EM locator 94 with the navigation space of visual locator 93. Registration can be a single registration or updated at appropriate times. For example, visual locator 93 can move or be moved, and the view of visual locator 93 on visual tracking device 264 can be used to confirm or determine the pose of DRF 262. The known pose of EM tracking device 268 relative to visual tracking device 264 can be known (e.g., predetermined and stored in a memory system invoked and / or measured by user 72) to allow the determination of common tracking points or poses in the navigation space of EM locator 94. Therefore, registration between two or more tracking systems in navigation system 26 can be performed using DRF 262, which includes two or more tracking devices.

[0088] Furthermore, the DRF 262 associated with subject 30 can be used to assist in determining or confirming the pose of US probe 33 and / or instrument 68 relative to known points on subject 30. According to various embodiments, DRF 262 can be positioned in a known or predetermined pose. For example, DRF 262 can be positioned at known and definable or identifiable features, such as anatomical features including the suprasternal notch, xiphoid process, or other selected points. Known point 276 can be used to assist in calibration and / or association of various spaces. For example, US probe 33 can be moved to image a predetermined or known point 276 within subject 30. The location of point 276 can be viewed in an image (e.g., image 280) on a display device. Selected portions 276 can be identified in image 280, such as 276i in image 280. Therefore, the tracked pose of US probe 33 (e.g., acquired using visual tracking device 180 and / or EM tracking device 81) can be calibrated relative to the imaging pose of selected portions 276 of subject 30 visible or identifiable in image 280.

[0089] Therefore, as discussed above, various tracking devices (including individual components) that are interconnected can be used to assist in the association of various navigation spaces. Furthermore, different tracking devices can include different features and / or advantages relative to each other to achieve confirmation, accuracy, update speed, etc. Thus, as further discussed herein, various tracking devices can be tracked using two or more tracking systems, allowing surgery to be performed on subject 30 with selected confidence levels and / or selected efficiencies, including precise and accurate navigation of instruments (including two or more instruments) during selected surgical procedures.

[0090] As discussed above, various systems can be used to track different items or parts. According to various embodiments, the visual tracking device system 93 can be used to track one or more items, and the EM locator 94 can be used to track one or more items. According to various embodiments, a single item (such as the US probe 33) may include tracking devices 180, 81 used respectively with the two tracking systems.

[0091] However, it should be further understood that other tracking systems may also be used, such as optical tracking systems (which can use cameras to track one or more objects that can be identified by a selected wavelength (e.g., infrared (IR)), inertial tracking systems, or other suitable tracking systems. Various systems can be used to allow for the correction and / or determination of the accuracy or reliable pose of one or more items.

[0092] The selected configuration allows for registration between multiple tracking systems. As discussed above, various systems and their corresponding operating modes and / or configurations can be used to track and determine the pose of a selected object or article. The selected object or article may include one or more instruments, including instrument 68 and / or US probe 33. The discussion of object or article herein is to be understood as referring to any suitable tracked portion. According to various embodiments, the specific tracked object or article need not be a specific article.

[0093] The process 300 for navigating objects Figure 8 The image is displayed and can be used to evaluate the output to determine the pose of one or more items. First, refer to... Figure 8 Process 300 may begin at start box 310. Then, process 300 may receive first tracking data from a first tracking system at box 314 and second tracking data from a second tracking system at box 318. One or more of the received tracking data may be, for example, first instance tracking data at a first time. As discussed herein, process 300 may be iterative, for example, the received tracking data 314, 318 may become second instances, such as when compared to the first instance. Therefore, tracking data may be collected over time (e.g., the time of surgery) and / or over movement (e.g., movement of the tracked object). As further discussed herein, various systems may be used to evaluate selected tracking data and will be described in more detail herein.

[0094] The two received tracking data can come from two tracking systems operating in different or the same modes but separated from each other. The first received tracking data can be any suitable first tracking data, such as tracking data received from visual locator 93 and optical locator 88, or other suitable tracking systems. According to various embodiments, the first tracking system may also include an EM tracking system or an inertial tracking system. The second tracking data received in block 318 can be any suitable tracking data, such as EM tracking data, inertial tracking data, etc. Further, according to various embodiments, the second tracking data may also include or only include visual tracking data, etc.

[0095] Nevertheless, process 300 may include receiving or acquiring at least first tracking data and second tracking data, which may come from different tracking systems or from two tracking systems. According to various embodiments, the two tracking systems may be the same tracking system and / or alternative tracking systems. In particular, when using alternative tracking systems, each of the tracking systems can be determined, verified, or weighted relative to each other. This can aid in determining the reliability or accuracy of one or more of the received tracking data.

[0096] As discussed above, each type of tracking data or tracking system can have advantages and / or disadvantages relative to a chosen application. For example, a visual or optical tracking system may require an unobstructed line of sight, which can be obstructed when the user, subject 30, object in the operating room, or other opaque objects may be moved into the area and / or the selected area may not be bright enough to be accurately viewed by one or more tracking systems in the tracking system. Furthermore, an EM tracking system may have distortions caused by interfering objects or interference fields relative to the tracked object. Interfering fields can be identified as disclosed above, such as due to changes in the tracked geometry of a tracking device having two or more tracking parts and / or determined or identified phase shifts. Therefore, the EM locator 94 may be unreliable or inaccurate for a selected area or volume or outside a selected (e.g., local) area or volume. An inertial tracking system may be accurate or precise within a selected time period but may drift over time. For example, an inertial tracking system can be globally accurate and precise for a selected time period (from a fraction of a second to several minutes) after registration. This could be approximately 5 seconds, 10 seconds, 30 seconds, or any appropriate time within the selected range, such as approximately 1 second to approximately 10 seconds, including various values ​​within that range. Therefore, comparing the first tracking data from box 314 with the second tracking data from box 318 can be used to determine the confidence or accuracy of the selected pose.

[0097] Therefore, the accuracy or reliability of the first tracking data can be determined in box 322. Similarly, the accuracy or reliability of the second tracking data can be determined in box 326. The determination of the accuracy or reliability of the first and second tracking data in the corresponding boxes 322 and 326 can be based on the corresponding tracking system. For example, the determination of gaze occlusion in a visual tracking system can be used to determine the accuracy or reliability of the first tracking data in box 322.

[0098] Determining line-of-sight occlusion can include the rate at which the view or determination of the pose of the tracked visual tracking device (such as visual tracking device 180) is updated. For example, the tracking system may understand or choose to update the view or determination of the pose of visual tracking device 180 at a selected rate (such as 10 times per second, 100 times per second, or any suitable rate). Accuracy can be determined based on the update rate, such as whether a certain number of updates have been performed recently, or whether updates were not performed due to the tracking device being unviewable.

[0099] Further determination of accuracy and / or whether the tracking device is not in the field of view can include various parameters, including those mentioned above and herein. According to various examples, visual pattern matching accuracy can vary with the number of matched pattern corner points; therefore, partially occluded patterns can still be identified, but accuracy may be reduced. The visual pattern can be any suitable pattern, such as a checkerboard design. According to various examples, visual pattern matching accuracy can vary with the pattern's viewing angle; therefore, larger viewing angles (e.g., greater than a selected threshold, such as greater than approximately 30 degrees) may reduce accuracy. According to various examples, optical tracker matching accuracy can vary with the number of matched markers; therefore, accuracy may be reduced when a selected number in a set or predetermined number is completely or partially occluded. According to various examples, optical marker matching accuracy can vary with the shape of the matched markers; therefore, partially occluded markers may reduce accuracy.

[0100] The determination of the accuracy or reliability of the second tracking data can be performed in a similar and / or different manner. For example, as disclosed above, in an EM tracking system, if it is determined that the phase shift is greater than a selected amount, the accuracy or reliability can be determined based on that phase shift. Furthermore, distortions of the known geometry of the EM tracking device can also be used. According to various embodiments, the accuracy of EM sensor signal matching can vary with distortion, and therefore distortions that cause signal errors may reduce accuracy.

[0101] Inertial tracking systems can have known or determinable drift. Therefore, the inaccuracy of an inertial tracking system can be based on any time elapsed since the last confirmation or registration of the inertial tracking system.

[0102] After the accuracy or reliability of both the first and second tracking data have been determined in the corresponding boxes 322 and 326, it can be determined in box 330 whether the accuracy or reliability of both the first and second tracking data is greater than a threshold. If it is determined that either or both of the accuracy or reliability is below a selected threshold, the process can proceed along path 334 to process box 338, as discussed further herein. The threshold accuracy or reliability can be any suitable factor or constraint. For example, whether the accuracy is within 0.1 mm, 1 mm, 2 mm, or other selected distances. The threshold can also be the time elapsed since calibration or verification. The threshold can be based on a selected desired or expected accuracy.

[0103] If the accuracy is determined to be greater than the selected threshold, the path 350 can be continued. The threshold accuracy can be determined relative to the corresponding tracking system, the robustness of different tracking devices, etc. For example, the reliability of a visual tracking system can be based on the number of updates to the tracked pose of the corresponding tracking device (such as visual tracking device 180) and the most recent update. The accuracy or threshold of an EM tracking system can be determined based on unexpected phase deviations or changes and / or determined geometry.

[0104] Nevertheless, a threshold can be compared to the determined accuracy, and if the determined accuracy is greater than the threshold, the process can proceed along ds path 350. Thereafter, the weights of both the first and second tracking data can be invoked in box 354. Invoking the weights in box 354 is optional, as is applying the weights in box 356. According to various embodiments, the weight of each of the first and second tracking data received from boxes 314 and 318 can vary between 0% and 100%, and within any portion thereof. For example, the weight of the first tracking data could be 10%, and the weight of the second tracking data could be 90%. The weights can also include assigning a weight of 0% to either the first or second tracking data, and assigning a weight of 100% to the other of the first and second tracking data.

[0105] The determined weights can vary based on the accuracy determined in boxes 322 and 326, and / or based on the determined accuracy. For example, the higher the determined accuracy of the second tracking data, the higher the weight applied to it. Similarly, the higher the determined accuracy or confidence of the first tracking data, the higher the weight applied to it. According to various embodiments, standard or nominal weights can be applied to the first and second tracking data, such as 50% each, 40% for the first tracking data and 60% for the second tracking data, 60% for the first tracking data and 40% for the second tracking data, or any suitable nominal weight. The invoked weights can be based on possible accuracy and confidence, and are predetermined based on possible accuracy and confidence. Therefore, the invoked weights can be stored in a lookup table and invoked based on the accuracy and confidence of the corresponding first and second tracking data determined in boxes 322, 326.

[0106] After the weights are determined, they can optionally be applied to box 356, as discussed above. Therefore, once the weights are determined in box 354, they can be applied in 356. As discussed above, the weights may include assigning zero weight to at least one of the first or second tracking data and / or may include a 50 / 50 weight. Therefore, the pose of the tracked object can be determined in box 360 regardless of whether the weights are invoked and applied.

[0107] The pose of the tracked object can be determined based on first and second tracking data received from blocks 314, 318 and / or only on one of them. For example, the system can determine that the confidence or accuracy of either or both of the first or second tracking data is greater than a selected threshold, and use only one of the two tracking systems. However, according to various embodiments, weights can be applied to allow the pose to be determined always based on a combination of tracking data from the first and second tracking data. Thus, the pose of the tracked object can be determined in block 360 based on the first tracking data received in blocks 314, 318, as discussed above.

[0108] The determined pose can then be output in box 364. The output may include display content, instructions for moving the robotic arm 20, instructions for the user to move the instrument 68, or any other suitable output. The output of box 364 may be based on the pose of the tracked object determined in box 360. According to various embodiments, the determined pose may be displayed as a graphical representation 68i on a display device 84. The graphical representation may show the tracked pose of the instrument 68 for viewing by the user 72. According to various embodiments, the determined pose of the object may also be used to control the movement of the robotic arm. Controlling the movement may include providing output to the robotic arm system processor, as discussed above, to move the robotic arm end effector 44 to a selected pose based on the current tracked pose.

[0109] After outputting the determined pose in box 364, process 300 can then receive input or determine whether the surgery has ended in box 370. If the surgery has not ended, it can proceed along path 374 to restart process 300. Therefore, process 300 can be iterative to continuously update the tracked pose or the tracked pose data of the receiving object (e.g., instrument 68).

[0110] If it is determined in box 370 that the procedure has ended, then the path 380 can be advanced to terminate procedure 300 in box 384. Therefore, in box 384, procedure 300 can stop tracking or stop receiving information about the tracked pose of the instrument or object.

[0111] Therefore, the tracking pose of an object (e.g., device 68) can be determined by receiving tracking data from at least two tracking systems in boxes 314 and 318 respectively, and the determined pose of the object can be output in box 364, as discussed above. Thus, the tracked device can be based on tracking data from more than one tracking system, as discussed above.

[0112] As discussed above, process 300 can be used to output the determined pose of the object. However, according to various embodiments, as discussed above, if the accuracy or reliability of the first and second tracking data in a global sense can be determined in block 330, and if the accuracy does not reach a threshold, the process proceeds to block 338 along the negative path.

[0113] Box 338 can be Figure 9 This is part of process 400. Process 400 may allow tracking of an object (e.g., instrument 68) when it is determined that the globally significant accuracy is below a selected threshold. Typically, globally significant accuracy may be determined to be insufficient to determine that the tracked pose of the object in the entire navigation space is inaccurate enough to perform the selected procedure. However, accuracy may be determined to be adequate, at least within selected constraints, as discussed further herein.

[0114] According to process 400, it can be determined in box 338 whether both the first tracking data and the second tracking data are below a threshold. If it is determined that neither is below the threshold, the process can proceed along the negative path 410 to reach the box in box 414 where it is determined whether the first tracking data is below the threshold.

[0115] If it is determined that the first tracking data is not lower than a threshold, the path 418 can be advanced to correct or update the second tracking data with the first tracking data in block 422. When correcting or updating the second tracking data with the first tracking data in block 422, the pose of the object determined using the first tracking data can be used to define a point or pose within the navigation space of the second tracking data or the second tracking navigation space. The use of accurate or reliable first tracking data can be used to update or define the current pose of the tracked object in the second tracking navigation space, allowing continued tracking of the object using the second tracking data based on the updated or corrected pose. Therefore, once the second tracking data is updated with the first tracking data in block 422, both the first and second tracking systems can continue to be used. According to various embodiments, the second tracking data is corrected or updated with the first tracking data in block 422 only if the first tracking data is not lower than a threshold (e.g., to generate or output updated second tracking data). In other words, the first tracking data has already met the threshold for updating or correcting the second tracking data.

[0116] Furthermore, the corrected and / or updated second tracking data can be used as input or to restart process 300. Therefore, correcting or updating the second tracking data with the first tracking data allows continued reception of both the first and second tracking data in blocks 314, 318, as discussed above. Thus, once the second tracking data has been corrected or updated with the first tracking data, process 400 allows continued tracking of the object using both the first and second tracking systems.

[0117] If, according to boxes 338 and 414, both the first and second tracking data are determined to be not at (e.g., below) a threshold and / or the first tracking data is below a threshold, then path 440 can be continued. While continuing along path 440, the previous accurate or reliable first tracking data can be recalled in box 444. The previous accurate or reliable tracking data from the first tracking system can be the previous tracking pose of a visual tracking device (e.g., visual tracking device 180) that satisfies a selected update rate or other parameters. Other parameters may include pattern matching metrics, which may include fitting errors expressed as a percentage, length, or area. The recall of the previous accurate or reliable first tracking data can be initiated from a temporary storage device for the first tracking data, the positioning of the tracked object in a selected location, etc.

[0118] Regardless of the process, in box 448, the previously invoked accurate or reliable first tracking data can be set as the reference point or origin of the second tracking system. Setting the previously invoked accurate or reliable first tracking data as the reference point or origin can include identifying a point in the navigation space relative to the tracked portion (such as subject 30, DRF 262, etc.). For example, the previously invoked accurate or reliable first tracking data could be a point within the subject's body, such as at a portion of the heart 127. The identified point or portion of the subject can be identified as or used as a reference point, and the object can be repositioned to the previously invoked accurate or reliable point. This allows the tracked object (such as device 68) to have an accurate or precise local origin or initial point.

[0119] As discussed above, EM tracking systems may exhibit distortion for various reasons. Nevertheless, the distorted EM field can be used for precise and accurate tracking under selected constraints (such as within a confined volume). Furthermore, as discussed above, inertial tracking systems may exhibit drift over a selected time period. However, within the constrained time period, the local movement or pose of the inertially tracked object can be determined precisely and accurately. Therefore, the identification of the origin or reference point in box 448 allows for the definition of tracking constraints within the second tracking system in box 452.

[0120] Regarding the above, the device 68, including the internal portion 234 within the subject 30, can be tracked using only the EM tracking locator 94. Therefore, if distortion occurs and the EM tracking device 66 is unreliable, navigation may need to be stopped. However, if local points can be determined, such as by setting local points in box 448, constraints for tracking using the second tracking system can be determined in box 452. Constraints may include selected areas or volumes or maximum movement from a set reference point. For example, constraints may include allowing tracking only within a range of 1 cm, 2 cm, 5 cm, 10 cm, or any value from the reference point.

[0121] Constraints can be defined in box 452. Constraints can be based on sensed distortions, parameters invoked according to the type of sensed distortion, or the distortion itself. For various examples, constraints can depend on distortions within a volume around a reference point; a larger area around the reference point is allowed if the distortion is not above a selected threshold (e.g., not too high), or the user may be able to set these constraints if tracking near the reference point is acceptable for the user's application. For various examples, time constraints can be determined by integrating the characterized accelerometer uncertainty over time to achieve a specified distance constraint. For various examples, pose constraints can be determined by correlating the measured geometric error and its pose derivative with previously characterized pose errors and specified distance or angle constraints.

[0122] After the constraints are defined in box 452, the object can be tracked using the second tracking system in box 456. As disclosed above, device 68 can be tracked using only the EM tracking system, and therefore can even be partially tracked using only the second tracking system according to process 400. It can be determined in box 416 whether tracking continues within the constraints. As discussed above, constraints may include time, volume of movement, one or both of these, or other suitable constraints. If it is determined that the object is within the constraints, such as within the maximum distance of movement, then it is determined that tracking remains within the constraints, and the path 464 can be continued to track the object using the second tracking system.

[0123] If it is determined that tracking is outside the constraints, in other words, has exceeded the constraints (e.g., in time or distance), then progress can be made along the negative path 470. For example, as disclosed above, it can be determined that the object has moved a distance greater than the constraint (e.g., greater than 10 cm), and therefore progress can be made along the negative path 470 to stop tracking in box 474. Upon stopping tracking, the device's pose can be removed from the display 84, a visual warning can be provided to the user, an auditory warning can be provided to the user, tactile or touch feedback can be provided to the user, or any appropriate feedback and / or combination of feedback can be provided. According to various embodiments, for example, a graphical representation of the device can be removed from the display device 84. Furthermore, the tracking system may no longer provide instructions for moving the selected device, such as maintaining or holding the robotic arm 40 in a selected pose, and may not allow or execute movement instructions.

[0124] After stopping tracking in box 474, it can be determined in box 478 whether tracking was selected. If tracking was not selected, you can proceed along path 480 to end the procedure in end box 484. When the procedure 400 ends in box 480, navigation can be ended, and the procedure can be performed or completed without tracking and / or the procedure can be terminated.

[0125] If tracking is selected in box 478, you can proceed along path 488 to reinitialize tracking in box 492 in some appropriate manner, such as instructing the user to move the instrument, move the locator, reset or reinitialize the tracking system, re-register the tracking system, or other appropriate instructions or movements. However, restarting tracking may require further input from the user and / or the system to maintain or continue tracking. Nevertheless, once reinitialized, the process can also be reset or reinitialized using imports or reinitializations from box 492, such as... Figure 8 The process shown is 300.

[0126] Therefore, according to various embodiments, navigation system 26 can be used to navigate or track an object using two or more tracking systems to allow for continuous and accurate or reliable tracking of the object. According to the processes and systems discussed above, using two or more tracking systems, the pose of the object can be determined, and the tracked procedure can continue or be maintained even if one or more tracking systems are determined to be unreliable. Further, as disclosed above, processes (such as processes or methods 300, 400) can be executed substantially automatically using a processor system. Therefore, processes 300, 400 can be used and / or accessed by navigation system 26 and its various processors to allow navigation or tracking of a selected object.

[0127] Example embodiments are provided to make this disclosure thorough and to fully convey the scope to those skilled in the art. Numerous specific details, such as examples of specific components, devices, and methods, are set forth to provide a thorough understanding of embodiments of this disclosure. It will be apparent to those skilled in the art that the example embodiments may be implemented in many different forms without requiring the specific details, and that neither these specific details nor the example embodiments should be construed as limiting the scope of this disclosure. In some example embodiments, well-known processes, well-known device structures, and well-known techniques are not described in detail.

[0128] Instructions can be executed by a processor and can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuit" includes a single processor circuit that executes some or all of the code from multiple modules. The term "group processor circuit" includes processor circuitry combined with additional processor circuitry to execute some or all of the code from one or more modules. References to multiple processor circuitry include multiple processor circuitry on a discrete die, multiple processor circuitry on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" includes a single memory circuitry that stores some or all of the code from multiple modules. The term "group memory circuit" includes memory circuitry combined with additional memory to store some or all of the code from one or more modules.

[0129] The apparatus and methods described in this application may be implemented, in part or in whole, by a processor (also referred to as a processor module) to perform one or more specific functions embodied in a computer program, which may include a special-purpose computer (i.e., created by configuring a processor) and / or a general-purpose computer. These computer programs include processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. The computer program may also include or depend on stored data. These computer programs may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, and applications, etc.

[0130] These computer programs may include: (i) assembly code; (ii) object code generated from source code by a compiler; (iii) source code for execution by an interpreter; (iv) source code for compilation and execution by a just-in-time (JIT) compiler; and (v) descriptive text for parsing, such as HTML (Hypertext Markup Language) or XML (Extensible Markup Language). As an example only, source code may be in C, C++, C#, Objective-C, Haskell, Go, SQL, Lisp, Java®, ASP, Perl, Javascript®, HTML5, Ada, ASP (Dynamic Server Pages), Perl, Scala, Erlang, Ruby, Flash®, Visual Basic®, Lua, or Python®.

[0131] The communications may include the wireless communications described in this disclosure, which may be wholly or partially compliant with IEEE Standard 802.11-2012, IEEE Standard 802.16-2009, and / or IEEE Standard 802.20-2008. In various implementations, IEEE 802.11-2012 may be supplemented by draft IEEE Standard 802.11ac, draft IEEE Standard 802.11ad, and / or draft IEEE Standard 802.11ah.

[0132] The terms processor, processor module, module, or “controller” are used interchangeably herein (unless expressly disclosed otherwise) and each may be replaced by the term “circuit”. Any of these terms may refer to, be a part of, or include: application-specific integrated circuit (ASIC); digital, analog, or mixed-signal analog / digital discrete circuit; digital, analog, or mixed-signal analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or grouped) that executes code; memory circuitry (shared, dedicated, or grouped) that stores code executed by the processor circuitry; other suitable hardware components that provide the described functionality; or some or all of the foregoing, such as in a system-on-a-chip.

[0133] Instructions can be executed by one or more processors or processor modules, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuit systems. Therefore, the terms "processor" or "processor module" as used herein can refer to any of the foregoing structures or any other physical structures suitable for implementing the described techniques. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements. One or more processors can operate fully automatically and / or substantially automatically. In automatic operation, the processor can execute instructions based on received inputs and execute instructions according to received inputs. Therefore, various outputs can be made without further or any manual (e.g., user) input.

[0134] For illustrative and descriptive purposes, the foregoing description of embodiments has been provided. This description is not intended to be exhaustive or limiting of the invention. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but are interchangeable where applicable and can be used in selected embodiments, even if not specifically shown or described. It can also be varied in many ways. Such variations should not be considered as departing from the invention, and all such modifications are intended to be included within the scope of the invention.

Claims

1. A system for assisting in surgical navigation, the system comprising: A first tracking system, the first tracking system having a first tracking locator configured to operate in a first mode; A second tracking system having a second tracking locator configured to operate in a second mode; A navigation processor configured to execute instructions to perform the following operations: Receive first instance first tracking data about the pose of an object in the navigation space of the first tracking system from the first tracking system, wherein the first instance first tracking data is related to a first time. Receive first instance second tracking data about the pose of the object in the navigation space of the second tracking system, wherein the first instance second tracking data is related to the first time. Register the first tracking system navigation space of the first tracking system and the second tracking system navigation space of the second tracking system; Evaluate the received first instance first tracking data and first instance second tracking data. The pose of the object in physical space is determined based on the evaluated first instance first tracking data and first instance second tracking data. Output the determined pose of the object.

2. The system of claim 1, further comprising: Display devices; The display device is configured to display a graphical representation of the object based on the output and determined pose of the object.

3. The system as described in claim 2, wherein, The graphic representation is superimposed on at least one of the subject's model or the subject's image.

4. The system of claim 3, further comprising: Ultrasound (US) imaging system; The object includes the ultrasound probe of the ultrasound imaging system.

5. The system as described in claim 4, wherein, The object further includes instruments; The device pose is determined relative to the US probe pose of the ultrasound probe.

6. The system of claim 5, wherein, Image data is used to generate an image for display on the display device, the image data being collected in an image plane generated using the US probe; The graphical representation is displayed based on the determined relative pose of the US probe and the instrument relative to the image.

7. The system of claim 1, further comprising: The first tracking device associated with the second tracking locator, The first tracking device is configured to be tracked by the first tracking system; The registration of the first navigation space of the first tracking system and the second navigation space of the second tracking system is based on: determining the second tracking locator pose of the second tracking locator using the first tracking system within the first tracking system navigation space and transforming the first tracking system navigation space into the second tracking system navigation space.

8. The system as claimed in claim 1, wherein, The first tracking system includes a first tracking device; The second tracking system includes a second tracking device; The first tracking device and the second tracking device are associated with the object; The registration of the first navigation space of the first tracking system and the second navigation space of the second tracking system is based on the tracking of the object; The object in question is a single object.

9. The system of claim 8, wherein, The object is an ultrasound probe configured to acquire image data of the subject.

10. A method for assisting in surgical navigation, the method comprising: The first tracking locator is operated in the first mode of the first tracking system; The second tracking locator is operated in the second mode of the second tracking system; First instance first tracking data about the pose of an object in the navigation space of the first tracking system is obtained from the first tracking system, wherein the first instance first tracking data is related to a first time. Obtain first instance second tracking data about the pose of the object in the navigation space of the second tracking system from the second tracking system, wherein the first instance second tracking data is related to the first time; Register the first navigation space of the first tracking system and the second navigation space of the second tracking system; The pose of the object in physical space is determined based on the first instance first tracking data and the first instance second tracking data obtained. Output the determined pose of the object.

11. The method of claim 10, further comprising: The navigation processor is operated to execute instructions for performing the following operations: at least registering the first navigation space and the second navigation space, determining the pose of the object, and outputting the determined pose of the object.

12. The method of claim 10, further comprising: The navigation processor is operated to execute instructions for performing all of the following operations: acquiring first tracking data of the first instance from the first tracking system, acquiring second tracking data of the first instance from the second tracking system, registering the first navigation space and the second navigation space, determining the pose of the object, and outputting the determined pose of the object.

13. The method of claim 10, further comprising: The operation of the display device is to display a graphical representation of the object based on the output and determined pose of the object.

14. The method of claim 10, further comprising: Use an ultrasound (US) imaging system to acquire image data; The object includes the US probe of the US imaging system.

15. The method of claim 14, further comprising: The object is provided as the US probe and as a separate instrument from the US probe; Determining the pose of the object includes determining both the instrument pose and the US probe pose.

16. The method of claim 13, further comprising: Collect image data in the image plane generated by the US probe; An image for display on the display device is generated based on the collected image data; The graphical representation is displayed based on the determined pose of the US probe and the instrument relative to the image.

17. The method of claim 16, further comprising overlaying the graphic representation onto at least one of a model of the subject or an image of the subject.

18. The method of claim 10, further comprising: Provide a first tracking device associated with the second tracking locator. Provide the first tracking device to be tracked by the first tracking system; The registration of the first navigation space of the first tracking system and the second navigation space of the second tracking system includes: In the navigation space of the first tracking system, the second tracking locator pose of the second tracking locator is determined using the first tracking system, and The navigation space of the first tracking system is transformed into the navigation space of the second tracking system.

19. The method of claim 10, further comprising: Use the first tracking system to track the first tracking device; Use the second tracking system to track the second tracking device; The object is provided as a single object; Associate the first tracking device and the second tracking device with the object; Track both the first tracking device and the second tracking device to register the first navigation space of the first tracking system and the second navigation space of the second tracking system.

20. The method of claim 19, wherein the object is provided as an ultrasound probe configured to acquire image data of the subject.

Images