Systems and methods for surgical instruments navigation using markers

Abstract

The presently disclosed subject matter pertains to a navigation system and method that integrates advanced imaging techniques with marker-based modeling to enhance precision in medical procedures. The system and method capture tissue- penetrating images of a target area in the human body, incorporating distinct markers detectable by imaging modalities such as X-ray, MRI, CT, or ultrasound. Strategically placed markers facilitate the creation of a detailed 3D model, enabling precise real- time tool tracking. This innovative approach enhances surgical navigation and procedural accuracy by combining multi-modality imaging with marker-based 3D modeling.

Description

SYSTEMS AND METHODS FOR SURGICAL INSTRUMENTS NAVIGATIONUSING MARKERSTECHNICAL FIELD

[0001] The present invention generally relates to systems and methods for assisting surgeons in navigation of a surgical instrument during a surgical procedure and more particularly to systems and methods for navigation of a surgical instrument using markers.BACKGROUND

[0002] Medical imaging is a wide term relating to any medical diagnostic, therapeutical, surgical and / or prognostic system or process that uses one or more imaging systems or devices that produce 2D or 3D imagery information of internal and / or external body parts, such as magnetic resonance imaging (MRI) systems, computerized tomography (CT) systems, ultrasound-based systems, X-ray systems / devices, isotopic tomography systems, endoscopic systems, etc.

[0003] Imagery systems are typically configured to produce one or more images indicative of anatomical properties of organs, body parts, body systems and / or physiological characteristics thereof.

[0004] When conducting a surgical procedure, involving intrusively penetrating of one or more surgical tools to a body of a patient, e.g., for operating an internal organ / tissue of the patient and / or for performing an intrusive diagnosis / detection / imaging (e.g., using endoscopic systems), the positioning (i.e., location within a 3D coordinate system and orientation) of the edge of the surgical tool being used to perform the operation should be constantly and controllably moved by the surgeon performing the operation, in order to reach a different zone / area / spot / circumference of the inner body part being operated at each given moment and to avoid / reduce erroneous adversities by having the edge of the surgical instrument engaged with inner tissue that is not to be operated, irradiated and / or detected. In many cases, any slight error in the instrument’s positioning may lead to sever and often irreversible outcomes.

[0005] To reduce such impairments, the internal area / organ to be operated / tested is often imaged in real time during the performance of the surgical procedure, using one or more fixed markers attached to the patient’s body or placed externally to the patient. Thesefixed markers are also detectable by the imaging system to identify the positioning of the operated internal organ / tissue / area for identifying position of the operated organ / area / tissue in respect to a fixed coordinate system. The one or more fixed markers allow identifying the position of the patient’s external body part or body at each given moment in respect to the fixed coordinate system and thereby deduce or estimate the position of the operated organ / tissue / area in respect to the surgical tool position. This technique can provide sufficient accuracy when dealing with operated body tissues, organs and / or areas that are likely to always be positioned in the same exact manner relative to the positioning of the larger external body part of the patient and in cases in which the surgical tools being used are not very sensitive to erroneous positioning thereof.

[0006] However, such techniques do not provide a sufficient solution when operating on elastic tissues such as soft tissue and / or when operating on tissues of varying elasticity levels as well as for operations using surgical tools and / or procedures that are highly sensitive to errors in the positioning of the surgical tool or part thereof. Moreover, the above-mentioned marking techniques may also cause inaccurate detection of the position of the surgical tool and / or part thereof, in respect to the actual position of the specific area in the body part being operated, which may lead to catastrophic outcomes such as removing of a wrong section of the tissue or part thereof. For example, in cases of removal of malignant, pre-malignant or benign tumor tissues, requiring the removed tissue volume or area to be larger than the area / volume of the tumor, to reduce chances of tumor cells from being cut off and released into other tissues or into the patient’s blood stream, errors in the positioning of the surgical tool or edge thereof may lead to leaving some of the tumor’s tissue unremoved or cut through, which may lead to fatal outcomes. Still further, the markers themselves could cause discomfort to the patient, and may also interfere with the operations of the surgeon.SUMMARY

[0007] In accordance with a first aspect of the presently disclosed subject matter, there is provided a navigation system, comprising: at least one image acquisition sensor configured to acquire images of an area of interest on a human body; and a processing circuitry configured to: obtain a tissue penetrating image of the area of interest, wherein (a) the area of interest includes a plurality of first markers, (b) each of the first markershas a distinct shape and / or a known size, (c) each of the first markers is visible to one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, (d) each of the first markers is placed in a respective location on the area of interest, and (e) the tissue penetrating image is acquired by at least one of the imaging devices; generate a three- dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the first markers; acquire, utilizing the image acquisition sensor, a second image of the area of interest, wherein: (a) the area of interest includes a tool and a plurality of second markers painted in the respective locations of least a subset of the first markers and replacing at least the subset of the first markers; (b) the second markers being painted in paint visible in a spectrum covered by the image acquisition sensor; and determine a location of the tool within the three-dimensional model by determining a disposition between the location of the plurality of second markers and the locations of the respective first markers.

[0008] In some cases, the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

[0009] In some cases, the first markers are placed in at least two layers with varying distances with respect to the imaging devices.

[0010] In accordance with a second aspect of the presently disclosed subject matter, there is provided a navigation system, comprising: at least one image acquisition sensor configured to acquire images of an area of interest on a human body; and a processing circuitry, configured to: obtain a tissue penetrating image of the area of interest, wherein (a) the area of interest includes a plurality of markers, (b) each of the markers has a distinct shape and / or a known size, (c) each of the markers is painted in a respective location on the area of interest with paint visible to the image acquisition sensor and to one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, and (d) the tissue penetrating image is acquired by at least one of the imaging devices; generate a three- dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the markers; acquire, utilizing the image acquisition sensor, a second image of the area of interest, wherein the area of interest includes a tool and at least a subset of the markers; and determine a location of the tool within the three-dimensionalmodel by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image.

[0011] In some cases, the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

[0012] In some cases, the markers are painted in at least two layers with varying distances with respect to the imaging devices.

[0013] In accordance with a third aspect of the presently disclosed subject matter, there is provided a navigation method, comprising: obtaining, by a processing circuitry, a tissue penetrating image of the area of interest, wherein (a) the area of interest includes a plurality of first markers, (b) each of the first markers has a distinct shape and / or a known size, (c) each of the first markers is visible to one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, (d) each of the first markers is placed in a respective location on the area of interest, and (e) the tissue penetrating image is acquired by at least one of the imaging devices; generating, by the processing circuitry, a three- dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the first markers; acquiring, utilizing an image acquisition sensor configured to acquire images of an area of interest on a human body, a second image of the area of interest, wherein: (a) the area of interest includes a tool and a plurality of second markers painted in the respective locations of least a subset of the first markers and replacing at least the subset of the first markers; (b) the second markers being painted in paint visible in a spectrum covered by the image acquisition sensor; and determining, by the processing circuitry, a location of the tool within the three-dimensional model by determining a disposition between the location of the plurality of second markers and the locations of the respective first markers.

[0014] In some cases, the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

[0015] In some cases, the markers are painted in at least two layers with varying distances with respect to the imaging devices.

[0016] In accordance with a fourth aspect of the presently disclosed subject matter, there is provided a navigation method, comprising: obtaining, by a processing circuitry, a tissue penetrating image of the area of interest, wherein (a) the area of interest includes a plurality of markers, (b) each of the markers has a distinct shape and / or a known size, (c) each of the markers is painted in a respective location on the area of interest with paintvisible to the image acquisition sensor and to one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, and (d) the tissue penetrating image is acquired by at least one of the imaging device; generating, by the processing circuitry, a three- dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the markers; acquiring, utilizing an image acquisition sensor configured to acquire images of an area of interest on a human body, a second image of the area of interest, wherein the area of interest includes a tool and at least a subset of the markers; and determining, by the processing circuitry, a location of the tool within the three- dimensional model by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image.

[0017] In some cases, the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

[0018] In some cases, the first markers are placed in at least two layers with varying distances with respect to the imaging devices.

[0019] In accordance with a fifth aspect of the presently disclosed subject matter, there is provided a navigation system, comprising: at least one image acquisition sensor configured to acquire images of an area of interest on a human body; and a processing circuitry, configured to: obtain a tissue penetrating image of the area of interest, wherein (a) the area of interest at least partially includes a soft tissue being pressed between two panels; (b) the area of interest includes a plurality of markers including at least one first marker placed on one side of the area of interest and at least one second marker placed substantially on an opposite side of the area of interest, (c) the tissue penetrating image is acquired by at least one of one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging; generate a three-dimensional model from the tissue penetrating image, utilizing the location of the markers in the tissue penetrating image, a contour of at least part of the area of interest visible in the tissue penetrating image, and a known distance between the panels; acquire, utilizing the image acquisition sensor, a second image of the area of interest, wherein the area of interest includes a tool and at least a subset of the markers; and determine a location of the tool within the three-dimensional model by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image.

[0020] In some cases, the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

[0021] In accordance with a sixth aspect of the presently disclosed subject matter, there is provided a navigation method, comprising: obtaining, by a processing circuitry, a tissue penetrating image of an area of interest on a human body, wherein (a) the area of interest at least partially includes a soft tissue being pressed between two panels; (b) the area of interest includes a plurality of markers including at least one first marker placed on one side of the area of interest and at least one second marker placed substantially on an opposite side of the area of interest, (c) the tissue penetrating image is acquired by at least one of one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging; generating, by the processing circuitry, a three-dimensional model from the tissue penetrating image, utilizing the location of the markers in the tissue penetrating image, a contour of at least part of the area of interest visible in the tissue penetrating image, and a known distance between the panels; acquiring, utilizing an image acquisition sensor configured to acquire images of the area of interest, a second image of the area of interest, wherein the area of interest includes a tool and at least a subset of the markers; and determining, by the processing circuitry, a location of the tool within the three- dimensional model by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image.

[0022] In some cases, the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

[0023] In accordance with a seventh aspect of the presently disclosed subject matter, there is provided a navigation system, comprising: at least one image acquisition sensor configured to acquire images of an area of interest within a human body; and a processing circuitry, configured to: obtain a three-dimensional model of the area of interest when the area of interest is in a first state; capture a series of images of the area of interest utilizing the image acquisition sensor, wherein: (a) at the time of capturing the series of images the area of interest is in a second state, different than the first state, and (b) the series of images including at least two images in which a known marker attached to a movable object is placed in contact with respective known sites within the area of interest, the known site being included in the three-dimensional model; determine a location of the tool within the three-dimensional model by determining a disposition between thelocations of the known sites in the images and the locations of the known sites in the three-dimensional model.

[0024] In some cases, the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

[0025] In some cases, the three-dimensional model is an MRI model.

[0026] In accordance with a eighth aspect of the presently disclosed subject matter, there is provided a navigation method, comprising: obtaining, by a processing circuitry, a three- dimensional model of the area of interest when the area of interest is in a first state; capturing, utilizing at least one image acquisition sensor configured to acquire images of an area of interest within a human body, a series of images of the area of interest, wherein: (a) at the time of capturing the series of images the area of interest is in a second state, different than the first state, and (b) the series of images including at least two images in which a known marker attached to a movable object is placed in contact with respective known sites within the area of interest, the known site being included in the three- dimensional model; determining, by the processing circuitry, a location of the tool within the three-dimensional model by determining a disposition between the locations of the known sites in the images and the locations of the known sites in the three-dimensional model.

[0027] In some cases, the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

[0028] In some cases, the three-dimensional model is an MRI model.BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to understand the presently disclosed subject matter and to see how it may be carried out in practice, the subj ect matter will now be described, by way of non-limiting examples only, with reference to the accompanying drawings, in which:

[0030] Fig. 1 is a block diagram, schematically illustrating a navigation system for assisting users in navigation of instruments during procedures by using painted markers, painted on the patient's body, according to some embodiments;

[0031] Fig. 2 is a flowchart schematically illustrating an exemplary navigation method for assisting users in navigation of instruments during procedures by using painted markers, painted on the patient's body, according to some embodiments;

[0032] Fig. 3 is a flowchart illustrating another example of a sequence of operations carried out by the navigation system for assisting users in navigation of instrumentsduring procedures by using painted markers, painted on the patient's body, in accordance with the presently disclosed subject matter;

[0033] Fig. 4a and 4b are schematic illustrations of converting two-dimensional images into a three-dimensional model, according to some embodiments;

[0034] Fig. 5a and 5c are schematic illustrations of converting two-dimensional images into a three-dimensional model, according to some embodiments;

[0035] Fig. 6 is shown a flowchart illustrating a sequence of operations carried out by the navigation system for assisting users in navigation of instruments during procedures, in accordance with the presently disclosed subject matter; and

[0036] Fig. 7 is a flowchart illustrating a sequence of operations carried out by the navigation system for assisting users in navigation of instruments within a human body during procedures, in accordance with the presently disclosed subject matter.

[0037] The drawings in the above listed figures only provide schematic illustrations of elements, operations, functionalities, devices, etc. and do not necessarily encompass all operational steps, elements, devices, etc. required in order to enable the system, method or part thereof to perform its full functioning or purpose, but merely provide a scheme according to which a person with skills in the art can understand how to implement at least some main principles of the system / method.

[0038] Elements or object shown in the drawings of the figures listed above are not necessarily drawn to scale.DETAILED DESCRIPTION

[0039] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the presently disclosed subject matter. However, it will be understood by those skilled in the art that the presently disclosed subject matter may be practiced without these specific details. In other instances, well- known methods, procedures, and components have not been described in detail so as not to obscure the presently disclosed subject matter.

[0040] In the drawings and descriptions set forth, identical reference numerals indicate those components that are common to different embodiments or configurations.

[0041] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as“obtaining^ “acquiring”, “determining”, “generating” and the like and / or any conjugations of such terms, include action and / or processes of a computer / controller / processor that manipulate data and / or transform data into other data, said data represented as physical quantities, e.g., such as electronic quantities, and / or said data representing the physical objects.

[0042] The terms “computer”, “processor”, “processing resource”, “processing circuitry”, and “controller” should be expansively construed to cover any kind of electronic device with data processing capabilities, including, by way of non-limiting example, a personal desktop / laptop computer, a server, a computing system, a communication device, a smartphone, a tablet computer, a smart television, a processor (e.g. digital signal processor (DSP), a microcontroller, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), a group of multiple physical machines sharing performance of various tasks, virtual servers co-residing on a single physical machine, any other electronic computing device, and / or any combination thereof.

[0043] The operations in accordance with the teachings herein may be performed by a computer specially constructed for the desired purposes or by a general-purpose computer specially configured for the desired purpose by a computer program stored in a non- transitory computer readable storage medium. The term "non-transitory" is used herein to exclude transitory, propagating signals, but to otherwise include any volatile or nonvolatile computer memory technology suitable to the application.

[0044] As used herein, the phrase "for example," "such as", "for instance" and variants thereof describe non-limiting embodiments of the presently disclosed subject matter. Reference in the specification to "one case", "some cases", "other cases" or variants thereof means that a particular feature, structure or characteristic described in connection with the embodiment s) is included in at least one embodiment of the presently disclosed subject matter. Thus, the appearance of the phrase "one case", "some cases", "other cases" or variants thereof does not necessarily refer to the same embodiment s).

[0045] It is appreciated that, unless specifically stated otherwise, certain features of the presently disclosed subject matter, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the presently disclosed subject matter, which are, forbrevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination.

[0046] Any reference in the specification to a method should be applied mutatis mutandis to a system capable of executing the method and should be applied mutatis mutandis to a non-transitory computer readable medium that stores instructions that once executed by a computer result in the execution of the method.

[0047] Any reference in the specification to a system should be applied mutatis mutandis to a method that may be executed by the system and should be applied mutatis mutandis to a non-transitory computer readable medium that stores instructions that may be executed by the system.

[0048] Any reference in the specification to a non-transitory computer readable medium should be applied mutatis mutandis to a system capable of executing the instructions stored in the non-transitory computer readable medium and should be applied mutatis mutandis to method that may be executed by a computer that reads the instructions stored in the non-transitory computer readable medium.

[0049] In embodiments of the presently disclosed subject matter, fewer, more and / or different stages than those shown in Figs. 2, 3, 6 and 7 may be executed. In embodiments of the presently disclosed subject matter one or more stages illustrated in Figs. 2, 3, 6 and 7 may be executed in a different order and / or one or more groups of stages may be executed simultaneously. Fig. 1 illustrates a general schematic of the system architecture in accordance with an embodiment of the presently disclosed subject matter. Each module in Fig- 1 can be made up of any combination of software, hardware and / or firmware that performs the functions as defined and explained herein. The modules in Fig. 1 may be centralized in one location or dispersed over more than one location. In other embodiments of the presently disclosed subject matter, the system may comprise fewer, more, and / or different modules than those shown in Fig. 1.

[0050] Aspects of disclosed embodiments pertain to navigation systems and methods for assisting users in navigation of instruments through an outer or inner body part by using painted markers, painted on the patient's body, in order to determine updated positioning (location and orientation) of the instrument being used by the user during a procedure.

[0051] Any one of the terms: “surgical instrument” or “surgical tool”, which may be interchangeably used herein, may refer to any device, object, equipment, etc., that has apart thereof or connected thereto that can be used for any purpose during a surgical / cosmetic / medical procedure. For example, the instruments may be used for cutting internal and or external tissue, illumination / radiation of internal and or external tissue (e.g., for using a laser beam to cut through body tissue or to illuminate an internal body part area / tissue for detection, navigation and / or diagnostical purposes), drilling, stitching, and / or connecting of tissues, etc.

[0052] The terms “surgical instrument(s)”, “instrument”, "tool" and “surgical tool” may be used interchangeably herein and may refer to any element, device, apparatus, object and / or system that is used for any medical purpose such as for irradiating or illuminating, cutting, impinging, stimulating, detecting, stitching, marking, contracting / expanding, implanting an object, removal of tissue, etc. of / to / from the body part or part thereof.

[0053] The term “surgical procedure” or “operation” may be used interchangeably herein and may refer to any intrusive medical procedure for any medical purpose such as for any therapeutical and / or diagnostical purposes.

[0054] The term “surgical procedure” may also refer to virtual or simulated intrusive medical procedures done as a simulation over a real or virtual model of a body part.

[0055] According to some embodiments, the term “body part” used herein may refer to any part of a body of an individual (human or animal) such as an internal organ, tissue, limb, bone, vessel, system or part thereof (such as digestion system, nervous system, etc.), etc., that can be subjected to any type and method of a surgical procedure for any purpose such as for any therapeutic and / or diagnostic purpose, and may also refer to a virtual or simulated body part used for surgeon’s training and / or performances-evaluation.

[0056] The term “surgeon” used herein may refer to any person who performs any part of a surgical procedure whether a medical staff member (a certified surgeon, a nurse, an intern, a technician etc.) and is not limited only to certified surgery physicians.

[0057] Reference is now made to Fig. 1 showing a block diagram, schematically illustrating a navigation system for assisting users in navigation of instruments during procedures by using painted markers, painted on the patient's body, according to some embodiments.

[0058] In accordance with the presently disclosed subject matter, the navigation system 100 (also interchangeably referred to herein as “system 100”) can comprise a network interface 110. The network interface 110 (e.g., a network card, a Wi-Fi client, a Li-Ficlient, 3G / 4G client, or any other component), enables system 100 to communicate over a network with external systems and handles inbound and outbound communications from such systems. For example, system 100 can receive, through network interface 110, images of an area of interest acquired using X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging.

[0059] System 100 can further comprise or be otherwise associated with a data repository 120 (e.g., a database, a storage system, a memory including Read Only Memory - ROM, Random Access Memory - RAM, or any other type of memory, etc.) configured to store data. Some examples of data that can be stored in the data repository 120 include images of an area of interest, including images acquired using image acquisition sensor 130 and images acquired using X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, etc.

[0060] Data repository 120 can be further configured to enable retrieval and / or update and / or deletion of the stored data. It is to be noted that in some cases, data repository 120 can be distributed, while the system 100 has access to the information stored thereon, e.g., via a wired or wireless network to which system 100 is able to connect (utilizing its network interface 110).

[0061] System 100 further comprises an image acquisition sensor 130 that can acquire images of an area of interest on, or within, a body of a patient. The image acquisition sensor 130 can be a camera, or any other device capable of acquiring images, optionally in the visible spectrum, however not thus limited.

[0062] System 100 further comprises processing circuitry 140. Processing circuitry 140 can be one or more processing units (e.g., central processing units), microprocessors, microcontrollers (e.g., microcontroller units (MCUs)) or any other computing devices or modules, including multiple and / or parallel and / or distributed processing units, which are adapted to independently or cooperatively process data for controlling relevant system 100 resources and for enabling operations related to system’s 100 resources.

[0063] The processing circuitry 140 comprises a navigation module 150, configured to perform a navigation process, as further detailed herein, inter alia with reference to Figs. 2 and 3.

[0064] Turning to Fig. 2 there is shown a flowchart illustrating one example of a sequence of operations carried out by the navigation system for assisting users in navigation ofinstruments during procedures by using painted markers, painted on the patient's body, in accordance with the presently disclosed subject matter.

[0065] Navigation system 100 can be configured to perform a navigation process 200, e.g., using navigation module 150.

[0066] For this purpose, system 100 obtains, by the processing circuitry 140, a tissue penetrating image of an area of interest, including one or more first markers, each having a distinct shape and / or a known size, placed in a respective location on the area of interest (block 210) The one or more markers may be visible to one or more imaging devices, e.g., X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT), ultrasound imaging, and the like, and the tissue penetrating image may be acquired by at least one of the imaging devices.

[0067] In some cases, the first markers may be placed in at least two layers with varying distances with respect to the imaging devices.

[0068] The first markers can be spherical, and in more specific cases globular (i.e. ball shaped). In such cases, each first marker can optionally have a different diameter. It is to be noted that using globular first markers can enable identifying each first marker in an image captured by the imaging devices as the levy of each first marker is identical and uniquely identifiable irrespective of the angle from which it is imaged. It is to be further noted that other methods can be used in order to distinguish between the markers, mutatis mutandis.

[0069] In one non-limiting example, a tissue penetrating image can be an MRI image of a woman's breast. In this example (and in other types of procedures), the first markers (that are visible to MRI in this specific example) can be placed on an upper surface of the woman's breast, and on the lower surface of the woman's breast.

[0070] In another non-limiting example, a tissue penetrating image can be an MRI image (or any one of an X-ray image, a CT scan, an ultrasound image, and the like) of a subject's dorsal side. In this example, the markers can be placed on different areas of the subject's posterior and anterior, so that they are detected by an imaging device regardless of the subject's posture (standing upright, bending down, etc.).

[0071] It is to be of note that the above examples are in no way intended to limit the scope of the presently disclosed subject matter and that the markers can be placed only on one surface / side of an organ / subject, mutatis mutandis.

[0072] Next, system 100 generates, by the processing circuitry, a three-dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the first markers (block 220).

[0073] It is to be of note that as each first marker can be distinguished from the other first markers, and in view of the fact that the first markers may be arranged in at least two layers (each having a different distance from the imaging devices), a three-dimensional model can be generated using known methods and / or techniques, or using proprietary methods and / or techniques.

[0074] System 100 then acquires, utilizing the image acquisition sensor, a second image of the area of interest, including a tool (e.g. a surgical tool, a cosmetic tool, a medical tool, etc.) and a plurality of second markers painted in the respective locations of least a subset of the first markers, replacing at least the subset of the first markers (block 230).

[0075] In some cases, the second markers may be painted in paint visible in a spectrum covered by the image acquisition sensor.

[0076] In some cases, the second markers can be painted in paint that will remain visible during the entire procedure, such as henna, permanent marker paint, etc., which ensures that the markers are visible to the image acquisition sensor throughout the procedure.

[0077] Lastly, system 100 determines a location of the tool within the three-dimensional model, by determining a disposition between the location of the plurality of second markers and the locations of the respective first markers (block 240).

[0078] It is to be noted that even if the area of interest moves so that the second markers change their position, the location of the tool with respect to the second markers (whose position is known with respect to, for example, a tumor to be removed) can be determined. In some cases, the three-dimensional model can be updated based on the change in the position of the second markers (which are placed in the position of the first markers that such second markers replace).

[0079] It is to be further noted that in some cases, more than one image acquisition sensor can acquire second images, which can enable identification of movements of the second markers in three-dimensions. It is also to be noted that the more image acquisition sensors exist, and the more images are captured, the resolution of the three-dimensional model can be improved.

[0080] It is to be still further noted that in some cases, the second markers are painted before capturing of the tissue penetrating image, and the first markers (or a subset thereof) are placed on the second markers (e.g. by gluing them).

[0081] Still further, it is to be noted that in some cases, the first markers can remain in place throughout the procedure, and navigation can be performed using the first markers alone. In such cases, the second markers can be useful for example in cases where one of the markers is removed for some reason (whether on purpose, by mistake or without intervention such as when the glue fails to hold the markers in place).

[0082] It is to be noted that, with reference to Fig. 2, some of the blocks can be integrated into a consolidated block or can be broken down to a few blocks and / or other blocks may be added. It is to be further noted that some of the blocks are optional. It should be also noted that whilst the flow diagram is described also with reference to the system elements that realizes them, this is by no means binding, and the blocks can be performed by elements other than those described herein.

[0083] Turning to Fig. 3 there is shown a flowchart illustrating another example of a sequence of operations carried out by the navigation system for assisting users in navigation of instruments during procedures by using painted markers, painted on the patient's body, in accordance with the presently disclosed subject matter.

[0084] Navigation system 100 can be configured to perform a navigation process 300, e.g., using navigation module 150.

[0085] For this purpose, system 100 obtains a tissue penetrating image of the area of interest, including one or more markers, each having a distinct shape and / or a known size and is painted in a respective location on the area of interest with paint visible to the image acquisition sensor and to one or more imaging devices (e.g., X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging (an example for such paint can be paint which includes metal grains incorporated therein), etc.) (block 310)

[0086] In some cases, the one or more markers may be painted in at least two layers, with varying distances with respect to the imaging devices, and the tissue penetrating image may be acquired by at least one of the imaging devices.

[0087] In one non-limiting example, a tissue penetrating image can be an MRI image of a woman's breast. In this example (and in other types of procedures), the markers (thatare visible to MRI in this specific example) can be painted on an upper surface of the woman's breast, and on the lower surface of the woman's breast.

[0088] In another non-limiting example, a tissue penetrating image can be an MRI image (or any one of an X-ray image, a CT scan, an ultrasound image, and the like) of a subject's dorsal side. In this example, the markers can be painted on different areas of the subject's posterior and anterior, so that they are detected by an imaging device regardless of the subject's posture (standing upright, bending down, etc.).

[0089] The markers can be in the form of a circle, each having a different diameter. It is to be noted that using circular markers can enable identifying each marker in an image captured by the imaging devices based on its size. In some cases, the markers can have other shapes, as long as the selected shapes enable identification of each specific marker and differentiation between the markers. It is to be further noted that other methods can be used in order to distinguish between the markers, mutatis mutandis.

[0090] Next, system 100 generates, using the processing circuitry 140, a three- dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the markers (block 320).

[0091] It is to be of note that as each marker can be distinguished from the other markers, and in view of the fact that the markers may be arranged in at least two layers (each having a different distance from the imaging devices), a three-dimensional model can be generated using known methods and / or techniques, or using proprietary methods and / or techniques.

[0092] System 100 then acquires, utilizing the image acquisition sensor, a second image of the area of interest, including a tool and at least a subset of the markers (block 330).

[0093] Lastly, system 100 determines, using the processing circuitry 140, a location of the tool within the three-dimensional model, by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image (block 340).

[0094] It is to be noted that even if the area of interest moves so that the markers change their position, the location of the tool with respect to the markers (whose position is known with respect to, for example, a tumor to be removed) can be determined. In some cases, the three-dimensional model can be updated based on the change in the position of the markers.

[0095] It is to be further noted that in some cases, more than one image acquisition sensor can acquire second images, which can enable identification of movements of the markers in three-dimensions. It is also to be noted that the more image acquisition sensors exist, and the more images are captured, the resolution of the three-dimensional model can be improved.

[0096] It is to be noted that, with reference to Fig. 3, some of the blocks can be integrated into a consolidated block or can be broken down to a few blocks and / or other blocks may be added. It is to be further noted that some of the blocks are optional. It should be also noted that whilst the flow diagram is described also with reference to the system elements that realizes them, this is by no means binding, and the blocks can be performed by elements other than those described herein.

[0097] Figs. 4a, 4b and 4c show illustrations of markers placed around a woman's breast in which a tumor exists. Fig. 4a is a side view of the woman's breast. The tumor is marked 410, a first layer of markers placed on one side of the woman's breast is marked 420 and a second layer of markers placed on one side of the woman's breast is marked 430. The markers can be the markers as described with reference to Fig. 2 or the markers as described with reference to Fig. 3. Fig. 4b shows a top view image of the woman's breast and Fig. 4c shows a top view image of the woman's breast when either imaged after scrolling the image acquisition sensor 130, or the imaging devices, to the right, or when the woman's breast position is changed. It can be appreciated that using the information of the markers can enable positioning a tool with respect to the tumor, based on the position of the markers in the image.

[0098] Figs. 5a to 5c show illustrations of markers placed on a subject's posterior and / or anterior. Fig. 5a is a front view of the subject, in which a first layer of markers, denoted 520, are placed on the subject's chest and abdomen. Fig. 5b is a rear view image of the subject, in which a second layer of markers, denoted 530, are placed on the subject's back. Fig. 5c is a side view image of the subject, in which both markers 520 and 530 can be visible. It can be appreciated that using the information of the markers can enable positioning a tool with respect to the subject's body, based on the position of the markers in the image.

[0099] The markers can be markers as described with reference to Fig. 2 or markers as described with reference to Fig. 3. It can be appreciated that using the information of the markers can enable positioning a tool, based on the position of the markers in the image.

[0100] Turning to Fig. 6 there is shown a flowchart illustrating a sequence of operations carried out by the navigation system for assisting users in navigation of instruments during procedures, in accordance with the presently disclosed subject matter.

[0101] Navigation system 100 can be configured to perform a navigation process 500, e.g., using navigation module 150.

[0102] For this purpose, system 100 obtains a tissue penetrating image of the area of interest, at least partially including a soft tissue (e.g. a human breast) being pressed between two panels (block 610). The area of interest includes a plurality of markers including at least one first marker placed on one side of the area of interest and at least one second marker placed substantially on an opposite side of the area of interest

[0103] In some cases, the tissue penetrating image may be acquired by at least one of one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, or any other type of tissue penetrating imaging technology.

[0104] In a non-limiting example, a tissue penetrating image can be an X ray image of a woman's breast. In this example (and in other types of procedures), the first markers (that are visible to X ray in this specific example) can be placed on one side of the woman's breast, and the second markers can be placed substantially on the other side of the woman's breast.

[0105] The markers can be in the form of a circle, each having a different diameter. It is to be noted that using circular markers can enable identifying each marker in an image captured by the imaging devices based on its size. In some cases, the markers can have other shapes, as long as the selected shapes enable identification of each specific marker and differentiation between the markers. It is to be further noted that other methods can be used in order to distinguish between the markers, mutatis mutandis.

[0106] Next, system 100 generates, using the processing circuitry 140, a three- dimensional model from the tissue penetrating image, utilizing the location of the markers in the tissue penetrating image, a contour of at least part of the area of interest visible in the tissue penetrating image, and a known distance between the panels (block 620).

[0107] It is to be of note that as each marker can be distinguished from the other markers, and in view of the fact that the markers can be arranged in at least two layers (each having a different distance from the imaging devices), a three-dimensional model can be generated using known methods and / or techniques, or using proprietary methods and / or techniques.

[0108] System 100 then acquires, utilizing the image acquisition sensor, a second image of the area of interest, including a tool and at least a subset of the markers (block 630).

[0109] Lastly, system 100 determines, using the processing circuitry 140, a location of the tool within the three-dimensional model by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image (block 640).

[0110] It is to be noted that even if the area of interest moves so that the markers change their position, the location of the tool with respect to the markers (whose position is known with respect to, for example, a tumor to be removed) can be determined. In some cases, the three-dimensional model can be updated based on the change in the position of the markers.

[0111] It is to be further noted that in some cases, more than one image acquisition sensor can acquire second images, which can enable identification of movements of the markers in three-dimensions. It is also to be noted that the more image acquisition sensors exist, and the more images are captured, the resolution of the three-dimensional model can be improved.

[0112] It is to be noted that, with reference to Fig. 6, some of the blocks can be integrated into a consolidated block or can be broken down to a few blocks and / or other blocks may be added. It is to be further noted that some of the blocks are optional. It should be also noted that whilst the flow diagram is described also with reference to the system elements that realizes them, this is by no means binding, and the blocks can be performed by elements other than those described herein.

[0113] Turning to Fig. 7 is a flowchart illustrating a sequence of operations carried out by the navigation system for assisting users in navigation of instruments within a human body during procedures, in accordance with the presently disclosed subject matter.

[0114] Navigation system 100 can be configured to perform a navigation process 600, e.g., using navigation module 150.

[0115] For this purpose, system 100 obtains a three-dimensional model of the area of interest when the area of interest is in a first state (the first state can be an uninflated state following which the area of interest is inflated for various reasons) (block 710).

[0116] The three-dimensional model can be generated using: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, or any other type of tissue penetrating imaging technology that is suitable for generating a three- dimensional model of an area of interest within the human body.

[0117] Next, system 100 captures a series of images of the area of interest, utilizing the image acquisition sensor, where at the time of capture the area of interest is in a second state (the second state can be an inflated state), different than the first state. The series of images includes at least two images in which a known marker attached to a movable object is placed in contact with respective known sites within the area of interest, being included in the three-dimensional model (block 720).

[0118] Lastly, system 100 determines, using the processing circuitry 140, a location of the tool within the three-dimensional model, by determining a disposition between the locations of the known sites in the images and the locations of the known sites in the three-dimensional model (block 730).

[0119] It is to be noted in this respect that when navigating within the human body, placement of markers within the area of interest can be extremely difficult, and it is desirable to have a solution that enables navigating without presence of fixed markers, but only using a calibration such as the one disclosed herein. Using the teachings here, once the location of the tool is determined based on the images acquired in step 620, the tool can be navigated without any markers, using the obtained three-dimensional model that is calibrated based on the images.

[0120] It is to be noted that in some cases, for example when using an MRLbased three- dimensional model, two images, each imaging a respective different site within the area of interest when the marker is in contact with such site, can suffice for the calibration as the MRI model enables determining the distance between the sites on one plane. Having an (x, y) coordinate based on the three-dimensional model and the images and a (z) coordinate based on the MRI model can enable calibrating the three-dimensional model.

[0121] It is to be further noted that the more images are captured in block 620, each imaging a different known site, the better calibration resolution can be achieved.

[0122] It is to be noted that, with reference to Fig. 7, some of the blocks can be integrated into a consolidated block or can be broken down to a few blocks and / or other blocks may be added. It is to be further noted that some of the blocks are optional. It should be also noted that whilst the flow diagram is described also with reference to the system elements that realizes them, this is by no means binding, and the blocks can be performed by elements other than those described herein.

[0123] It is to be understood that the above disclosed subject matter is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings. The presently disclosed subject matter is capable of other embodiments and of being practiced and carried out in various ways. Hence, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. As such, those skilled in the art will appreciate that the conception upon which this disclosure is based may readily be utilized as a basis for designing other structures, methods, and systems for carrying out the several purposes of the present presently disclosed subject matter.

[0124] It will also be understood that the system according to the presently disclosed subject matter can be implemented, at least partly, as a suitably programmed computer. Likewise, the presently disclosed subject matter contemplates a computer program being readable by a computer for executing the disclosed method. The presently disclosed subject matter further contemplates a machine-readable memory tangibly embodying a program of instructions executable by the machine for executing the disclosed method.

Claims

Claims:

1. A navigation system, comprising: at least one image acquisition sensor configured to acquire images of an area of interest on a human body; and a processing circuitry, configured to: obtain a tissue penetrating image of the area of interest, wherein (a) the area of interest includes a plurality of first markers, (b) each of the first markers has a distinct shape and / or a known size, (c) each of the first markers is visible to one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, (d) each of the first markers is placed in a respective location on the area of interest, and (e) the tissue penetrating image is acquired by at least one of the imaging devices; generate a three-dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the first markers; acquire, utilizing the image acquisition sensor, a second image of the area of interest, wherein: (a) the area of interest includes a tool and a plurality of second markers painted in the respective locations of at least a subset of the first markers and replacing at least the subset of the first markers; (b) the second markers being painted in paint visible in a spectrum covered by the image acquisition sensor; and determine a location of the tool within the three-dimensional model by determining a disposition between the location of the plurality of second markers and the locations of the respective first markers.

2. The navigation system of claim 1, wherein the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

3. The navigation system of claim 1, wherein the first markers are placed in at least two layers with varying distances with respect to the imaging devices.

4. A navigation system, comprising:at least one image acquisition sensor configured to acquire images of an area of interest on a human body; and a processing circuitry, configured to: obtain a tissue penetrating image of the area of interest, wherein (a) the area of interest includes a plurality of markers, (b) each of the markers has a distinct shape and / or a known size, (c) each of the markers is painted in a respective location on the area of interest with paint visible to the image acquisition sensor and to one or more imaging devices selected from a group consisting of X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, and (d) the tissue penetrating image is acquired by at least one of the imaging devices; generate a three-dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the markers; acquire, utilizing the image acquisition sensor, a second image of the area of interest, wherein the area of interest includes a tool and at least a subset of the markers; and determine a location of the tool within the three-dimensional model by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image.

5. The navigation system of claim 4, wherein the tool is one of a surgical tool, a cosmetic tool or a medical tool.

6. The navigation system of claim 4, wherein the markers are painted in at least two layers with varying distances with respect to the imaging devices7. A navigation method, comprising: obtaining, by a processing circuitry, a tissue penetrating image of the area of interest, wherein (a) the area of interest includes a plurality of first markers, (b) each of the first markers has a distinct shape and / or a known size, (c) each of the first markers is visible to one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasoundimaging, (d) each of the first markers is placed in a respective location on the area of interest, and (e) the tissue penetrating image is acquired by at least one of the imaging devices; generating, by the processing circuitry, a three-dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the first markers; acquiring, utilizing an image acquisition sensor configured to acquire images of an area of interest on a human body, a second image of the area of interest, wherein: (a) the area of interest includes a tool and a plurality of second markers painted in the respective locations of least a subset of the first markers and replacing at least the subset of the first markers; (b) the second markers being painted in paint visible in a spectrum covered by the image acquisition sensor; and determining, by the processing circuitry, a location of the tool within the three- dimensional model by determining a disposition between the location of the plurality of second markers and the locations of the respective first markers.

8. The navigation method of claim 7, wherein the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

9. The navigation method of claim 7, wherein the first markers are placed in at least two layers with varying distances with respect to the imaging devices10. A navigation method, comprising: obtaining, by a processing circuitry, a tissue penetrating image of the area of interest, wherein (a) the area of interest includes a plurality of markers, (b) each of the markers has a distinct shape and / or a known size, (c) each of the markers is painted in a respective location on the area of interest with paint visible to the image acquisition sensor and to one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging, and (d) the tissue penetrating image is acquired by at least one of the imaging devices; generating, by the processing circuitry, a three-dimensional model from the tissue penetrating image, based on the distinct shape and / or the known size of the markers;acquiring, utilizing an image acquisition sensor configured to acquire images of an area of interest on a human body, a second image of the area of interest, wherein the area of interest includes a tool and at least a subset of the markers; and determining, by the processing circuitry, a location of the tool within the three- dimensional model by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image.

11. The navigation method of claim 10, wherein the tool is one of a surgical tool, a cosmetic tool or a medical tool.

12. The navigation method of claim 10, wherein the markers are painted in at least two layers with varying distances with respect to the imaging devices.

13. A navigation system, comprising: at least one image acquisition sensor configured to acquire images of an area of interest on a human body; and a processing circuitry, configured to: obtain a tissue penetrating image of the area of interest, wherein (a) the area of interest at least partially includes a soft tissue being pressed between two panels; (b) the area of interest includes a plurality of markers including at least one first marker placed on one side of the area of interest and at least one second marker placed substantially on an opposite side of the area of interest, (c) the tissue penetrating image is acquired by at least one of one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging; generate a three-dimensional model from the tissue penetrating image, utilizing the location of the markers in the tissue penetrating image, a contour of at least part of the area of interest visible in the tissue penetrating image, and a known distance between the panels; acquire, utilizing the image acquisition sensor, a second image of the area of interest, wherein the area of interest includes a tool and at least a subset of the markers; anddetermine a location of the tool within the three-dimensional model by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image.

14. The navigation system of claim 13, wherein the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

15. A navigation method, comprising: obtaining, by a processing circuitry, a tissue penetrating image of an area of interest on a human body, wherein (a) the area of interest at least partially includes a soft tissue being pressed between two panels; (b) the area of interest includes a plurality of markers including at least one first marker placed on one side of the area of interest and at least one second marker placed substantially on an opposite side of the area of interest, (c) the tissue penetrating image is acquired by at least one of one or more imaging devices selected from a group consisting of: X-Ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or ultrasound imaging; generating, by the processing circuitry, a three-dimensional model from the tissue penetrating image, utilizing the location of the markers in the tissue penetrating image, a contour of at least part of the area of interest visible in the tissue penetrating image, and a known distance between the panels; acquiring, utilizing an image acquisition sensor configured to acquire images of the area of interest, a second image of the area of interest, wherein the area of interest includes a tool and at least a subset of the markers; and determining, by the processing circuitry, a location of the tool within the three- dimensional model by determining a disposition between the location of the markers in the tissue penetrating image and the locations of the markers in the second image.

16. The navigation method of claim 15, wherein the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

17. A navigation system, comprising:at least one image acquisition sensor configured to acquire images of an area of interest within a human body; and a processing circuitry, configured to: obtain a three-dimensional model of the area of interest when the area of interest is in a first state; capture a series of images of the area of interest utilizing the image acquisition sensor, wherein: (a) at the time of capturing the series of images the area of interest is in a second state, different than the first state, and (b) the series of images including at least two images in which a known marker attached to a movable object is placed in contact with respective known sites within the area of interest, the known site being included in the three-dimensional model; determine a location of the tool within the three-dimensional model by determining a disposition between the locations of the known sites in the images and the locations of the known sites in the three-dimensional model.

18. The navigation system of claim 17, wherein the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

19. The navigation system of claim 17, wherein the three-dimensional model is an MRI model.

20. A navigation method, comprising: obtaining, by a processing circuitry, a three-dimensional model of the area of interest when the area of interest is in a first state; capturing, utilizing at least one image acquisition sensor configured to acquire images of an area of interest within a human body, a series of images of the area of interest, wherein: (a) at the time of capturing the series of images the area of interest is in a second state, different than the first state, and (b) the series of images including at least two images in which a known marker attached to a movable object is placed in contact with respective known sites within the area of interest, the known site being included in the three-dimensional model;determining, by the processing circuitry, a location of the tool within the three- dimensional model by determining a disposition between the locations of the known sites in the images and the locations of the known sites in the three-dimensional model.

21. The navigation method of claim 20, wherein the tool is one of: a surgical tool, a cosmetic tool or a medical tool.

22. The navigation method of claim 20, wherein the three-dimensional model is an MRI model.

Images