Systems and methods for aiming and aligning of treatment tool within x-ray device or ultrasound device environment
The system uses a calibration plate with markers and a processing unit to align treatment instruments in X-ray or ultrasound environments, addressing the need for complex mechanical connections and reducing patient radiation exposure, thereby enhancing alignment efficiency and reducing treatment time.
Patent Information
- Application Number
- JP2024133843
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2019-11-12
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2040-03-25
AI Technical Summary
Current methods for aiming and positioning treatment instruments in X-ray or ultrasound environments require robust mechanical connections and expose patients to multiple X-ray radiations, increasing cost and complexity while necessitating complex tracking systems.
A system comprising a calibration plate with radiopaque and optical markers, a camera, and a processing unit that determines the position and orientation of the treatment instrument relative to the imaging device using visual representations and algorithms, allowing alignment without rigid mechanical connections and reducing patient exposure to radiation.
The system effectively aligns treatment instruments with target areas while minimizing patient exposure to X-ray radiation and reducing overall treatment process time and complexity.
Smart Images

Figure 2024156967000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to the field of systems and methods for aiming surgical instruments, and more particularly to systems and methods for aiming surgical instruments using x-ray or ultrasound devices. [Background technology]
[0002] Typically, aiming an instrument at a target area and / or aligning the instrument with respect to an imaging device (e.g., x-ray, ultrasound, etc.) requires a rigid mechanical connection between the instrument and the imaging device and / or an expensive and / or complex tracking unit capable of monitoring the relative position and / or orientation between the instrument and the imaging device at a relatively high update rate (e.g., multiple times per second). Using x-ray imaging to track the instrument, for example, aiming and / or aligning the instrument with respect to the x-ray device may require multiple exposures of the patient to x-ray radiation.
[0003] What is needed is a system and method for aiming and / or aligning a treatment tool in an imaging device environment that can reduce the cost and complexity of aiming and / or alignment compared to current processes while eliminating the need for a rigid mechanical connection between the treatment tool and the imaging device.With respect to x-ray devices, what is needed is a system and method for aiming and / or aligning a treatment tool in an x-ray device environment that can reduce patient exposure to x-ray radiation compared to current processes. Summary of the Invention
[0004] Some embodiments may provide a system for aiming and aligning a treatment tool in an X-ray machine environment, the system may include a calibration plate mountable to an X-ray machine, the calibration plate including at least one radiopaque marker and at least one optical marker at a predetermined location within the calibration plate; a camera mountable to the treatment tool at a predetermined location and orientation relative to the treatment tool; and a processing unit in communication with the camera and an X-ray imaging unit of the X-ray machine, the processing unit configured to receive an X-ray image from the X-ray imaging unit including a visual representation of the at least one radiopaque marker in the X-ray image and to process a visual representation of the at least one radiopaque marker in the calibration plate. the calibration plate is adapted to receive from the camera a camera image including a visual representation of the at least one optical marker and a predetermined position of the at least one optical marker within the calibration plate; and the calibration plate is adapted to determine a position and orientation of the calibration plate relative to the X-ray device based on the predetermined positions of the at least one radiopaque marker and certain parameters of the X-ray device; the calibration plate is adapted to receive from the camera a camera image including a visual representation of the at least one optical marker and a predetermined position of the at least one optical marker within the calibration plate; and the calibration plate is adapted to determine a position and orientation of the treatment tool relative to the X-ray device based on the determined position and orientation of the calibration plate relative to the X-ray device, the determined position and orientation of the camera within the calibration plate, and a predetermined position and orientation of the camera relative to the treatment tool.
[0005] In some embodiments, the calibration plate may include at least one of: at least one asymmetric radiopaque marker and a plurality of symmetric radiopaque markers asymmetrically positioned within the calibration plate.
[0006] In some embodiments, the system may include a display, and the processing unit is configured to present a visual indicator on the display indicative of the determined position and orientation of the treatment tool.
[0007] In some embodiments, the processing unit is configured to determine whether the determined position and orientation of the treatment tool relative to the X-ray device is within an acceptable error range by comparing it with a required position and orientation of the treatment tool relative to the X-ray device, and to indicate whether the position and orientation of the treatment tool is within the acceptable error range by modifying the visual indicator.
[0008] In some embodiments, the processing unit is configured to superimpose the visual indicator onto an X-ray image of a target area within a patient's body.
[0009] In some embodiments, the processing unit is configured to detect or mark the target area in the X-ray image, determine whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the X-ray image, and indicate whether the treatment tool is aligned with respect to the target area by modifying the visual indicator.
[0010] In some embodiments, the processing unit is configured to generate treatment tool guidance instructions based on the determined position and orientation of the treatment tool relative to the X-ray device, and display the treatment tool guidance instructions on the display.
[0011] In some embodiments, the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and the amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
[0012] In some embodiments, the processing unit is configured to display at least one of: treatment tool visual data including a visual representation of at least a portion of the treatment tool; and system component visual data including a visual representation of at least one component of the system and indicating an actual position and orientation of each of the at least one component relative to the treatment tool.
[0013] In some embodiments, the treatment device is one of a focused ultrasound transducer and an interventional treatment device.
[0014] Some embodiments may provide a method of aiming and aligning a treatment tool in an X-ray machine environment, the method including: mounting a calibration plate to an X-ray machine, the calibration plate may include at least one radiopaque marker and at least one optical marker positioned at a predetermined location within the calibration plate; mounting a camera to the treatment tool at a predetermined location and orientation relative to the treatment tool; acquiring, with the X-ray machine, an X-ray image of the calibration plate, the X-ray image may include a visual representation of the at least one radiopaque marker; and acquiring, with a processing unit, the visual representation of the at least one radiopaque marker in the X-ray image and the predetermined location of the at least one radiopaque marker within the calibration plate. determining a position and orientation of the calibration plate relative to the X-ray device based on a position of the calibration plate relative to the X-ray device and certain parameters of the X-ray device; acquiring with the camera a camera image including a visual representation of the at least one optical marker; determining a position and orientation of the camera relative to the calibration plate based on the visual representation of the at least one optical marker in the camera image and a predetermined position of the at least one optical marker within the calibration plate; and determining a position and orientation of the treatment tool relative to the X-ray device based on the determined position and orientation of the calibration plate relative to the X-ray device, the determined position and orientation of the camera within the calibration plate, and the predetermined position and orientation of the camera relative to the treatment tool.
[0015] In some embodiments, the method may include displaying a visual indicator on a display that indicates the determined position and orientation of the treatment tool.
[0016] In some embodiments, the method may include comparing the determined position and orientation of the treatment tool relative to the X-ray device with a required position and orientation of the treatment tool relative to the X-ray device to determine whether it is within an acceptable error range.
[0017] In some embodiments, the method may include indicating whether the position and orientation of the tool is within the acceptable error range by modifying the visual indicator.
[0018] In some embodiments, the method may include superimposing the visual indicator on an x-ray image of a target area within a patient's body.
[0019] In some embodiments, the method may include detecting or marking the target area in the X-ray image, determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the X-ray image, and indicating whether the treatment tool is aligned with respect to the target area by modifying the visual indicator.
[0020] In some embodiments, the method may include generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool relative to the X-ray device, and displaying the treatment tool guidance instruction on the display.
[0021] In some embodiments, the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and the amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
[0022] In some embodiments, the method may include displaying at least one of: treatment tool visual data including a visual representation of at least a portion of the treatment tool; and system component visual data including a visual representation of at least one component of the system and indicating an actual position and orientation of each of the at least one component relative to the treatment tool.
[0023] In some embodiments, the method may include aiming and aligning the treatment tool according to the visual indicator displayed on the display without exposing a patient to X-ray imaging by the X-ray device.
[0024] Some embodiments may provide a system for aiming and aligning a treatment tool in an ultrasound imaging device environment, the system may include a calibration plate attachable to an ultrasound imaging probe of the ultrasound imaging device, the calibration plate including at least one optical marker at a predetermined position within the calibration plate; a camera attachable to the treatment tool at a predetermined position and orientation relative to the treatment tool; and a processing unit in communication with the camera and an ultrasound imaging unit of the ultrasound imaging device, the processing unit being configured to receive a camera image from the camera including a visual representation of the at least one optical marker, determine a position and orientation of the camera relative to the calibration plate based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate, and determine a position and orientation of the treatment tool relative to the ultrasound imaging probe based on the predetermined position and orientation of the calibration plate relative to the ultrasound imaging probe, the determined position and orientation of the camera relative to the calibration plate, and a known position and orientation of the camera relative to the treatment tool.
[0025] In some embodiments, the system may include a display, and the processing unit is configured to present a visual indicator on the display indicative of the determined position and orientation of the treatment tool.
[0026] In some embodiments, the processing unit is configured to determine whether the determined position and orientation of the treatment tool relative to the ultrasound imaging probe is within an acceptable error range by comparing it with a required position and orientation of the treatment tool relative to the ultrasound imaging probe, and to indicate whether the position and orientation of the treatment tool relative to the ultrasound imaging probe is within the acceptable error range by modifying the visual indicator.
[0027] In some embodiments, the processing unit is configured to superimpose the visual indicator onto an ultrasound image of a target region within a patient's body.
[0028] In some embodiments, the processing unit is configured to detect or mark the target area in the ultrasound image, determine whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the ultrasound image, and indicate whether the treatment tool is aligned with respect to the target area by modifying the visual indicator.
[0029] In some embodiments, the processing unit is configured to generate treatment tool guidance instructions based on the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe, and to display the treatment tool guidance instructions on the display.
[0030] In some embodiments, the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and the amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
[0031] In some embodiments, the processing unit is configured to display at least one of: treatment tool visual data including a visual representation of at least a portion of the treatment tool; and system component visual data including a visual representation of at least one component of the system and indicating an actual position and orientation of each of the at least one component relative to the treatment tool.
[0032] In some embodiments, the treatment device is one of a focused ultrasound transducer and an interventional treatment device.
[0033] Some embodiments may provide a method for aiming and aligning a treatment tool in an ultrasound imaging device environment, the method may include attaching a calibration plate to an ultrasound imaging probe of an ultrasound imaging device, the calibration plate may include at least one optical marker positioned at a predetermined position within the calibration plate; attaching a camera to the treatment tool at a predetermined position and orientation relative to the treatment tool; acquiring a camera image by the camera including a visual representation of the at least one optical marker; acquiring a camera image by the camera including a visual representation of the at least one optical marker; determining a position and orientation of the camera relative to the calibration plate based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; and determining a position and orientation of the treatment tool relative to the ultrasound imaging probe based on the determined position and orientation of the calibration plate relative to the ultrasound imaging probe, the determined position and orientation of the camera within the calibration plate, and the predetermined position and orientation of the camera relative to the treatment tool.
[0034] In some embodiments, the method may include displaying a visual indicator on a display that indicates the determined position and orientation of the treatment tool.
[0035] In some embodiments, the method may include comparing the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe with a required position and orientation of the treatment tool relative to the ultrasonic imaging probe to determine whether it is within an acceptable error range.
[0036] In some embodiments, the method may include indicating whether a position and orientation of the treatment tool relative to the ultrasound imaging probe is within the acceptable error range by modifying the visual indicator.
[0037] In some embodiments, the method may include superimposing the visual indicator on an ultrasound image of a target region within a patient's body.
[0038] In some embodiments, the method may include detecting or marking the target area in the ultrasound image, determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the ultrasound image, and indicating whether the treatment tool is aligned with respect to the target area by modifying the visual indicator.
[0039] In some embodiments, the method may include generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe, and displaying the treatment tool guidance instruction on the display.
[0040] In some embodiments, the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and the amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
[0041] In some embodiments, the method may include displaying at least one of: treatment tool visual data including a visual representation of at least a portion of the treatment tool; and system component visual data including a visual representation of at least one component of the system and indicating an actual position and orientation of each of the at least one component relative to the treatment tool.
[0042] In some embodiments, the method may include aiming and aligning the treatment tool according to the visual indicator displayed on the display.
[0043] Some embodiments may provide a system for aiming and aligning a treatment tool in an X-ray device environment, the system may include a calibration plate attachable to a treatment tool, the calibration plate including at least one optical marker at a predetermined position within the calibration plate; a camera attachable to the X-ray device; and a processing unit in communication with the camera and an X-ray imaging unit of the X-ray device, the processing unit being configured to: receive from the camera a camera image including a visual representation of the at least one optical marker; determine a position and orientation of the calibration plate relative to the camera based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; and determine a position and orientation of the treatment tool relative to the X-ray device based on the determined position and orientation of the calibration plate relative to the camera and the position and orientation of the camera relative to the X-ray device.
[0044] In some embodiments, the position and orientation of the camera is known.
[0045] In some embodiments, the camera includes at least one radiopaque marker positioned at least partially within a field of view of an X-ray source of the X-rays, and the processing unit is configured to receive an X-ray image including a visual representation of the at least one radiopaque marker and determine a position and orientation of the camera relative to the X-ray device based on the visual representation of the at least one radiopaque marker.
[0046] In some embodiments, the system may include a display, and the processing unit is configured to present a visual indicator on the display indicative of the determined position and orientation of the treatment tool.
[0047] In some embodiments, the processing unit is configured to compare the determined position and orientation of the treatment tool relative to the X-ray device with a required position and orientation of the treatment tool relative to the X-ray device to determine whether it is within an acceptable error range, and to indicate whether the position and orientation of the treatment tool relative to the X-ray device is within the acceptable error range by modifying the visual indicator.
[0048] In some embodiments, the processing unit is configured to superimpose the visual indicator onto an X-ray image of a target area within a patient's body.
[0049] In some embodiments, the processing unit is configured to detect or mark the target area in the X-ray image, determine whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the X-ray image, and indicate whether the treatment tool is aligned with respect to the target area by modifying the visual indicator.
[0050] In some embodiments, the processing unit is configured to generate treatment tool guidance instructions based on the determined position and orientation of the treatment tool relative to the X-ray device, and display the treatment tool guidance instructions on the display.
[0051] In some embodiments, the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and the amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
[0052] In some embodiments, the processing unit is configured to display at least one of: treatment tool visual data including a visual representation of at least a portion of the treatment tool; and system component visual data including a visual representation of at least one component of the system and indicating an actual position and orientation of each of the at least one component relative to the treatment tool.
[0053] In some embodiments, the treatment device is one of a focused ultrasound transducer and an interventional treatment device.
[0054] Some embodiments may provide a method for aiming and aligning a treatment tool in an X-ray machine environment, the method may include attaching a calibration plate to a treatment tool, the calibration plate may include at least one optical marker positioned at a predetermined position within the calibration plate; attaching a camera to the X-ray machine; acquiring a camera image with the camera including a visual representation of the at least one optical marker; determining a position and orientation of the treatment tool relative to the camera based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; and determining a position and orientation of the treatment tool relative to the X-ray machine based on the determined position and orientation of the calibration plate relative to the camera and the position and orientation of the camera relative to the X-ray machine.
[0055] In some embodiments, the position and orientation of the camera is known.
[0056] In some embodiments, the method may include mounting the camera to the X-ray device such that at least one radiopaque marker is at least partially located within a field of view of an X-ray source of the X-ray device, acquiring an X-ray image including a visual representation of the at least one radiopaque marker, and determining a position and orientation of the camera relative to the X-ray device based on the visual representation of the at least one radiopaque marker.
[0057] In some embodiments, the method may include displaying a visual indicator on a display that indicates the determined position and orientation of the treatment tool.
[0058] In some embodiments, the method may include comparing the determined position and orientation of the treatment tool relative to the X-ray device with a required position and orientation of the treatment tool relative to the X-ray device to determine whether it is within an acceptable error range.
[0059] In some embodiments, the method may include indicating whether a position and orientation of the treatment tool relative to the X-ray device is within the acceptable error range by modifying the visual indicator.
[0060] In some embodiments, the method may include superimposing the visual indicator on an x-ray image of a target area within a patient's body.
[0061] In some embodiments, the method may include detecting or marking the target area in the X-ray image, determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the X-ray image, and indicating whether the treatment tool is aligned with respect to the target area by modifying the visual indicator.
[0062] In some embodiments, the method may include generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool relative to the X-ray device, and displaying the treatment tool guidance instruction on the display.
[0063] In some embodiments, the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and the amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
[0064] In some embodiments, the method may include displaying at least one of: treatment tool visual data including a visual representation of at least a portion of the treatment tool; and system component visual data including a visual representation of at least one component of the system and indicating an actual position and orientation of each of the at least one component relative to the treatment tool.
[0065] In some embodiments, the method may include aiming and aligning the treatment tool according to the visual indicator displayed on the display without exposing a patient to X-ray imaging by the X-ray device.
[0066] Some embodiments may provide a system for aiming and aligning a treatment tool in an ultrasound imaging device environment, the system may include a calibration plate attachable to a treatment tool, the calibration plate may include at least one optical marker at a predetermined position within the calibration plate; a camera attachable to the ultrasound imaging probe at a predetermined position and orientation relative to the ultrasound imaging probe; and a processing unit in communication with the camera and an ultrasound imaging unit of the ultrasound imaging device, the processing unit being configured to receive a camera image from the camera including a visual representation of the at least one optical marker, determine a position and orientation of the calibration plate relative to the camera based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate, and determine a position and orientation of the treatment tool relative to the ultrasound imaging probe based on the predetermined position and orientation of the calibration plate relative to the treatment tool, the determined position and orientation of the calibration plate relative to the camera, and a known position and orientation of the camera relative to the ultrasound imaging probe.
[0067] In some embodiments, the system may include a display, and the processing unit is configured to present a visual indicator on the display indicative of the determined position and orientation of the treatment tool.
[0068] In some embodiments, the processing unit is configured to determine whether the determined position and orientation of the treatment tool relative to the ultrasound imaging probe is within an acceptable error range by comparing it with a required position and orientation of the treatment tool relative to the ultrasound imaging probe, and to indicate whether the position and orientation of the treatment tool relative to the ultrasound imaging probe is within the acceptable error range by modifying the visual indicator.
[0069] In some embodiments, the processing unit is configured to superimpose the visual indicator onto an ultrasound image of a target region within a patient's body.
[0070] In some embodiments, the processing unit is configured to detect or mark the target area in the ultrasound image, determine whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the ultrasound image, and indicate whether the treatment tool is aligned with respect to the target area by modifying the visual indicator.
[0071] In some embodiments, the processing unit is configured to generate treatment tool guidance instructions based on the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe, and to display the treatment tool guidance instructions on the display.
[0072] In some embodiments, the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and the amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
[0073] In some embodiments, the processing unit is configured to display at least one of: treatment tool visual data including a visual representation of at least a portion of the treatment tool; and system component visual data including a visual representation of at least one component of the system and indicating an actual position and orientation of each of the at least one component relative to the treatment tool.
[0074] In some embodiments, the treatment tool is one of a focused ultrasound transducer and an interventional instrument.
[0075] Some embodiments may provide a method for aiming and aligning a treatment tool in an ultrasound imaging device environment, the method may include attaching a calibration plate to a treatment tool, the calibration plate including at least one optical marker positioned at a predetermined position within the calibration plate; attaching a camera to an ultrasound imaging probe of an ultrasound imaging device at a predetermined position and orientation relative to the treatment tool; acquiring a camera image with the camera including a visual representation of the at least one optical marker; determining a position and orientation of the calibration plate relative to the camera based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; and determining a position and orientation of the treatment tool relative to the ultrasound imaging probe based on the predetermined position and orientation of the calibration plate relative to the treatment tool, the determined position and orientation of the calibration plate relative to the camera, and a known position and orientation of the camera relative to the ultrasound imaging probe.
[0076] In some embodiments, the method may include displaying a visual indicator on a display that indicates the determined position and orientation of the treatment tool.
[0077] In some embodiments, the method may include comparing the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe with a required position and orientation of the treatment tool relative to the ultrasonic imaging probe to determine whether it is within an acceptable error range.
[0078] In some embodiments, the method may include indicating whether a position and orientation of the treatment tool relative to the ultrasound imaging probe is within the acceptable error range by modifying the visual indicator.
[0079] In some embodiments, the method may include superimposing the visual indicator on an ultrasound image of a target region within a patient's body.
[0080] In some embodiments, the method may include detecting or marking the target area in the ultrasound image, determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the ultrasound image, and indicating whether the treatment tool is aligned with respect to the target area by modifying the visual indicator.
[0081] In some embodiments, the method may include generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe, and displaying the treatment tool guidance instruction on the display.
[0082] In some embodiments, the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and the amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
[0083] In some embodiments, the method may include displaying at least one of: treatment tool visual data including a visual representation of at least a portion of the treatment tool; and system component visual data including a visual representation of at least one component of the system and indicating an actual position and orientation of each of the at least one component relative to the treatment tool.
[0084] In some embodiments, the method may include aiming and aligning the treatment tool according to the visual indicator displayed on the display.
[0085] These additional and / or other aspects and / or advantages of the invention will be set forth in the detailed description that follows, or may be inferred from the detailed description, and / or may be learned by practice of the invention. [Brief description of the drawings]
[0086] For a better understanding of embodiments of the present invention and to show how they may be carried into effect, reference is now made to the accompanying drawings, which are merely exemplary and in which like numerals indicate corresponding elements or parts throughout.
[0087] [Figure 1] 1 is a schematic diagram of a first embodiment of a system for aiming and / or aligning a treatment tool in an X-ray environment, according to some embodiments of the present invention. [Figure 2A] 1 is a schematic diagram of a calibration plate of a system for aiming and / or aligning a treatment tool in an X-ray device environment, according to some embodiments of the present invention. [Figure 2B] 1 is a schematic diagram of a calibration plate of a system for aiming and / or aligning a treatment tool in an X-ray device environment, according to some embodiments of the present invention. [Figure 2C] 1 is an image of a calibration plate of a system for aiming and / or aligning a treatment tool in an X-ray environment, according to some embodiments of the present invention. [Diagram 3] 1 is an X-ray image of a calibration plate of a system for aiming and / or aligning a treatment tool in an X-ray environment, according to some embodiments of the present invention. [Figure 4] 1 is a camera image of a calibration plate of a system for aiming and / or aligning a treatment tool in an X-ray device environment according to some embodiments of the present invention. [Figure 5A] 1 is a schematic diagram of a system for aiming and / or aligning a treatment tool that needs to be aligned relative to an X-ray device in an X-ray device environment, according to some embodiments of the present invention. [Figure 5B] 1 is a schematic diagram of a system for aiming and / or aligning a treatment tool that needs to be aligned relative to an X-ray device in an X-ray device environment, according to some embodiments of the present invention. [Figure 5C] 1 is a schematic diagram of a system for aiming and / or aligning a treatment tool that needs to be aligned relative to an X-ray device in an X-ray device environment, according to some embodiments of the present invention. [Figure 5D]1 is a schematic diagram of a system for aiming and / or aligning a treatment tool in an X-ray environment having an X-ray aiming and alignment device according to some embodiments of the present invention. [Figure 5E] 1 is a schematic diagram of a visual representation of an X-ray aiming and alignment device for a treatment tool and visual indicators generated by a processing unit of a system for aiming and / or aligning a treatment tool in an X-ray device environment according to some embodiments of the present invention. [Figure 5F] 1 is a schematic diagram of a visual representation of an X-ray aiming and alignment device for a treatment tool and visual indicators generated by a processing unit of a system for aiming and / or aligning a treatment tool in an X-ray device environment according to some embodiments of the present invention. [Figure 5G] 1 is a schematic diagram of a system for aiming and / or aligning a treatment tool in an X-ray environment and a flowchart of a method of using the system according to some embodiments of the present invention. [Figure 6] FIG. 1 is a schematic diagram of a first embodiment of a system for projecting focused ultrasound energy, in accordance with some embodiments of the present invention. [Figure 7(1)] 4 is a flowchart of a first method for aiming and / or aligning a treatment tool in an X-ray device environment according to some embodiments of the present invention. [Figure 7(2)] 4 is a flowchart of a first method for aiming and / or aligning a treatment tool in an X-ray device environment according to some embodiments of the present invention. [Figure 7(3)] 4 is a flowchart of a first method for aiming and / or aligning a treatment tool in an X-ray device environment according to some embodiments of the present invention. [Figure 8] 1 is a schematic diagram of a first embodiment of a system for aiming and / or aligning a treatment tool in an ultrasound imaging environment, according to some embodiments of the present invention. [Figure 9A] 1 is a schematic diagram of a calibration plate of a system for aiming and / or aligning a treatment tool in an ultrasound imaging device environment according to some embodiments of the present invention. [Figure 9B]1 is a schematic diagram of a calibration plate of a system for aiming and / or aligning a treatment tool in an ultrasound imaging device environment according to some embodiments of the present invention. [Figure 10] 1 is a camera image of a calibration plate of a system for aiming and / or aligning a treatment tool in an ultrasound imaging device environment according to some embodiments of the present invention. [Figure 11] FIG. 2 is a schematic diagram of a second embodiment of a system for projecting focused ultrasonic energy, in accordance with some embodiments of the present invention. [Figure 12(1)] 1 is a flowchart of a first method for aiming and / or aligning a treatment tool in an ultrasound imaging environment according to some embodiments of the present invention. [Figure 12 (2)] 1 is a flowchart of a first method for aiming and / or aligning a treatment tool in an ultrasound imaging environment according to some embodiments of the present invention. [Figure 13] 1 is a schematic diagram of a calibration setup for a system for aiming and / or aligning a treatment tool in an ultrasound imaging environment according to some embodiments of the present invention. [Figure 14A] 1 is a flowchart of a method for guiding the aiming and / or alignment of a treatment tool in an imaging device environment according to some embodiments of the present invention. [Figure 14B] 1 is a schematic diagram of a guidance feature for guiding the aiming and / or alignment of a treatment tool in an imaging device environment according to some embodiments of the present invention. [Figure 15A] 1 is a schematic diagram of a second embodiment of a system for aiming and / or aligning a treatment tool in an X-ray environment, according to some embodiments of the present invention. [Figure 15B] 1 is a schematic diagram of a second embodiment of a system for aiming and / or aligning a treatment tool in an X-ray environment, according to some embodiments of the present invention. [Figure 16(1)] 11 is a flowchart of a second method for aiming and / or aligning a treatment tool in an X-ray device environment according to some embodiments of the present invention. [Figure 16(2)]11 is a flowchart of a second method for aiming and / or aligning a treatment tool in an X-ray device environment according to some embodiments of the present invention. [Figure 17] 1 is a schematic diagram of a second embodiment of a system for aiming and / or aligning a treatment tool in an ultrasound imaging environment, according to some embodiments of the present invention. [Figure 18(1)] 11 is a flowchart of a second method for aiming and / or aligning a treatment tool in an ultrasound imaging environment according to some embodiments of the present invention. [Figure 18(2)] 11 is a flowchart of a second method for aiming and / or aligning a treatment tool in an ultrasound imaging environment according to some embodiments of the present invention.
[0088] It will be appreciated that for simplicity and clarity of the drawings, elements in the figures have not necessarily been drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0089] In the following description, various aspects of the present invention will be described. For the purpose of explanation, specific configurations and details are set forth so that the present invention is fully understood. However, it will be apparent to those skilled in the art that the present invention may be practiced without the specific details presented herein. Also, well-known features may be omitted or simplified so as not to obscure the present invention. With particular reference to the drawings, it is emphasized that the details shown are presented by way of example only for the purpose of illustrative description of the present invention, and are presented for the reason of providing what is believed to be the most useful and easily understood explanation of the principles and conceptual aspects of the present invention. In this regard, no attempt is made to show structural details of the present invention in more detail than is necessary for a fundamental understanding of the present invention, and the description given with respect to the drawings will make clear to those skilled in the art how some forms of the present invention may be embodied in practice.
[0090] Before describing at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of elements set forth in the following specification or illustrated in the drawings. The invention is applicable to combinations of the disclosed embodiments as well as to other embodiments that can be practiced or carried out in various ways. It is also to be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting.
[0091] Unless otherwise indicated, and as will be apparent from the discussion that follows, discussions utilizing terms such as "processing," "operating," "calculating," "determining," "enhancing," and the like throughout the specification are understood to refer to operations and / or processing of a computer, computing system, or similar electronic computing device that manipulates and / or transforms data represented as physical quantities, such as electronic quantities, in registers and / or memory of the computing system into other data similarly represented as physical quantities in the memory, registers, or other such information storage, transmission, or display device of the computing system. Any of the disclosed modules or units may be implemented at least in part by a computer processor.
[0092] Reference is now made to FIG. 1, which is a schematic illustration of a first embodiment of a system 100 for aiming and / or aligning a treatment tool 90 in an X-ray machine 80 environment, according to some embodiments of the present invention.
[0093] According to some embodiments, the system 100 may include a calibration plate 110, a camera 120, and a processing unit 130 (e.g., as shown in FIG. 1). FIG. 1 illustrates a side view of the system 100. Compared to a treatment process performed without the system 100, the system 100 may enable targeting of the treatment tool 90 to a target region 72 within the patient 70 and / or alignment of the treatment tool 90 relative to the X-ray device 80 while significantly reducing exposure of the patient 70 to X-ray radiation.
[0094] The X-ray device 80 may be any type of fluoroscopy device, for example, a C-arm type, a G-arm type, or an O-arm type (e.g., a 9 inch, 12 inch or flat screen device), and may include an X-ray intensifier 82, an X-ray source 83, an X-ray imaging unit 84, and an X-ray display 86 (e.g., as shown in FIG. 1 ).
[0095] The treatment tool 90 may be, for example, an invasive treatment tool (such as a needle (e.g., a biopsy needle, a high-frequency needle) and / or a probe) that needs to be aimed at the target region 72 and / or aligned with the X-ray device 80, or a non-invasive treatment tool (such as an ultrasound transducer or a focused ultrasound transducer). For example, FIG. 1 shows a needle as an example of the treatment tool 90. As another example, FIGS. 5A to 5G and 6 show a focused ultrasound transducer as an example of the treatment tool 90.
[0096] According to some embodiments, the calibration plate 110 may include at least one radiopaque marker 112 and at least one optical marker 114. The radiopaque markers 112 and the optical markers 114 may be positioned at predetermined and / or known positions within the calibration plate 110 (e.g., around the calibration plate, and optionally on its surface). The calibration plate 110 may be attachable or removably attachable to the X-ray device 80 such that the calibration plate 110 is within the field of view 83a of the X-ray source 83. For example, the calibration plate 110 may be attachable or removably attachable to the X-ray intensifier 82 of the X-ray device 80. Various embodiments of the calibration plate 110 are described below with reference to Figures 2A, 2B, and 2C.
[0097] In some embodiments, the camera 120 is attachable or removably attachable to the treatment tool 90. The camera 120 may be attachable to the treatment tool 90 at a predetermined and / or known position and orientation relative to the treatment tool 90.
[0098] The camera 120 may be mountable to the treatment tool 90 such that, for example, at least a portion of the calibration plate 110 is within the field of view 121 of the camera 120 during the treatment process and / or such that the camera 120 is outside (or substantially outside) the field of view 83a of the X-ray device 80 (e.g., as shown in FIG. 1).
[0099] In some embodiments, the system 100 may include a camera connector 122. The camera connector 122 may be configured to securely and stably attach the camera 120 to the procedure tool 90 and prevent unintended movement of the camera 120 relative to the procedure tool 90. In some embodiments, the camera connector 122 may be formed of a radiolucent material such that the camera connector 122 does not interfere with x-ray imaging performed by the x-ray device 80.
[0100] In some embodiments, the camera connector 122 can allow for controlled rotation of the camera 120. This allows it to follow the calibration plate 110, for example, when the treatment tool 90 moves or when the C-arm of the x-ray machine 80 tilts 90 degrees to capture a lateral view of the patient's anatomy.
[0101] In some embodiments, the system 100 may include a second camera. The second camera may be securely and stably attached (e.g., using a connector such as connector 122) to the treatment tool 90 at a predetermined angle relative to the camera 120. For example, the second camera may be attached to the treatment tool 90 at a 90° angle relative to the camera 120. This allows the second camera to capture the calibration plate 110 when it moves out of the field of view 121 of the camera 120, for example, due to the treatment tool 90 moving or the C-arm tilting 90 degrees to capture a side view of the patient's anatomy.
[0102] In some embodiments, the processing unit 130 may be in communication (eg, wired or wireless) with the camera 120 and the X-ray imaging unit 84 of the X-ray device 80 .
[0103] In some embodiments, processing unit 130 may be configured to receive one or more x-ray images of calibration plate 110 from radiography unit 84. The x-ray image of calibration plate 110 may include a visual representation of radiopaque markers 112 (e.g., as shown in and described below with respect to FIG. 3).
[0104] According to some embodiments, the processing unit 130 may be configured to determine a position and orientation of the calibration plate 110 relative to the X-ray device 80 based on a visual representation of the radiopaque markers 112, a known position of the radiopaque markers 112 within the calibration plate 110, and specific parameters or a specific model of the X-ray device 80. The specific parameters / model of the X-ray device 80 may include, for example, a distance value of the X-ray source 83 relative to the X-ray intensifier 82 and a size of the field of view of the X-ray intensifier 82 (e.g., a metric size of an image pixel on the X-ray intensifier 82).
[0105] In some embodiments, the processing unit 130 may be configured to identify a visual representation of the radiopaque markers 112 in an x-ray image of the calibration plate 110. The identification may be utilized, for example, using a pattern matching algorithm.
[0106] In some embodiments, the processing unit 130 may be further configured to determine one or more vectors extending from the X-ray source 83 towards the X-ray intensifier 82 and the calibration plate 110 attached thereto, and intersecting with the calibration plate 110. The vectors may represent, for example, the X-rays generated by the X-ray source 83. The determination of the vectors may be based, for example, on a particular model of the X-ray device 80.
[0107] In some embodiments, processing unit 130 may be further configured to determine one or more corresponding intersection points of the one or more vectors with calibration plate 110.
[0108] In some embodiments, the processing unit 130 may be further configured to compare the predetermined / known positions of the radiopaque markers 112 within the calibration plate 110 with the intersection of the determined vector with the calibration plate 110. The comparison may be utilized, for example, using a point cloud matching algorithm (e.g., a brute force algorithm, an iterative closest point algorithm).
[0109] In some embodiments, the processing unit 130 may be further configured to determine the position and orientation of the calibration plate 110 relative to the X-ray device 80 based on a comparison of the predetermined / known positions of the radiopaque markers 112 in the calibration plate 110 and the intersection of the determined vector with the calibration plate 110. The determination may be utilized using a registration algorithm, such as, for example, a singular value decomposition (SVD) algorithm.
[0110] According to some embodiments, processing unit 130 may be configured to receive, from camera 120, one or more camera images of calibration plate 110. The camera images of calibration plate 110 may include visual representations of optical markers 114 (e.g., as shown in and described below with respect to FIG. 4).
[0111] According to some embodiments, processing unit 130 may be configured to determine the position and orientation of camera 120 relative to calibration plate 110 (which may be mounted, for example, on x-ray intensifier 82) based on a visual representation of optical markers 114 in the camera image, known positions of optical markers 114 in calibration plate 110, and parameters of camera 120 (e.g., distortion, field of view 121, etc.). The determination may be utilized, for example, using a bundle adjustment / PnP algorithm.
[0112] According to some embodiments, the processing unit 130 may be configured to determine the position and orientation of the treatment tool 90 relative to the X-ray device 80 based on the determined position and orientation of the calibration plate 110 relative to the X-ray device 80, the determined position and orientation of the camera 120 relative to the calibration plate 110, and the known position and orientation of the camera 120 relative to the treatment tool 90.
[0113] According to some embodiments, the system 100 may include a display 140. The processing unit 130 may be configured to present at least one visual indicator 142 indicative of the determined position and orientation of the treatment tool 90, for example, on the display 140 and / or the X-ray display 86.
[0114] In some embodiments, the processing unit 130 may be configured to determine whether the determined position and orientation of the treatment tool 90 relative to the X-ray device 80 is within an acceptable error range, compared to a required position and orientation of the treatment tool 90 relative to the X-ray device 80. In some embodiments, the processing unit 130 may be configured to indicate whether the position and orientation of the treatment tool 90 is within an acceptable error range, by updating the visual indicator 142. Changing the visual indicator 142 may include, for example, changing the color and / or shape of the visual indicator 142 (e.g., according to a predetermined rule).
[0115] In various embodiments, the processing unit 130 may be configured, for example, to superimpose a visual indicator 142 on an X-ray image of the target area 72 of the patient 70 to generate a superimposed image and display the superimposed image on the display 140 and / or the X-ray display 86.
[0116] In some embodiments, the processing unit 130 may be configured to detect or mark the target region 72 in the X-ray image. In some embodiments, the marking may be based on user input. The processing unit 130 may be configured to determine whether the treatment tool 90 is aligned with respect to the target region 72 based on the determined position and orientation of the treatment tool 90 and the X-ray image. In some embodiments, the processing unit 130 may be configured to indicate whether the treatment tool 90 is aligned with respect to the target region 72 by updating the visual indicator 142.
[0117] In some embodiments, the processing unit 130 may be configured to generate tool guidance instructions based on the determined position and orientation of the treatment tool 90 relative to the X-ray device 80 (e.g., as described below with respect to FIGS. 14A and 14B ), and display the tool guidance instructions on the display 86 / 140. The guidance instructions may indicate one or more directions and / or an amount of movement in each direction in which the treatment tool 90 should be moved (e.g., displaced and / or tilted) to bring the treatment tool 90 into a position and orientation within an acceptable error range. The tool guidance instructions may, for example, make it easy and intuitive to aim and / or align the treatment tool 90 relative to the target region 72 and / or the X-ray device 80.
[0118] The system 100 allows a user (e.g., a treating physician) to aim the treatment tool 90 at the target area 72 and / or align (e.g., translate, tilt, pitch, yaw, roll, etc.) the treatment tool 90 relative to the x-ray machine 80 using visual indicators 142 displayed on the display 140 and / or x-ray display 86. The visual indicators 142 can indicate the position and orientation of the treatment tool 90 relative to the x-ray machine 80, which eliminates (or substantially eliminates) the need for x-rays of patent 70 during the aiming and / or alignment of the treatment tool 90. When utilizing the system 100, x-rays may only be required at the final stages of aiming and / or aligning the treatment tool 90 to verify that the actual position and orientation of the treatment tool 90 relative to the x-ray machine 80 corresponds to or is within an acceptable error range for the required position and orientation prior to administering treatment. This enables the system 100 to significantly reduce the patient's and / or user's exposure to X-ray radiation during the treatment process, as compared to current treatment processes that require multiple exposures of the patient 70 to X-ray radiation during aiming and / or positioning of the treatment tool 90.
[0119] Reference is made to FIGS. 2A and 2B, which are schematic illustrations of a calibration plate 200 that may be used in a system (such as system 100) for aiming and / or aligning a treatment tool 90 in an X-ray machine 80 environment, according to some embodiments of the present invention.
[0120] Reference is also made to FIG. 2C, which is an image of a calibration plate 200 that may be used in a system (such as system 100) for aiming and / or aligning a treatment tool 90 in an X-ray machine 80 environment, according to some embodiments of the present invention.
[0121] According to some embodiments, the calibration plate 200 may be used in a system (such as the system 100 described above with respect to FIG. 1) for aiming and / or aligning a treatment tool 90 in the environment of an x-ray machine 80. For example, the calibration plate 200 may be the calibration plate 110 described above with respect to FIG.
[0122] 2A, 2B and 2C show a surface of a calibration plate 200 that is marked with optical and / or radiopaque markers.
[0123] According to some embodiments, the calibration plate 200 (e.g., as shown in FIG. 2A ) may include at least one radiopaque marker 210 at a known location within the calibration plate 200. The radiopaque marker 210 may be asymmetric about at least one of its axes. For example, FIG. 2A shows a single radiopaque marker 210 having an arrow shape (e.g., asymmetric about its horizontal axis). In some embodiments, the radiopaque marker 210 may be the radiopaque marker 112 described above with respect to FIG. 1 .
[0124] In the embodiment shown in FIG. 2A, the radiopaque marker 210 may be of any asymmetric shape that enables determination / identification of the position and orientation of the calibration plate 200 (e.g., relative to the X-ray machine 80) based on a visual representation of the radiopaque marker 210 in an X-ray image of the calibration plate 200 (e.g., as described above with respect to FIG. 1).
[0125] In some embodiments, the calibration plate 200 may include a plurality of radiopaque markers 220 (e.g., as shown in FIGS. 2B and 2C). The radiopaque markers 220 may be symmetrical (e.g., about at least one of its axes) or asymmetrical. For example, the radiopaque markers 220 may be spheres (e.g., symmetrical markers) or arc-shaped arrows (e.g., asymmetrical markers). The radiopaque markers 220 may be asymmetrically distributed within the calibration plate 220 (e.g., about at least one of the axes of the calibration plate 220). In some embodiments, the radiopaque markers 220 may be the radiopaque markers 112 described above with respect to FIG. 1.
[0126] In the embodiment shown in Figures 2B and 2C, the position of the radiopaque markers 220 within the calibration plate 200 may be predetermined (e.g., as described above with respect to Figure 1) to enable determination / identification of the position and orientation of the calibration plate 200 (e.g., relative to the X-ray device 80) based on a visual representation of the radiopaque markers 220 in an X-ray image of the calibration plate 200.
[0127] In some embodiments, a processing unit (e.g., processing unit 130 described above with respect to FIG. 1 ) may be configured to detect and correct, based on the visual representation of the radiopaque markers 220 in an x-ray image of the calibration plate 200, a rotation and / or inversion of the x-ray image relative to the visual representation of the radiopaque markers 220 in a previous x-ray image of the calibration plate 200. In some embodiments, the detection of the rotation and / or inversion of the x-ray image may be performed for each obtained x-ray image, as the detection is fast in terms of computation time.
[0128] According to some embodiments, the calibration plate 200 (e.g., as shown in FIGS. 2A, 2B, and 2C) includes a plurality of optical markers 230 positioned at predetermined locations within the calibration plate 200. In some embodiments, the optical markers 230 may be the optical markers 114 described above with respect to FIG.
[0129] In some embodiments, each optical marker 230 may include its own unique visual label. The visual label of the optical marker 230 may include, for example, a barcode, a QR code, a graphic pattern or shape, an alcomarker, an april tag, an AR tag, etc. that may have additional data associated with the visual label. The visual label may encode, for example, the location of the optical marker 230 within the calibration plate 200.
[0130] Parameters of the optical markers 230, such as the amount of optical markers 230, the dimensions of the optical markers 230, and / or the visual labels of the optical markers 230, may be determined based on, for example, the resolution of the camera 120. The parameters of the optical markers 230 may further be determined to enable determination / identification of the position and orientation of a camera (e.g., camera 120, discussed above with respect to FIG. 1) relative to the calibration plate 200 based on a visual representation of the optical markers 230 in a camera image of the calibration plate 200 (e.g., as discussed above with respect to FIG. 1).
[0131] According to various embodiments, the calibration plate 200 may be made from a radiolucent and / or rigid material. For example, the calibration plate 200 may be made from plexiglass, cardboard, foam board, and / or adhesive to prevent deformation of the calibration plate 200 and thus displacement of the radiopaque markers 210, 220, and optical markers 230 from their predetermined positions.
[0132] In various embodiments, radiopaque markers (e.g., radiopaque markers 210 and / or 220) may be embedded in or attached to a surface of calibration plate 200 (e.g., as shown in FIG. 2C). In some embodiments, optical markers 230 may be printed on a surface of calibration plate 200 (e.g., as shown in FIG. 2C).
[0133] In some embodiments, the calibration plate 200 may have a matte surface that can, for example, reduce light reflections from light sources in the operating room and eliminate (or substantially eliminate) glare in the camera image due to the light reflections, thereby avoiding loss of information derived from the optical markers 230.
[0134] Reference is now made to FIG. 3, which is an X-ray image 300 of a calibration plate (such as calibration plate 110, 200) of a system (such as system 100) for aiming and / or aligning a treatment tool 90 in an X-ray device 80 environment, according to some embodiments of the present invention.
[0135] 3, for example, illustrates an x-ray image 300 of a calibration plate (e.g., calibration plate 200 shown in FIG. 2C). X-ray image 300 can be acquired using x-ray device 80 before or during a treatment process (e.g., as described above with respect to FIG. 1). X-ray image 300 includes a visual representation 310 of a radiopaque marker (e.g., radiopaque marker 220 shown in FIG. 2C) asymmetrically positioned at a predetermined location within the calibration plate.
[0136] Reference is now made to FIG. 4, which is a camera image 400 of a calibration plate (such as calibration plate 110, 200) of a system (such as system 100) for aiming and / or aligning a treatment tool 90 in an X-ray device 80 environment, according to some embodiments of the present invention.
[0137] 4, for example, shows a camera image 400 of a calibration plate (e.g., calibration plate 200 shown in FIG. 2C). The camera image 400 can be acquired, for example, during a treatment process using a camera (e.g., camera 120 described above with respect to FIG. 1) attached to a treatment tool 90. The camera image 400 may include a visual representation 410 of an optical marker (e.g., optical marker 230 shown in FIG. 2C) positioned at a predetermined location within the calibration plate.
[0138] Reference is now made to Figures 5A, 5B and 5C, which are schematic illustrations of a system 500 for aiming and / or aligning a treatment tool 90 that needs to be aligned relative to an X-ray device 80 in an X-ray device 80 environment, in accordance with some embodiments of the present invention.
[0139] Some treatment tools 90 need to be aligned with respect to the X-ray device 80 to safely administer the treatment. For example, the required position and orientation of such treatment tools (e.g., as described above with respect to FIG. 1) may include alignment of a central longitudinal axis 90a of the treatment tool 90 with respect to a central longitudinal axis 81 of the X-ray device 80 (e.g., an axis extending between the center of the X-ray source 83 and the center of the X-ray intensifier 82). An example of such a treatment tool 90 may include, for example, a focused ultrasound (FUS) transducer that projects focused ultrasound energy 98 to a target region 72 within a patient 70, such as a human patient or any other mammal, as shown generally in FIGS. 5A, 5B, and 5C.
[0140] For example, Figure 5A shows a treatment tool 90 aligned along a central longitudinal axis 90a relative to the central longitudinal axis 81 of the x-ray device 80. Figures 5B and 5C show a treatment tool 90 misaligned relative to the central longitudinal axis 81 of the x-ray device 80.
[0141] According to some embodiments, system 500 (e.g., as shown in FIG. 5A ) may include a calibration plate 510, a camera 520, and a processing unit 530. For example, system 500 may be a system such as system 100 described above with respect to FIG.
[0142] According to some embodiments, the calibration plate 510 may be attachable or removably attachable to the X-ray device 80 such that the calibration plate 510 is within the field of view 83a of the X-ray source 83. For example, the calibration plate 510 may be attachable or removably attachable to the X-ray intensifier 82 of the X-ray device 80.
[0143] For example, calibration plate 510 may be calibration plate 110 (described above with respect to FIG. 1) or calibration plate 200 (described above with respect to FIGS. 2A, 2B, and 2C). Radiopaque marker 512 may be radiopaque marker 112 (described above with respect to FIG. 1), radiopaque marker 210 (described above with respect to FIG. 2A), or radiopaque marker 220 (described above with respect to FIGS. 2B and 2C). Optical marker 514 may be optical marker 114 (described above with respect to FIG. 1) or optical marker 230 (described above with respect to FIGS. 2A, 2B, and 2C).
[0144] According to some embodiments, the camera 520 may be attachable or removably attachable to the treatment tool 90 (e.g., using the camera connector 522) at a predetermined / known position and orientation relative to the treatment tool 90. For example, the camera 520 and the camera connector 522 may be a camera and a connector, respectively, such as the camera 120 and the camera connector 122 described above with respect to FIG.
[0145] According to some embodiments, processing unit 530 may be in communication (wired or wireless) with radiography unit 84 and camera 520. For example, processing unit 530 may be processing unit 130 described above with respect to FIG.
[0146] The processing unit 530 may be configured to determine the position and orientation of the treatment tool 90 relative to the X-ray device 80 (eg, as described above with respect to FIG. 1).
[0147] In various embodiments, the processing unit 530 may be configured to determine, based on the determined position and orientation of the treatment tool 90 relative to the X-ray device 80, whether the central longitudinal axis 90a of the treatment tool 90 is aligned along or misaligned relative to the central longitudinal axis 81 of the X-ray device 80, and / or whether the misalignment is within an acceptable error range (e.g., the acceptable error range described above with respect to FIG. 1).
[0148] According to various embodiments, the processing unit 530 may be configured to display, for example, on the display 540 and / or the X-ray display 86, a visual indicator 542 indicative of the determined position and orientation of the treatment tool 90 relative to the X-ray device 80 (e.g., as described above with respect to FIG. 1). The visual indicator 542 may be the visual indicator 142 described above with respect to FIG.
[0149] The visual indicator 542 may further be configured to indicate whether the central longitudinal axis 90a of the treatment tool 90 is aligned along or misaligned with respect to the central longitudinal axis 81 of the X-ray device 80, and / or whether the misalignment is within an acceptable error range.
[0150] In some embodiments, the visual indicator 542 may include a first visual element 542a and a second visual element 542b (e.g., as shown in FIGS. 5A, 5B, and 5C). The first visual element 542a and the second visual element 542b may have different shapes, dimensions, or other visual parameters (e.g., color, line width, etc.). For example, the first visual element 542a and the second visual element 542b may have any combination of a ring-shaped marker (e.g., an "O"), a cross-shaped marker (e.g., a "+"), etc.
[0151] For example, if the processing unit 530 determines that the central longitudinal axis 90a of the treatment tool 90 is aligned along the central longitudinal axis 81 of the X-ray device 80, the first visual element 542a and the second visual element 542b may coincide with each other when displayed on the display 540 and / or the X-ray display 86.
[0152] Also, in this example, if the processing unit 530 determines that the central longitudinal axis 90a of the treatment tool 90 is misaligned with respect to the central longitudinal axis 81 of the X-ray device 80, the first visual element 542a and the second visual element 542b will not coincide with each other when displayed on the display 540 and / or the X-ray display 86.
[0153] In some embodiments, the processing unit 530 may be configured to update visual parameters of the visual indicator 542 (or the first visual element 542a and the second visual element 542b) based on the determined position and orientation of the treatment tool 90 relative to the X-ray device 80.
[0154] For example, when the central longitudinal axis 90a of the treatment tool 90 is aligned along the central longitudinal axis 81 of the X-ray device 80, or when the misalignment is within an acceptable error range, the visual indicator 542 may be, for example, green.
[0155] Also, in this example, if the misalignment between the central longitudinal axis 90a of the treatment tool 90 and the central longitudinal axis 81 of the X-ray device 80 is not within the allowable error range, the visual indicator 542 may be, for example, red.
[0156] 5A, 5B and 5C show non-limiting examples of ring-shaped first and second visual elements 542a, 542b, where the diameter of the second visual element 542b is smaller than the diameter of the first visual element 542a.
[0157] In this example, if the processing unit 530 determines that the central longitudinal axis 90a of the treatment tool 90 is aligned along the central longitudinal axis 81 of the X-ray device 80 (e.g., as shown in FIG. 5A), the first visual element 542a and the second visual element 542b may coincide with each other when displayed on the display 540 and / or the X-ray display 86.
[0158] Also, in this example, if the processing unit 530 determines that the central longitudinal axis 90a of the treatment tool 90 is misaligned with respect to the central longitudinal axis 81 of the X-ray device 80 (e.g., as shown in FIG. 5B), and if the misalignment is within an acceptable error range, the second visual element 542b may be positioned entirely within, but not coincident with, the first visual element 542a when displayed on the display 540 and / or the X-ray display 86.
[0159] Also, in this example (e.g., as shown in FIG. 5C , if the processing unit 530 determines that the misalignment between the central longitudinal axis 90a of the treatment tool 90 and the central longitudinal axis 81 of the X-ray device 80 is not within the acceptable error range, the second visual element 542b may only partially overlap or not at all with the first visual element 542a when displayed on the display 540 and / or the X-ray display 86.
[0160] Reference is now made to FIG. 5D, which is a schematic illustration of a system 500 for aiming and / or aligning a treatment tool 90 in an X-ray device 80 environment having an X-ray aiming and alignment device 92, according to some embodiments of the present invention.
[0161] Also, reference is made to Figures 5E and 5F, which are schematic illustrations of a visual representation 85 of an X-ray aiming and alignment device 92 of a treatment tool 90 and a visual indicator 542 generated by a processing unit 530 of a system 500 for aiming and / or aligning a treatment tool 90 in an X-ray device 80 environment, according to some embodiments of the present invention.
[0162] Some treatment tools that need to be aligned with the X-ray device 80 prior to administering treatment (e.g., as described above with respect to Figures 5A, 5B, and 5C) may include an X-ray aiming and alignment device 92 (e.g., as shown in Figure 5D).
[0163] The X-ray aiming and alignment device 92 may include, for example, two radiopaque surfaces (e.g., a first radiopaque surface 92a, a second radiopaque surface 92b) that are parallel and congruent to one another and spaced apart along a central longitudinal axis 92c of the X-ray aiming and alignment device 92 (e.g., as shown in FIG. 5D ). The X-ray aiming and alignment device 92 may be mountable to the treatment device 90 such that the central longitudinal axis 92c of the X-ray aiming and alignment device is aligned along the central longitudinal axis 90a of the treatment device 90.
[0164] A visual representation 85 in the X-ray image of the X-ray targeting and alignment device 92 (e.g., visual representation 85a of the first surface 92a and visual representation 85b of the second surface 92b) can provide an indication of whether the position and orientation of the treatment tool 90 relative to the X-ray device 80 is within an acceptable error range compared to the required position and orientation.
[0165] For example, if the position and orientation of the treatment tool 90 relative to the X-ray device 80 is within an acceptable error range compared to the required position and orientation (e.g., as shown in FIG. 5D ), the visual representation 85a of the first surface 92a may match (or substantially match) a visual representation 85b of the second surface 92b in the X-ray image of the X-ray targeting and alignment device 92 (e.g., as displayed on the X-ray display 86).
[0166] In some embodiments, processing unit 530 may generate a visual indicator 542 corresponding to X-ray aiming and alignment device 92. For example, first visual marker 542a and second visual marker 542b of visual indicator 542 may have the same (or substantially the same) shape and / or dimensions as visual representation 85a of first surface 92a and visual representation 85b of second surface 92b of X-ray aiming and alignment device 92, respectively (e.g., as shown in FIG. 5D ).
[0167] According to various embodiments, processing unit 530 may be configured to display visual indicator 542 on display 540 and / or x-ray display 86 (e.g., as shown in FIG. 5D ). In various embodiments, processing unit 530 may be configured to display both visual representation 85 of x-ray targeting and alignment device 92 and visual indicator 542 on display 540 and / or x-ray display 86 (e.g., as shown in FIGS. 5E and 5F ).
[0168] In various embodiments, the processing unit 530 may be configured to superimpose the visual representation 85 and visual indicator 542 of the X-ray targeting and / or alignment device 92 onto the X-ray image of the target area 72 of the patient 70 displayed on the display 540 and / or the X-ray display 86.
[0169] In some embodiments, the processing unit 530 may be configured to detect or mark the target region 72 in the x-ray image. The processing unit 530 may be configured to determine whether the treatment tool 90 is aligned with respect to the target region 72 based on the determined position and orientation of the treatment tool 90 and the x-ray image. In some embodiments, the processing unit 530 may be configured to indicate whether the treatment tool 90 is aligned with respect to the target region 72 by updating the visual indicator 542.
[0170] According to some embodiments, the determined position and orientation of the treatment tool 90 relative to the X-ray device 80 may differ from the actual position and orientation by a certain error value. The certain error value may depend, for example, on an error in the metric resolution of the X-ray intensifier 82, an error in the distance value between the X-ray source 83 and the X-ray intensifier 82, an incorrect connection of the calibration plate 510 to the X-ray intensifier 82 (e.g., the calibration plate 510 is not parallel to the X-ray intensifier 82), and / or an incorrect connection of the camera 520 to the treatment tool 90 (e.g., the camera 520 is not in a predefined position and orientation relative to the treatment tool 90).
[0171] In some embodiments, the processing unit 530 may be configured to determine the position and orientation of the treatment tool 90 relative to the X-ray device 80 to have a specified error value of 1 millimeter or less.
[0172] If the determined position and orientation of the treatment tool 90 relative to the X-ray device 80 differs from the actual position and orientation, the visual indicator 542 may be moved relative to the visual representation 85 of the X-ray aiming and alignment device 92 (e.g., representing the actual position and orientation), as shown generally in FIG. 5F.
[0173] In some embodiments, the processing unit 530 may be configured to minimize the error by updating the determined position and orientation of the treatment tool 90 relative to the X-ray device 80 based on the visual representation 85 of the X-ray aiming and alignment device 92 to obtain an updated position and orientation of the treatment tool 90 relative to the X-ray device 80. This updating may be utilized, for example, by identifying radiopaque markers (e.g., the first surface 92a and the second surface 92b) of the X-ray aiming and alignment device 92, estimating the position of the treatment region 72 in the space between the X-ray source 83 and the X-ray intensifier 82 within the field of view 83a based on the position of the radiopaque markers, and further superimposing the visual representation of the radiopaque markers on the X-ray image.
[0174] In some embodiments, the processing unit 530 may be configured to generate tool guidance instructions based on the determined position and orientation of the treatment tool 90 relative to the X-ray device 80 (e.g., as described below with respect to FIGS. 14A and 14B ), and display the tool guidance instructions on the display 86 / 540. The guidance instructions may indicate one or more directions and / or an amount of movement in each direction in which the treatment tool 90 should be moved (e.g., displaced and / or tilted) to bring the treatment tool 90 into a position and orientation within an acceptable error range. The tool guidance instructions may, for example, make it easy and intuitive to aim and / or align the treatment tool 90 relative to the target region 72 and / or the X-ray device 80.
[0175] It should be noted that treatment may be administered without relying on virtual indicator 542 and based solely on the visual representation 85 of x-ray targeting and alignment device 92 and the x-ray image of target area 72 .
[0176] Reference is now made to FIG. 5G, which is a schematic illustration of a system 500 for aiming and / or aligning a treatment tool 90 in an X-ray device 80 environment and a flowchart of a method of using the system 500, according to some embodiments of the present invention.
[0177] It should be noted that the method is not limited to the flowchart depicted in Figure 5G and the corresponding description, for example, in various embodiments, the method need not proceed through each box or stage depicted, or in the exact same order depicted and described.
[0178] Stage 590 of the method may include, for example, aiming the tool 90 at the target region 72 within the patient 70 and / or aligning the tool 90 relative to the x-ray device 80 based on a visual indicator 542 (e.g., displayed on the display 540) indicating the determined position and orientation of the tool 90 relative to the x-ray device 80 (e.g., as described above with respect to Figures 5A, 5B, 5C, 5D, 5E, and 5F). The aiming and / or alignment may include, for example, translation, tilt, pitch, yaw, roll, etc. of the tool 90.
[0179] For example, stage 590 of the method may be repeated until visual indicator 542 indicates that the position and orientation of treatment tool 90 relative to X-ray device 80 is within an acceptable error range compared to the required position and orientation (e.g., as described above with respect to Figures 5A, 5B, 5C, 5D, 5E and 5F).
[0180] Stage 590 of the method may be performed without obtaining an x-ray image of the tool 90 / x-ray aiming and alignment device 92 (e.g., as described above with respect to Figures 5A, 5B, 5C, 5D, 5E, and 5F), for example, based solely on visual indicator 542. This is in contrast to current treatment processes that do not utilize system 500, for example, and require frequent x-rays of the tool 90 and patient 70 during the aiming and / or alignment stages.
[0181] Stage 592 of the method may include, for example, acquiring an X-ray image of the X-ray aiming and alignment device 92 of the treatment tool 90 and displaying a visual representation 85 of the X-ray aiming and alignment device 92, optionally together with a visual indicator 542, e.g., on a display 540 (e.g., as described above with respect to Figures 5A, 5B, 5C, 5D, 5E and 5F).
[0182] For example, as described above with respect to Figures 5A, 5B, 5C, 5D, 5E and 5F, stage 594 of the method may include determining whether the actual position and orientation of the treatment tool 90 is within an acceptable error range relative to a required position and orientation, for example based on a visual representation 85 of the X-ray targeting and alignment device 92.
[0183] If the actual position and orientation of the treatment tool 90 is within the tolerance range, the method may proceed to stage 596 which may include, for example, administering treatment with the treatment tool 90 .
[0184] If the actual position and orientation of the treatment tool 90 is not within the tolerance range, the method may include repeating stage 590.
[0185] Reference is now made to FIG. 6, which is a schematic illustration of a first embodiment of a system 600 for projecting focused ultrasound energy, in accordance with some embodiments of the present invention.
[0186] According to some embodiments, the system 600 may include a focused ultrasound (FUS) transducer 610 (e.g., the treatment tool 90 described above with respect to Figures 5A-5G) arranged to generate FUS energy 619.
[0187] According to some embodiments, the system 600 may include an X-ray aiming and alignment device 612 (eg, the X-ray aiming and alignment device 92 described above with respect to Figures 5D-5G) that is attachable to the FUS transducer 610.
[0188] According to some embodiments, the system 600 may include a support 620 suitable for housing the FUS transducer 610. In some embodiments, the support 622 may be handheld. In some embodiments, the system 600 may include an articulated arm 622 having a first end coupled to the table 88 and a second end coupled to the support 620. The articulated arm 622 may be positioned to allow translation, tilt, pitch, yaw, and / or roll of the support 620 and the FUS transducer 610 housed therein.
[0189] According to some embodiments, the system 600 may include an acoustic coupler 640 positioned to acoustically couple the FUS transducer 610 to the patient 70 to enable delivery of FUS energy 619 to a target region 72 within the patient 70.
[0190] According to some embodiments, the system 600 may include a calibration plate 650 (e.g., the calibration plate 110 described above with respect to FIG. 1, or the calibration plate 200 described above with respect to FIGS. 2A-2C) that is attachable or removably attachable to the X-ray device 80, e.g., the X-ray intensifier 82.
[0191] According to some embodiments, the system 600 may include a camera 660 (e.g., camera 120 described above with respect to FIG. 1, or camera 520 described above with respect to FIGS. 5A-5G) that is removably mountable to the support 620 / FUS transducer 610.
[0192] According to some embodiments, system 600 may include a processing unit 670 (e.g., processing unit 130 described above with respect to FIG. 1 or processing unit 530 described above with respect to FIGS. 5A-5G). Processing unit 670 may be configured to determine a position and orientation of FUS transducer 610 relative to x-ray device 80 (e.g., as described above with respect to FIGS. 5A-5G).
[0193] According to some embodiments, the system 600 may include a display 680. The processing unit 670 may be configured to display a visual indicator 682 (e.g., visual indicator 542 described above with respect to FIGS. 5A-5G ) on the display 680 and / or the X-ray display 86 indicative of the determined position and orientation of the FUS transducer 610.
[0194] In various embodiments, the processing unit 670 may be configured, for example, to superimpose a visual indicator 682 on an X-ray image of the target area 72 of the patient 70 to generate a superimposed image and display the superimposed image on the display 680 and / or the X-ray display 86.
[0195] In some embodiments, the processing unit 670 may be configured to detect or mark the target region 72 in the x-ray image. The processing unit 670 may be configured to determine whether the treatment tool 90 is aligned with respect to the target region 72 based on the determined position and orientation of the treatment tool 90 and the x-ray image. In some embodiments, the processing unit 670 may be configured to indicate whether the treatment tool 90 is aligned with respect to the target region 72 by updating the visual indicator 682.
[0196] In some embodiments, the processing unit 670 may be configured to generate tool guidance instructions based on the determined position and orientation of the tool 90 relative to the X-ray device 80 and display the tool guidance instructions on the display 86 / 680 (e.g., as described below with respect to FIGS. 14A and 14B ). The tool guidance instructions may, for example, make aiming and / or alignment of the tool 90 relative to the target region 72 and / or the X-ray device 80 easy and intuitive.
[0197] The system 600 allows a user (e.g., a treating physician) to aim the FUS transducer 610 at a target region 72 within the patient 70 and / or align the FUS transducer 610 relative to the x-ray machine 80 using visual indicators 682 displayed on the display 680 and / or x-ray display 86. The visual indicators 682 can indicate the determined position and orientation of the FUS transducer 610 relative to the x-ray machine 80, which eliminates (or substantially eliminates) the need for x-rays of the patent 70 during the aiming and / or alignment of the FUS transducer 610. When utilizing the system 600 (e.g., as described above with respect to FIGS. 5A-5G), x-rays may only be required at the final stages of aiming and / or alignment of the FUS transducer 610 to verify that the actual position and orientation of the FUS transducer 610 relative to the x-ray machine 80 corresponds to or is within an acceptable error range as required prior to administering treatment. As such, system 600 allows for a significant reduction in the patient's and / or user's exposure to x-ray radiation during the treatment process, as compared to current treatment processes that require multiple exposures of the patient 70 to x-ray radiation during the aiming and / or alignment of the FUS transducer 610. System 600 also allows for a significant reduction in the overall duration of the treatment process, as compared to current treatment processes that do not utilize system 600.
[0198] Reference is now made to FIG. 7, which is a flow chart of a first method for aiming and / or aligning a treatment tool in an X-ray environment, according to some embodiments of the present invention.
[0199] The method may be performed by a system for aiming and aligning a treatment tool in an x-ray environment (such as system 100 described above with respect to FIG. 1 or system 500 described above with respect to FIGS. 5A-5G), which may be configured to perform the method. It should be noted that the method is not limited to the flowchart shown in FIG. 7 and the corresponding description. For example, in various embodiments, the method need not proceed through each box or stage shown or in the exact same order as shown and described.
[0200] According to some embodiments, the method includes mounting (or removably mounting) a calibration plate to an x-ray intensifier of an x-ray device (stage 702), the calibration plate including at least one radiopaque marker and at least one optical marker positioned at predetermined locations within the calibration plate.
[0201] For example, the calibration plate 110, radiopaque markers 112 and optical markers 114 described above with respect to FIG. 1, or the calibration plate 200, radiopaque markers 210, 220 and optical markers 230 described above with respect to FIGS. 2A, 2B and 2C.
[0202] Some embodiments may include mounting (or removably mounting) a camera (stage 704) to the treatment tool at a predetermined position and orientation relative to the treatment tool (e.g., as described above with respect to Figures 1 and 5A-5G), such as camera 120 described above with respect to Figure 1, or camera 520 described above with respect to Figures 5A-5G.
[0203] Some embodiments may include acquiring an X-ray image of the calibration plate (stage 706) with an X-ray device (e.g., as described above with respect to Figures 1 and 3), the X-ray image including a visual representation of at least one radiopaque marker.
[0204] Some embodiments may include determining (stage 708) a position and orientation of the calibration plate relative to the x-ray device based on a visual representation of the at least one radiopaque marker in the x-ray image, a predetermined location of the at least one radiopaque marker in the calibration plate, and certain parameters of the x-ray device, by a processing unit (e.g., as described above with respect to FIG. 1), such as processing unit 130 described above with respect to FIG. 1, or processing unit 530 described above with respect to FIG. 5A-5G.
[0205] Some embodiments may include identifying a visual representation of the at least one radiopaque marker in the x-ray image of the calibration plate (stage 709), for example using a pattern matching algorithm as described above with respect to FIG.
[0206] Some embodiments may include determining (stage 710) one or more vectors that extend from an X-ray source of the X-ray device toward an X-ray intensifier and a calibration plate attached thereto, and that intersect with the calibration plate, based on a particular model of the X-ray device (e.g., as described above with respect to FIG. 1).
[0207] Some embodiments may include determining (stage 711) one or more corresponding intersection points of one or more vectors with the calibration plate (eg, as described above with respect to FIG. 1).
[0208] Some embodiments may include comparing a predetermined / known location of at least one radiopaque marker within the calibration plate to the intersection of the determined vector with the calibration plate (stage 712), for example using a point cloud matching algorithm (e.g., a brute force algorithm, an iterative closest point algorithm), as described above with respect to FIG.
[0209] Some embodiments may include determining the position and orientation of the calibration plate relative to the x-ray device (stage 713) based on a comparison of predetermined / known positions of the radiopaque markers in the calibration plate and the intersection of the determined vector with the calibration plate, for example using a registration algorithm (such as a singular value decomposition (SVD) algorithm, etc.), as described above with respect to FIG.
[0210] Some embodiments may include acquiring, with a camera, a camera image (stage 714) that includes a visual representation of at least one optical marker (eg, as described above with respect to Figures 1 and 4).
[0211] Some embodiments may include determining (stage 716) a position and pose of the camera relative to the calibration plate based on a visual representation of the at least one optical marker in the camera image and a predetermined position of the at least one optical marker within the calibration plate (e.g., as described above with respect to FIG. 1), for example using a bundle adjustment / PnP algorithm, as described above with respect to FIG.
[0212] Some embodiments may include determining (stage 718) a position and orientation of the treatment tool relative to the X-ray device based on a determined position and orientation of the calibration plate relative to the X-ray device (e.g., as described above with respect to Figures 1 and 5A-5G), a determined position and orientation of the camera within the calibration plate, and a predetermined position and orientation of the camera relative to the treatment tool.
[0213] Some embodiments may include displaying (stage 720) a visual indicator on a display that indicates the determined tool position and orientation, such as visual indicator 142 described above with respect to Figure 1 or visual indicator 542 described above with respect to Figures 5A-5G.
[0214] Some embodiments may include comparing the determined position and orientation of the treatment tool relative to the X-ray device (e.g., as described above with respect to Figures 1 and 5A-5G) to determine whether it is within an acceptable error range (stage 722) with a required position and orientation of the treatment tool relative to the X-ray device.
[0215] Some embodiments may include updating a visual indicator (e.g., as described above with respect to Figures 1 and 5A-5G) to indicate whether the position and orientation of the treatment tool relative to the X-ray device is within an acceptable error range (stage 724).
[0216] Some embodiments may include superimposing the visual indicator with an x-ray image of the target area within the patient to generate a superimposed image and displaying the superimposed image on a display (stage 725).
[0217] Some embodiments may include detecting or marking the target area in the X-ray image and determining whether the treatment tool is aligned with the target area based on the determined position and orientation of the treatment tool and the X-ray image (stage 726).
[0218] Some embodiments may include updating a visual indicator to indicate whether the tool is aligned with the target area (stage 727).
[0219] Some embodiments may include generating and displaying (stage 728) tool guidance instructions on the display that indicate one or more directions and / or amounts of movement in each direction in which the tool 90 should be moved (e.g., displaced and / or tilted) to bring the tool 90 into a position and orientation within an acceptable tolerance range. For example, as described below with respect to Figures 14A and 14B, the tool guidance instructions may be determined based on the determined position and orientation of the tool relative to the x-ray device. The tool guidance instructions may, for example, make it easy and intuitive to aim and / or align the tool 90 relative to the target area and / or the x-ray device.
[0220] Some embodiments may include aiming and aligning the treatment tool (stage 729) according to visual indicators displayed on a display without exposing the patient to X-ray imaging by an X-ray machine (e.g., as described above with respect to Figures 1 and 5A-5G).
[0221] Reference is now made to FIG. 8, which is a schematic illustration of a first embodiment of a system 800 for aiming and / or aligning a treatment tool 90 in an ultrasound imaging device 60 environment, according to some embodiments of the present invention.
[0222] In some embodiments, the system 800 may include a calibration plate 810, a camera 820, and a processing unit 830. Figure 8 shows a side view of the system 800. The system 800 allows for targeting of the treatment tool 90 to a target area 72 within the patient 70 and / or alignment of the treatment tool 90 relative to the ultrasound imaging probe 62.
[0223] The ultrasound imaging device 60 may be any type of ultrasound imaging device, such as a handheld device, a laptop device, a probe connected to a cell phone or tablet, or a system on a portable cart, and may include one or more probes and other accessories.
[0224] The treatment tool 90 may be, for example, an invasive treatment tool (such as a needle (e.g., a biopsy needle, a high frequency needle) and / or a probe) that needs to be aimed at the target region 72 and / or aligned with the ultrasonic imaging probe 62, or may be a non-invasive treatment tool (such as an ultrasonic transducer or a focused ultrasonic transducer). For example, FIG. 8 shows a needle as an example of the treatment tool 90.
[0225] In some embodiments, the calibration plate 810 may include at least one optical marker. The optical marker may be located at a predetermined and / or known position within the calibration plate 810. The calibration plate 810 may be attachable or removably attachable to the ultrasound imaging probe 62 of the ultrasound imaging device 60. Various embodiments of the calibration plate 810 are described below with reference to Figures 9A and 9B.
[0226] In some embodiments, the camera 820 is attachable or removably attachable to the treatment tool 90. The camera 820 may be attachable to the treatment tool 90 at a predetermined and / or known position and orientation relative to the treatment tool 90.
[0227] The camera 820 may be mountable to the treatment tool 90 such that at least a portion of the calibration plate 810 is within the field of view 821 of the camera 820 .
[0228] In some embodiments, the system 800 may include a camera connector 822. The camera connector 822 may be configured to securely and stably attach the camera 820 to the procedure tool 90 and prevent unintended movement of the camera 820 relative to the procedure tool 90.
[0229] In some embodiments, the camera connector 822 can allow for controlled rotation of the camera 820. This allows, for example, for the treatment tool 90 to follow the calibration plate 810 as it moves.
[0230] In some embodiments, the system 800 may include a second camera. The second camera may be securely and stably attached (e.g., using a connector such as connector 822) to the treatment tool 90 at a predetermined angle relative to the camera 820. For example, the second camera may be attached to the treatment tool 90 at a 90° angle relative to the camera 820. This allows the second camera to capture the calibration plate 810 when the calibration plate 810 moves out of the field of view 821 of the camera 820, for example, due to the treatment tool 90 moving.
[0231] According to some embodiments, the processing unit 830 may be in communication (eg, wired or wireless) with the camera 820 and the ultrasound imaging unit 64 of the ultrasound imaging device 60 .
[0232] According to some embodiments, processing unit 830 may be configured to receive, from camera 820, one or more camera images of calibration plate 810. The camera images of calibration plate 810 may include visual representations of the optical markers (e.g., as shown in and described below with respect to FIGS. 9A and 9B).
[0233] According to some embodiments, the processing unit 830 may be configured to determine the position and pose of the camera 820 relative to the calibration plate 810 (which may be attached to, for example, the ultrasound imaging probe 62) based on the visual representation of the optical markers in the camera image, the known positions of the optical markers in the calibration plate 810, and the parameters of the camera 820 (e.g., distortion, field of view 821, etc.). The determination may be utilized, for example, using a bundle adjustment / PnP algorithm.
[0234] According to some embodiments, the processing unit 830 may be configured to determine the position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62 based on a predetermined position and orientation of the calibration plate 810 relative to the ultrasonic imaging probe 62, the determined position and orientation of the camera 820 relative to the calibration plate 810, and the known position and orientation of the camera 820 relative to the treatment tool 90.
[0235] According to some embodiments, the system 800 may include a display 840. The processing unit 830 may be configured to present at least one visual indicator 842 indicative of the determined position and orientation of the treatment tool 90, for example, on the display 840 and / or the ultrasound display 66.
[0236] In some embodiments, the processing unit 830 may be configured to determine whether the determined position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62 is within an acceptable error range compared to a required position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62. The processing unit 830 may be configured to indicate whether the position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62 is within an acceptable error range by modifying the visual indicator 842.
[0237] In various embodiments, the processing unit 830 may be configured, for example, to superimpose a visual indicator 842 on an ultrasound image of the target area 72 of the patient 70 to generate a superimposed image and display the superimposed image on the display 840 and / or the ultrasound display 66.
[0238] In some embodiments, the processing unit 830 may be configured to detect or mark the target region 72 in the ultrasound image. In some embodiments, the marking may be based on user input. The processing unit 830 may be configured to determine whether the treatment tool 90 is aligned with respect to the target region 72 based on the determined position and orientation of the treatment tool 90 and the ultrasound image. In some embodiments, the processing unit 830 may be configured to indicate whether the treatment tool 90 is aligned with respect to the target region 72 by updating the visual indicator 142.
[0239] In some embodiments, the processing unit 830 may be configured to generate tool guidance instructions based on the determined position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62 (e.g., as described below with respect to FIGS. 14A and 14B ), and display the tool guidance instructions on the display 86 / 840. The tool guidance instructions may indicate one or more directions and / or an amount of movement in each direction by which the treatment tool 90 should be moved (e.g., displaced and / or tilted) to bring the treatment tool 90 into a position and orientation within a tolerance range and / or into a position and orientation that is aligned with respect to the target region 72. The tool guidance instructions may, for example, make it easy and intuitive to aim and / or align the treatment tool 90 relative to the target region 72 and / or ultrasonic imaging probe 62.
[0240] The system 800 allows a user (e.g., a treating physician) to aim the treatment tool 90 at the target area 72 and / or align (e.g., translate, tilt, pitch, yaw, roll, etc.) the treatment tool 90 relative to the ultrasound imaging probe 62 using visual indicators 842 displayed on the display 840 and / or ultrasound display 66. The visual indicators 842 can indicate the position and orientation of the treatment tool 90 relative to the ultrasound imaging probe 62, which eliminates (or substantially eliminates) the need for ultrasound imaging of the tip of the treatment tool. When utilizing the system 800, ultrasound imaging of the (applicable) tip of the treatment tool may only be required at the final stage of aiming and / or aligning the treatment tool 90 to ensure that the actual position and orientation of the treatment tool 90 relative to the patient's anatomy is correct before administering treatment. This allows the system 800 to significantly reduce process time compared to current treatment processes.
[0241] Reference is made to FIGS. 9A and 9B, which are schematic illustrations of a calibration plate 900 of a system (such as system 800) for aiming and / or aligning a treatment tool 90 in an ultrasound imaging environment, according to some embodiments of the present invention.
[0242] According to some embodiments, a calibration plate 900 (eg, as shown in FIGS. 9A and 9B) includes a plurality of optical markers 930 positioned at predetermined locations within the calibration plate 900.
[0243] In some embodiments, each optical marker 930 may include its own unique visual label. The visual label of the optical marker 930 may include, for example, a barcode, a QR code, a graphic pattern or shape, an alcomarker, an april tag, an AR tag, etc. that may have additional data associated with the visual label. The visual label may encode, for example, the location of the optical marker 930 within the calibration plate 900.
[0244] Parameters of the optical markers 930, such as the amount of optical markers 930, the dimensions of the optical markers 930, and / or the visual labels of the optical markers 930, may be determined based on, for example, the resolution of a camera (e.g., camera 820 described above with respect to FIG. 8). The parameters of the optical markers 930 may further be determined to enable determination / identification of the position and orientation of a camera (e.g., camera 820 described above with respect to FIG. 8) relative to the calibration plate 900 based on a visual representation of the optical markers 930 in a camera image of the calibration plate 900 (e.g., as described above with respect to FIG. 8).
[0245] In some embodiments, the calibration plate 900 may have a matte surface that can, for example, reduce light reflections from light sources in the operating room and eliminate (or substantially eliminate) glare in the camera image due to the light reflections, thereby avoiding loss of information obtained from the optical markers 930.
[0246] Reference is now made to FIG. 10, which is a camera image 1000 of a calibration plate (such as calibration plates 810, 900) of a system (such as system 800) for aiming and / or aligning a treatment tool 90 in an ultrasound imaging device environment according to some embodiments of the present invention.
[0247] 10, for example, shows a camera image 1000 of a calibration plate (e.g., calibration plate 900 shown in FIG. 9B). The camera image 1000 can be acquired, for example, during a treatment process using a camera attached to the treatment tool 90 (e.g., camera 820 described above with respect to FIG. 8). The camera image 1000 can include a visual representation 1010 of an optical marker (e.g., optical marker 930 shown in FIG. 9B) positioned at a predetermined location within the optical tracking plate.
[0248] Reference is now made to FIG. 11, which is a schematic illustration of a second embodiment of a system 1100 for projecting focused ultrasonic energy, according to some embodiments of the present invention.
[0249] According to some embodiments, the system 1100 may include a focused ultrasound (FUS) transducer 1110 positioned to generate FUS energy 1119.
[0250] According to some embodiments, the system 1100 may include an ultrasound imaging probe 62 .
[0251] In some embodiments, the system 1100 may include a support 1120 suitable for housing the FUS transducer 1110. In some embodiments, the support 1120 may be handheld. In some embodiments, the system 1100 may include an articulated arm 1122 having a first end coupled to the table 88 and a second end coupled to the support 1120. The articulated arm 1122 may be arranged to allow translation, tilt, pitch, yaw, and / or roll of the support 1120 and the FUS transducer 1110 housed therein.
[0252] In some embodiments, the system 1100 may include an acoustic coupler 1140 positioned to acoustically couple the FUS transducer 1110 to the patient 70 to enable delivery of FUS energy 1119 to a target region 72 within the patient 70.
[0253] According to some embodiments, the system 1100 may include a calibration plate 1150 (eg, calibration plate 810 described above with respect to FIG. 8, or calibration plate 900 described above with respect to FIGS. 9A-9B) that is attachable to the ultrasound imaging probe 62.
[0254] According to some embodiments, the system 1100 may include a camera 1160 (eg, camera 820 described above with respect to FIG. 8) that may be attached to the support 1120 or the FUS transducer 1110.
[0255] According to some embodiments, the system 1100 may include a processing unit 1170 (e.g., processing unit 830 described above with respect to FIG. 8 ). The processing unit 1170 may be configured to determine a position and orientation of the FUS transducer 1110 relative to the ultrasound imaging probe 62.
[0256] According to some embodiments, the system 1100 may include a display 1180. The processing unit 1170 may be configured to display a visual indicator 1142 on the display 1180 and / or the ultrasound display 66 that is indicative of the determined position and orientation of the FUS transducer 1110.
[0257] In some embodiments, the processing unit 1170 may be configured to determine whether the determined position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62 is within an acceptable error range compared to a required position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62. The processing unit 1170 may be configured to indicate whether the position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62 is within an acceptable error range by modifying the visual indicator 842.
[0258] In various embodiments, the processing unit 1170 may be configured, for example, to superimpose the visual indicator 1142 on an ultrasound image of the target area 72 of the patient 70 to generate a superimposed image and display the superimposed image on the display 1180 and / or the ultrasound display 66.
[0259] In some embodiments, the processing unit 1170 may be configured to detect or mark the target region 72 in the ultrasound image. The processing unit 1170 may be configured to determine whether the treatment tool 90 is aligned with respect to the target region 72 based on the determined position and orientation of the treatment tool 90 and the ultrasound image. In some embodiments, the processing unit 1170 may be configured to indicate whether the treatment tool 90 is aligned with respect to the target region 72 by updating the visual indicator 1142.
[0260] In some embodiments, the processing unit 1170 may be configured to generate tool guidance instructions and display the tool guidance instructions on the display 86 / 1180 (e.g., as described below with respect to FIGS. 14A and 14B). The tool guidance instructions may be determined based on the position and orientation of the tool 90 relative to the ultrasonic imaging probe 62 (e.g., as described below with respect to FIGS. 14A and 14B). The tool guidance instructions may, for example, make it easy and intuitive to aim and / or align the tool 90 relative to the target region 72 and / or the ultrasonic imaging probe 62.
[0261] The system 1100 allows a user (e.g., a treating physician) to aim the FUS transducer 1110 at a target region 72 within the patient 70 and / or align the FUS transducer 1110 relative to the ultrasound imaging probe 62 using visual indicators 1142 displayed on the display 1180 and / or ultrasound display 66. The visual indicators 1142 may indicate a determined position and orientation of the FUS transducer 1110 relative to the ultrasound imaging probe 62. This allows the system 1100 to significantly reduce the overall duration of the treatment process as compared to current treatment processes that do not utilize the system 1100.
[0262] Reference is now made to FIG. 12, which is a flow chart of a first method for aiming and / or aligning a treatment tool in an ultrasound imaging environment, according to some embodiments of the present invention.
[0263] The method may be performed by a system for aiming and aligning a treatment tool in an ultrasound imaging environment (such as system 800 described above with respect to FIG. 8 or system 1100 described above with respect to FIG. 11) that may be configured to perform the method. It should be noted that the method is not limited to the flowchart shown in FIG. 12 and the corresponding description. For example, in various embodiments, the method need not proceed through each box or stage shown or in the exact same order as shown and described.
[0264] According to some embodiments, the method includes attaching (or removably attaching) a calibration plate to an ultrasound imaging probe of an ultrasound imaging device (stage 1202), the calibration plate including at least one optical marker positioned at a predetermined position within the calibration plate.
[0265] For example, the calibration plate 810 described above with respect to FIG. 8, or the calibration plate 900 and optical markers 930 described above with respect to FIGS. 9A and 9B.
[0266] Some embodiments may include mounting (or removably mounting) a camera (stage 1204) to the instrument at a predetermined position and orientation relative to the instrument (e.g., as described above with respect to FIG. 8), such as camera 820 described above with respect to FIG.
[0267] Some embodiments may include acquiring, by a camera, a camera image including a visual representation of at least one optical marker (stage 1206) (eg, as described above with respect to Figures 8 and 11).
[0268] Some embodiments may include determining (stage 1208) a position and pose of the camera relative to the calibration plate based on a visual representation of the at least one optical marker in the camera image and a predetermined position of the at least one optical marker within the calibration plate (e.g., as described above with respect to FIG. 8), for example, utilizing a bundle adjustment / PnP algorithm, as described above with respect to FIG.
[0269] Some embodiments may include determining (stage 1210) a position and orientation of the treatment tool relative to the ultrasound imaging probe based on a predetermined position and orientation of the calibration plate relative to the ultrasound imaging probe, a determined position and orientation of the camera relative to the calibration plate, and a predetermined position and orientation of the camera relative to the treatment tool (e.g., as described above with respect to FIG. 8).
[0270] Some embodiments may include displaying a visual indicator on a display (stage 1212) that indicates the determined position and orientation of the treatment tool relative to the ultrasound imaging probe, such as the visual indicator 842 described above with respect to FIG.
[0271] Some embodiments may also include comparing the determined position and orientation of the instrument relative to the ultrasound imaging probe with a required position and orientation of the instrument relative to the ultrasound imaging probe to determine whether it is within an acceptable error range (stage 1213), which may allow, for example, rapid and accurate targeting of the instrument to the target area.
[0272] Some embodiments may include altering a visual indicator to indicate whether the position and orientation of the treatment tool relative to the ultrasound imaging probe is within an acceptable error range (stage 1214).
[0273] Some embodiments may include superimposing the visual indicator with an ultrasound image of the target area within the patient to generate a superimposed image and displaying the superimposed image on a display (stage 1215).
[0274] Some embodiments may include detecting or marking a target area in the ultrasound image and determining whether the treatment tool is aligned with the target area based on the determined position and orientation of the treatment tool and the ultrasound image (stage 1216).
[0275] Some embodiments may include updating a visual indicator to indicate whether the tool is aligned with the target area (stage 1217).
[0276] Some embodiments may include generating and displaying on the display (stage 1218) tool guidance instructions indicating one or more directions and / or the amount of movement in each direction in which the tool should be moved (e.g., displaced and / or tilted) to bring the tool into a position and orientation within an acceptable tolerance range, for example as described below with respect to Figures 14A and 14B. This may thus allow, for example, for rapid and accurate targeting of the tool to the target area.
[0277] Some embodiments may include aiming and / or aligning the treatment tool according to visual indicators displayed on the display (stage 1219).
[0278] Reference is now made to FIG. 13, which is a schematic illustration of a calibration setup 1300 for a system for targeting and / or aligning a treatment tool using an ultrasound imaging device (such as systems 800, 1100 described above), according to some embodiments of the present invention.
[0279] The calibration facility 1300 may be used to calibrate a system (such as systems 800, 1100 described above) for aiming and / or aligning a treatment tool in an ultrasound imaging environment. For example, the calibration facility 1300 may be used to determine the position and orientation of the calibration plate 1324 (such as calibration plates 810, 900, 1000, 1150 described above) relative to the ultrasound imaging probe 62 (e.g., after the calibration plate 1324 is attached to the ultrasound imaging probe 62). If the calibration plate 1324 is permanently attached to the ultrasound imaging probe 62, the calibration may be performed only once, for example, at a factory, and may be repeated each time the calibration plate 1324 is removably attached to the ultrasound imaging probe 62.
[0280] The calibration setup 1300 may include one or more imaging targets 1304 embedded in an acoustically transparent medium 1302 (e.g., ultrasound gel, water). In some embodiments, the position of the imaging targets 1304 may be known. The ultrasound imaging probe 62 may be positioned at a known position and orientation of the imaging probe relative to the imaging targets 1304. The calibration plate 1324 may be attached or removably attached to the ultrasound imaging probe 62. A camera 1311 (e.g., cameras 820, 1160, etc., discussed above) may be positioned at a known position and orientation of the camera relative to the imaging targets 1304 such that at least a portion of the calibration plate 1324 is within the field of view 1312.
[0281] The ultrasound imaging probe 62 can acquire at least one ultrasound image of the imaging target 1304. The camera 1311 can acquire at least one camera image calibration plate 1324 attached to the ultrasound imaging probe 62. A processing unit (e.g., processing units 830, 1170, etc. described above) can receive the ultrasound image of the imaging target 1304 and the camera image 1324 of the calibration plate 1324. The processing unit can determine calibration data based on at least the ultrasound image and the camera image. For example, the calibration data may include a position and orientation of the calibration plate 1324 relative to the ultrasound imaging probe 62. The processing unit can determine a position and orientation of the calibration plate 1324 relative to the ultrasound imaging probe 62 based on the ultrasound image, the camera image, a known position of the imaging target 1304, a known position and orientation of the imaging probe, and a known position and orientation of the camera.
[0282] The calibration data may be stored and available for further processing (e.g., as described above with respect to FIG. 8 ), for example, the calibration data may be loaded into the processing unit 830 or processing unit 1170 described above and used by the processing units 830, 1170 to determine the position and orientation of the treatment tool 90 relative to the ultrasound imaging probe 62.
[0283] Reference is now made to FIG. 14A, which is a flowchart of a method for guiding the aiming and / or alignment of a treatment tool in an imaging device environment, according to some embodiments of the present invention.
[0284] The method can be performed, for example, by a processing unit of a system for aiming and / or aligning a treatment tool in an imaging device environment (such as systems 100, 500, 600, 500, 1100 described above with respect to Figures 1, 5A to 5G, 6, 8 and 11, respectively).
[0285] Reference is also made to FIG. 14B, which is a schematic illustration of a guidance feature for guiding the aiming and / or alignment of a treatment tool in an imaging device environment, according to some embodiments of the present invention.
[0286] According to some embodiments, the method may include presenting, by a processing unit 1410, at least one visual indicator 1420 on a display 1430 (stage 1402).
[0287] In various embodiments, the visual indicator 1420 may indicate the actual position and orientation of the tool relative to the imaging device (e.g., visual representation 85 of X-ray aiming and alignment tool 92, as described above with respect to FIGS. 5D-5F) and / or the determined position and orientation of the tool relative to the imaging device (e.g., visual indicators 142, 542, 682, 842, 1142, as described above with respect to FIGS. 1, 5A-5D, 6, 8, and 11, respectively). The visual indicator 1420 may also indicate whether the position and orientation of the tool relative to the imaging device is within an acceptable error range. This may allow, for example, for quick and accurate targeting of the tool to the target area. In some embodiments, the visual indicator 1420 may be superimposed on an image that has been or is being acquired by the imaging device (e.g., as described above with respect to FIGS. 1, 5A-5D, and 8).
[0288] According to some embodiments, the method may include generating, by the processing unit 1410, tool guidance instructions 1440 based on the determined position and orientation of the tool relative to the imaging device (stage 1404).
[0289] In some embodiments, the method may include presenting (stage 1406), by the processing unit 1410, on the display 1430 tool guidance instructions 1440. The tool guidance instructions 1440 may, for example, make targeting and / or alignment of a tool relative to a target area and / or imaging device easy and intuitive.
[0290] The treatment tool guidance instructions 1440 can be displayed, for example, in a specific area 1432 of the display 1430. The specific area 1432 can be selected, for example, by the processing unit 1410 so as not to obstruct important information displayed on the display 1430.
[0291] In various embodiments, the tool guidance instructions 1440 may include tool movement data indicating one or more directions in which the tool should be moved (e.g., displaced and / or tilted) and / or the amount of movement in each direction to bring the tool into a position and orientation within an acceptable error range. For example, each direction may be presented using an arrow, with the length of the arrow indicating the magnitude of movement required in the respective direction. However, other symbols may be used. The tool guidance instructions 1440 may, for example, enable rapid and accurate targeting of the tool to a target area.
[0292] According to some embodiments, the method may include presenting, by the processing unit 1410, the tool visual data 1450 on the display 1430. The tool visual data 1450 may include, for example, a visual representation of at least a portion of the tool 1452. For example, the visual representation of the tool 1452 may be displayed in proximity to the tool guidance instructions 1440.
[0293] According to some embodiments, the method may include presenting, by the processing unit 1410, system component visual data 1460 on the display 1430. The system component visual data 1460 may include, for example, a visual representation of one or more parts of the system (e.g., a camera), and may indicate, for example, the actual position and orientation of each component relative to a treatment tool.
[0294] The tool visual data 1450 and the system component visual data 1460 can, for example, enhance a user's understanding of the tool guidance instructions 1440.
[0295] Reference is now made to FIGS. 15A and 15B, which are schematic illustrations of a second embodiment of a system 1500 for aiming and / or aligning a treatment tool 90 in an X-ray device 80 environment, according to some embodiments of the present invention.
[0296] According to some embodiments, the system 1500 may include a calibration plate 1510, a camera 1520, and a processing unit 1530 (e.g., as shown in FIGS. 15A and 15B). FIGS. 15A and 15B show side views of the system 1500. The system 1500 may enable targeting of the treatment tool 90 to a target region 72 within the patient 70 and / or alignment of the treatment tool 90 relative to the X-ray device 80 while significantly reducing exposure of the patient 70 to X-ray radiation as compared to a treatment process performed without the system 1500.
[0297] According to some embodiments, the calibration plate 1510 may include at least one optical marker 1514. The optical marker 1514 may be positioned at a predetermined and / or known location on a surface of the calibration plate 1510. The calibration plate 1510 may be attachable or removably attachable to the treatment tool 90. For example, the calibration plate 1510 and optical markers 1514 may be similar to the calibration plate 110 and optical markers 114 described above with respect to FIG. 1 (without the radiopaque marker 112).
[0298] According to some embodiments, camera 1520 is attachable or removably attachable to X-ray device 80. Camera 1520 may be attachable to X-ray device 80 at a predetermined and / or known position and orientation relative to X-ray device 80 (e.g., as shown in FIG. 15A ).
[0299] In some embodiments, the camera 1520 may include at least one radiopaque marker 1512. In this case, the camera 1520 may be mounted to the X-ray device 80 such that at least the radiopaque marker 1512 is located within the field of view 83a of the X-ray source 83. For example, the camera 1520 may be mountable to the X-ray intensifier 82 (e.g., as shown in FIG. 15B). In the embodiment shown in FIG. 15B, the position and orientation of the camera 1520 relative to the X-ray device 80 may be unknown and may be determined based on an X-ray image of the radiopaque marker 1512 mounted to the camera 1520 (e.g., as described above with respect to FIG. 1).
[0300] In some embodiments, the system 1500 may include a camera connector 1522. The camera connector 1522 may be configured to securely and stably attach the camera 1520 to the X-ray device 80 and prevent unintended movement of the camera 1520 relative to the X-ray device 80.
[0301] According to some embodiments, the processing unit 1530 may be in communication (eg, wired or wireless) with the camera 1520 and the X-ray imaging unit 84 of the X-ray device 80 .
[0302] According to some embodiments, the processing unit 1530 may be configured to receive, from the camera 1520, one or more camera images of the calibration plate 1510. The camera images of the calibration plate 1510 may include a visual representation of the optical markers 1514.
[0303] According to some embodiments, the processing unit 1530 may be configured to determine a position and orientation of the calibration plate 1510 (which may be attached to the treatment tool 90, for example) relative to the camera 1520 based on a visual representation of the optical markers 1514 in the camera image, the known positions of the optical markers 1514 in the calibration plate 1510, and parameters of the camera 1520 (e.g., distortion, field of view 121, etc.). The determination may be utilized, for example, using a bundle adjustment / PnP algorithm.
[0304] According to some embodiments, the processing unit 1530 may be configured to determine the position and orientation of the treatment tool 90 relative to the X-ray device 80 based on the determined position and orientation of the calibration plate 1510 relative to the camera 1520 and the known position and orientation of the camera 1520 relative to the X-ray device 80.
[0305] In the embodiment shown in FIG. 15B, where the position and orientation of the camera 1520 relative to the X-ray device 80 is not known and the camera 1520 includes a radiopaque marker 1512, the processing unit 1530 may be configured to receive an X-ray image including a visual representation of the at least one radiopaque marker and determine the position and orientation of the camera relative to the X-ray device based on the visual representation of the at least one radiopaque marker.
[0306] According to some embodiments, the system 1500 may include a display 1540. The processing unit 1530 may be configured to present at least one visual indicator 1542 indicative of the determined position and orientation of the treatment tool 90, for example, on the display 1540 and / or the X-ray display 86.
[0307] In some embodiments, the processing unit 1530 may be configured to determine whether the determined position and orientation of the treatment tool 90 relative to the X-ray device 80 is within an acceptable error range, compared to a required position and orientation of the treatment tool 90 relative to the X-ray device 80. In some embodiments, the processing unit 1530 may be configured to indicate whether the position and orientation of the treatment tool 90 is within an acceptable error range, by updating the visual indicator 1542. Changing the visual indicator 1542 may include, for example, changing the color and / or shape of the visual indicator 1542 (e.g., according to a predetermined rule).
[0308] In various embodiments, the processing unit 1530 may be configured, for example, to superimpose a visual indicator 1542 on an X-ray image of the target area 72 of the patient 70 to generate a superimposed image and display the superimposed image on the display 1540 and / or the X-ray display 86.
[0309] In some embodiments, the processing unit 1530 may be configured to detect or mark the target region 72 in the x-ray image. The processing unit 1530 may be configured to determine whether the treatment tool 90 is aligned with respect to the target region 72 based on the determined position and orientation of the treatment tool 90 and the x-ray image. In some embodiments, the processing unit 1530 may be configured to indicate whether the treatment tool 90 is aligned with respect to the target region 72 by updating the visual indicator 1542.
[0310] In some embodiments, the processing unit 1530 may be configured to generate tool guidance instructions based on the determined position and orientation of the treatment tool 90 relative to the X-ray device 80 (e.g., as described above with respect to FIGS. 14A and 14B ) and display the tool guidance instructions on the display 86 / 1540. The guidance instructions may indicate one or more directions and / or an amount of movement in each direction in which the treatment tool 90 should be moved (e.g., displaced and / or tilted) to bring the treatment tool 90 into a position and orientation within an acceptable error range. The tool guidance instructions may, for example, make it easy and intuitive to aim and / or align the treatment tool 90 relative to the target region 72 and / or the X-ray device 80.
[0311] The system 1500 allows a user (e.g., a treating physician) to aim the treatment tool 90 at the target area 72 and / or align (e.g., translate, tilt, pitch, yaw, roll, etc.) the treatment tool 90 relative to the X-ray device 80 using visual indicators 1542 displayed on the display 1540 and / or the X-ray display 86. The visual indicators 1542 can indicate the position and orientation of the treatment tool 90 relative to the X-ray device 80, which eliminates (or substantially eliminates) the need for X-rays of the patent 70 during the aiming and / or alignment of the treatment tool 90. When utilizing the system 1500, X-rays may only be required at the final stages of aiming and / or aligning the treatment tool 90 to verify that the actual position and orientation of the treatment tool 90 relative to the X-ray device 80 corresponds to or is within an acceptable error range for the required position and orientation before administering treatment. This enables system 1500 to significantly reduce the patient's and / or user's exposure to X-ray radiation during the treatment process, as compared to current treatment processes that require multiple exposures of the patient 70 to X-ray radiation during aiming and / or positioning of the treatment tool 90.
[0312] Reference is now made to FIG. 16, which is a flowchart of a second method for aiming and / or aligning a treatment tool in an X-ray environment, according to some embodiments of the present invention.
[0313] The method may be performed by a system for aiming and aligning a treatment tool in an x-ray environment (such as system 1500 described above with respect to Figures 15A and 15B) that may be configured to perform the method. It should be noted that the method is not limited to the flowchart shown in Figure 16 and the corresponding description. For example, in various embodiments, the method need not proceed through each box or stage shown or in the exact same order as shown and described.
[0314] According to some embodiments, the method includes attaching (or removably attaching) a calibration plate to the treatment tool, the calibration plate including at least one optical marker positioned at a predetermined location within the calibration plate (stage 1602), such as the calibration plate 1510 and optical marker 1514 described above with respect to Figures 15A and 15B.
[0315] Some embodiments may include mounting (or removably mounting) a camera (stage 1604) to an x-ray device (e.g., as described above with respect to Figures 15A and 15B), such as camera 1520 described above with respect to Figures 15A and 15B.
[0316] Some embodiments may include acquiring, by a camera, a camera image including a visual representation of at least one optical marker (stage 1606) (eg, as described above with respect to Figures 1 and 4).
[0317] Some embodiments may include determining a position and pose of the calibration plate relative to the camera (stage 1607) based on a visual representation of the at least one optical marker in the camera image and a position of the at least one optical marker within the calibration plate (e.g., as described above with respect to Figures 15A and 15B), for example using a bundle adjustment / PnP algorithm, as described above with respect to Figures 15A and 15B.
[0318] Some embodiments may include determining a position and orientation of the treatment tool relative to the X-ray device (stage 1608) based on the determined position and orientation of the calibration plate relative to the camera and the position and orientation of the camera relative to the X-ray device (e.g., as described above with respect to Figures 15A and 15B).
[0319] In some embodiments, the position and orientation of the camera relative to the X-ray machine may be known / predetermined.
[0320] In some other embodiments, the camera may be mounted to the x-ray device in any position and orientation such that at least a portion is within the field of view of an x-ray source of the x-ray device, and may include at least one radiopaque marker. These embodiments may include obtaining, with the x-ray device, an x-ray image of at least a portion of the camera, and determining the position and orientation of the camera relative to the x-ray device based on a visual representation of the at least one radiopaque marker in the x-ray image.
[0321] Some embodiments may include displaying (stage 1610) a visual indicator on a display that indicates the determined tool position and orientation, such as the visual indicator 1542 described above with respect to Figures 15A and 15B.
[0322] Some embodiments may include comparing the determined position and orientation of the treatment tool relative to the X-ray device (e.g., as described above with respect to Figures 15A and 15B) to a required position and orientation of the treatment tool relative to the X-ray device to determine whether it is within an acceptable error range (stage 1612).
[0323] Some embodiments may include updating a visual indicator (e.g., as described above with respect to Figures 15A and 15B) to indicate whether the position and orientation of the treatment tool relative to the X-ray device is within an acceptable error range (stage 1614).
[0324] Some embodiments may also include superimposing the visual indicator with an x-ray image of the target area within the patient to generate a superimposed image and displaying the superimposed image on a display (stage 1616).
[0325] Some embodiments may include detecting or marking the target area in the X-ray image and determining whether the treatment tool is aligned with the target area based on the determined position and orientation of the treatment tool and the X-ray image (stage 1618).
[0326] Some embodiments may include updating a visual indicator to indicate whether the tool is aligned with the target area (stage 1620).
[0327] Some embodiments may include generating and displaying on the display (stage 1622) tool guidance instructions indicating one or more directions and / or amounts of movement in each of the directions in which the tool should be moved (e.g., displaced and / or tilted) to bring the tool into a position and orientation within an acceptable tolerance range. For example, as described above with respect to Figures 14A and 14B, the tool guidance instructions may be determined based on the determined position and orientation of the tool relative to the x-ray device. The tool guidance instructions may, for example, make it easy and intuitive to aim and / or align the tool relative to the target area and / or the x-ray device.
[0328] Some embodiments may include aiming and aligning the treatment tool (stage 1624) according to visual indicators displayed on a display without exposing the patient to X-ray imaging by an X-ray machine (e.g., as described above with respect to Figures 15A and 15B).
[0329] Reference is now made to FIG. 17, which is a schematic illustration of a second embodiment of a system 1700 for aiming and / or aligning a treatment tool 90 in an ultrasound imaging device 60 environment, according to some embodiments of the present invention.
[0330] According to some embodiments, the system 1700 may include a calibration plate 1710, a camera 1720, and a processing unit 1730. Figure 17 shows a side view of the system 1700. The system 1700 allows for targeting of the treatment tool 90 to a target area 72 within the patient 70 and / or alignment of the treatment tool 90 relative to the ultrasound imaging probe 62.
[0331] The ultrasound imaging device 60 may be any type of ultrasound imaging device, such as a handheld device, a laptop device, a probe connected to a cell phone or tablet, or a system on a portable cart, and may include one or more probes and other accessories.
[0332] The treatment tool 90 may be, for example, an invasive treatment tool (such as a needle (e.g., a biopsy needle, a high-frequency needle) and / or a probe) that needs to be aimed at the target region 72 and / or aligned with the ultrasonic imaging probe 62, or may be a non-invasive treatment tool (such as an ultrasonic transducer or a focused ultrasonic transducer). For example, FIG. 17 shows a needle as an example of the treatment tool 90.
[0333] According to some embodiments, the calibration plate 1710 may include at least one optical marker. The optical marker may be located at a predetermined and / or known location within the calibration plate 1710. The calibration plate 1710 may be attachable or removably attachable to the treatment tool 90.
[0334] According to some embodiments, the camera 1720 is attachable or removably attachable to the ultrasound imaging probe 62 of the ultrasound device 60. The camera 1720 may be attachable to the ultrasound imaging probe 62 at a predetermined and / or known position and orientation relative to the treatment tool 90. The camera 1720 may be attachable to the ultrasound imaging probe 62 such that at least a portion of the calibration plate 1710 is within the field of view 1721 of the camera 1720.
[0335] In some embodiments, the system 1700 may include a camera connector 1722. The camera connector 1722 may be configured to securely and stably attach the camera 1720 to the ultrasound imaging probe 62 and prevent unintended movement of the camera 1720 relative to the ultrasound imaging probe 62.
[0336] According to some embodiments, the processing unit 1730 may be in communication (eg, wired or wireless) with the camera 1720 and the ultrasound imaging unit 64 of the ultrasound imaging device 60 .
[0337] According to some embodiments, processing unit 1730 may be configured to receive, from camera 1720, one or more camera images of calibration plate 1710. The camera images of calibration plate 1710 may include visual representations of the optical markers.
[0338] According to some embodiments, the processing unit 1730 may be configured to determine the position and orientation of the calibration plate 1710 relative to the camera 1720 (which may be mounted, for example, on the ultrasound imaging probe 62) based on a visual representation of the optical markers in the camera image, the known positions of the optical markers in the calibration plate 1710, and parameters of the camera 1720 (e.g., distortion, field of view 1721, etc.). The determination may be utilized, for example, using a bundle adjustment / PnP algorithm.
[0339] According to some embodiments, the processing unit 1730 may be configured to determine the position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62 based on a predetermined position and orientation of the calibration plate 1710 relative to the treatment tool 90, a determined position and orientation of the calibration plate 1710 relative to the camera 1720, and a known position and orientation of the camera 1720 relative to the ultrasonic imaging probe 62.
[0340] According to some embodiments, the system 1700 may include a display 1740. The processing unit 1730 may be configured to present at least one visual indicator 1742 indicative of the determined position and orientation of the treatment tool 90, for example, on the display 1740 and / or the ultrasound display 66.
[0341] In some embodiments, the processing unit 1730 may be configured to determine whether the determined position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62 is within an acceptable error range compared to a required position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62. The processing unit 1730 may be configured to indicate whether the position and orientation of the treatment tool 90 relative to the ultrasonic imaging probe 62 is within an acceptable error range by modifying the visual indicator 1742.
[0342] In various embodiments, the processing unit 1730 may be configured, for example, to superimpose a visual indicator 1742 on an ultrasound image of the target area 72 of the patient 70 to generate a superimposed image and display the superimposed image on the display 1740 and / or the ultrasound display 66.
[0343] In some embodiments, the processing unit 1730 may be configured to detect or mark the target region 72 in the ultrasound image. The processing unit 1730 may be configured to determine whether the treatment tool 90 is aligned with respect to the target region 72 based on the determined position and orientation of the treatment tool 90 and the ultrasound image. In some embodiments, the processing unit 1730 may be configured to indicate whether the treatment tool 90 is aligned with respect to the target region 72 by updating the visual indicator 142.
[0344] In some embodiments, the processing unit 1730 may be configured to generate tool guidance instructions based on the determined position and orientation of the tool 90 relative to the ultrasonic imaging probe 62 (e.g., as described above with respect to FIGS. 14A and 14B ) and display the tool guidance instructions on the display 86 / 1740. The tool guidance instructions may indicate one or more directions and / or an amount of movement in each of the directions in which the tool 90 should be moved (e.g., displaced and / or tilted) to bring the tool 90 into a position and orientation within an acceptable tolerance and / or into a position and orientation that is aligned with respect to the target region 72. The tool guidance instructions may, for example, make it easy and intuitive to aim and / or align the tool 90 relative to the target region 72 and / or ultrasonic imaging probe 62.
[0345] The system 1700 allows a user (e.g., a treating physician) to aim the treatment tool 90 at the target area 72 and / or align (e.g., translate, tilt, pitch, yaw, roll, etc.) the treatment tool 90 relative to the ultrasound imaging probe 62 using visual indicators 1742 displayed on the display 1740 and / or ultrasound display 66. The visual indicators 1742 can indicate the position and orientation of the treatment tool 90 relative to the ultrasound imaging probe 62, which eliminates (or substantially eliminates) the need for ultrasound imaging of the tip of the treatment tool. When utilizing the system 1700, ultrasound imaging of the (applicable) tip of the treatment tool may only be required at the final stage of aiming and / or aligning the treatment tool 90 to ensure that the actual position and orientation of the treatment tool 90 relative to the patient's anatomy is accurate before administering treatment. This allows the system 1700 to significantly reduce process time compared to current treatment processes.
[0346] Reference is now made to FIG. 18, which is a flow chart of a first method for aiming and / or aligning a treatment tool in an ultrasound imaging environment, according to some embodiments of the present invention.
[0347] The method may be performed by a system for aiming and aligning a treatment tool in an ultrasound imaging environment (such as system 1700 described above with respect to FIG. 17) that may be configured to perform the method. It should be noted that the method is not limited to the flowchart shown in FIG. 18 and the corresponding description. For example, in various embodiments, the method need not proceed through each box or stage shown or in the exact order shown and described.
[0348] According to some embodiments, the method includes attaching (or removably attaching) a calibration plate to the treatment tool, the calibration plate including at least one optical marker positioned at a predetermined location within the calibration plate (stage 1802), such as the calibration plate 1710 described above with respect to FIG.
[0349] Some embodiments may include mounting (or removably mounting) a camera (stage 1804) to the ultrasound imaging probe at a predetermined position and orientation relative to the ultrasound imaging probe (e.g., as described above with respect to FIG. 17), such as camera 1720 described above with respect to FIG.
[0350] Some embodiments may include acquiring, by a camera, a camera image (stage 1806) that includes a visual representation of at least one optical marker (eg, as described above with respect to FIG. 17).
[0351] Some embodiments may include determining a position and pose of the calibration plate relative to the camera (stage 1808) based on a visual representation of the at least one optical marker in the camera image and a predefined position of the at least one optical marker within the calibration plate (e.g., as described above with respect to FIG. 17), for example using a bundle adjustment / PnP algorithm, as described above with respect to FIG.
[0352] Some embodiments may include determining (stage 1810) a position and orientation of the treatment tool relative to the ultrasound imaging probe based on a predetermined position and orientation of the calibration plate relative to the treatment tool, a determined position and orientation of the calibration plate relative to the camera, and a predetermined position and orientation of the camera relative to the ultrasound imaging probe (e.g., as described above with respect to FIG. 17).
[0353] Some embodiments may include displaying (stage 1812) a visual indicator on a display that indicates the determined position and orientation of the treatment tool relative to the ultrasound imaging probe, such as the visual indicator 1742 described above with respect to FIG.
[0354] Some embodiments may also include determining whether the determined position and orientation of the instrument relative to the ultrasound imaging probe is within an acceptable error range compared to a required position and orientation of the instrument relative to the ultrasound imaging probe (stage 1813), which may allow, for example, rapid and accurate targeting of the instrument to the target area.
[0355] Some embodiments may include altering a visual indicator to indicate whether the position and orientation of the treatment tool relative to the ultrasound imaging probe is within an acceptable error range (stage 1814).
[0356] Some embodiments may include superimposing the visual indicator with an ultrasound image of the target area within the patient to generate a superimposed image and displaying the superimposed image on a display (stage 1815).
[0357] Some embodiments may include detecting or marking a target area in the ultrasound image and determining whether the treatment tool is aligned with the target area based on the determined position and orientation of the treatment tool and the ultrasound image (stage 1816).
[0358] Some embodiments may include updating a visual indicator to indicate whether the tool is aligned with the target area (stage 1817).
[0359] Some embodiments may include generating and displaying on the display (stage 1818) tool guidance instructions indicating one or more directions in which the tool should be moved (e.g., displaced and / or tilted) and the amount of movement in each of the directions to bring the tool into position and orientation within an acceptable tolerance range, for example as described below with respect to Figures 14A and 14B. This thus allows, for example, for rapid and accurate targeting of the tool to the target area.
[0360] Some embodiments may include aiming and / or aligning the treatment tool according to visual indicators displayed on the display (stage 1819).
[0361] Advantageously, the disclosed system and method allows a user (e.g., a treating physician) to aim the treatment tool at a target area in the patient's body and / or align the treatment tool with respect to the imaging device using a visual indicator (e.g., displayed on a display) configured to indicate the determined position and orientation of the treatment tool with respect to the imaging device. This can be achieved without the need to mechanically connect the treatment tool to the imaging device and without the need for complex and / or expensive tracking units required in current processes. Furthermore, compared to current treatment processes that do not utilize the disclosed system and method, the disclosed system and method can improve the accuracy of the process, reduce the dependency of that accuracy on the user's skill level, and / or reduce the duration of the treatment process.
[0362] In the case of an X-ray device, the disclosed system and method can eliminate the need for X-raying the patient during the aiming and / or alignment of the treatment tool, and can significantly reduce the patient and / or user's exposure to X-ray radiation during the treatment process and / or significantly shorten the duration of the treatment process compared to treatment processes that require X-raying the patient during the aiming and / or alignment of the treatment tool. When utilizing the disclosed system and method, X-raying of the treatment tool and patient may only be required at the final stage of aiming and / or alignment of the treatment tool to verify that the actual position and orientation of the treatment tool relative to the X-ray device corresponds to or is within an acceptable error range for the required position and orientation before treatment is administered.
[0363] Aspects of the present invention have been described above with reference to flowcharts and / or partial diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It will be understood that each part of the flowcharts and / or partial diagrams, and combinations of parts of the flowcharts and / or partial diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, and the instructions executed via the processor of the computer or other programmable data processing apparatus generate means for implementing the functions / acts specified in the flowcharts and / or partial diagrams or portions thereof.
[0364] These computer program instructions may also be stored on a computer readable medium that causes a computer, other programmable data processing apparatus, or other device to function in a particular manner, such that the instructions stored on the computer readable medium create an article of manufacture including instructions that perform the functions / acts identified in the flowcharts and / or sub-diagrams, or portions thereof. The computer program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device such that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to create a computer-implemented process, such that the instructions executing on the computer or other programmable device provide a process for performing the functions / acts identified in the flowcharts and / or sub-diagrams, or portions thereof.
[0365] The flow charts and block diagrams described above illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each portion of the flow chart or sub-diagrams may represent a module, segment, or part of code, including one or more executable instructions for implementing specific logical functions. It should be noted that in some alternative implementations, the functions described in the portions may occur out of the order described in the figures. For example, two portions shown in succession may in fact be executed substantially simultaneously, or the portions may be executed in reverse order depending on the functionality involved. Also, each portion of the sub-diagrams and / or flow charts, and combinations of portions of the sub-diagrams and / or flow charts, may be implemented by a dedicated hardware-based system performing specific functions or acts, or a combination of dedicated hardware and computer instructions.
[0366] In the above description, the embodiments are examples or embodiments of the invention. The various appearances of "one embodiment", "embodiment", "particular embodiment" or "some embodiments" do not necessarily all refer to the same embodiment. Although various features of the invention may be described in the context of a single embodiment, the features may also be provided separately or in any suitable combination. Conversely, although the invention may be described in the context of separate embodiments herein for clarity, the invention may also be implemented in a single embodiment. A particular embodiment of the invention may include features from different embodiments described above, and a particular embodiment may incorporate elements from other embodiments described above. The disclosure of elements of the invention in the context of a particular embodiment should not be construed as limiting their use only in the particular embodiment. Furthermore, it should be understood that the invention may be implemented or carried out in various ways, and that the invention may be implemented in embodiments other than those outlined in the description above.
[0367] The present invention is not limited to these figures or the corresponding descriptions. For example, the flow need not proceed through each box or state shown, or in the exact same order as shown and described. The meanings of technical and scientific terms used herein should be broadly understood by those skilled in the art, unless otherwise specified. Although the present invention has been described with respect to a limited number of embodiments, these should not be interpreted as limitations on the scope of the present invention, but rather as illustrations of some preferred embodiments. Other possible variations, modifications, and applications are also within the scope of the present invention. Thus, the scope of the present invention should not be limited by what has been described above, but should be limited by the appended claims and their legal equivalents.
Claims
1. 1. A system for aiming and aligning a treatment tool in an x-ray environment, comprising: a calibration plate mountable to an x-ray device, the calibration plate including at least one radiopaque marker and at least one optical marker at a predetermined location within the calibration plate; a camera that can be attached to the treatment tool at a predetermined position and attitude with respect to the treatment tool; a processing unit in communication with the camera and an X-ray imaging unit of the X-ray device; The processing unit includes: receiving an x-ray image from the x-ray imaging unit, the x-ray image including a visual representation of the at least one radiopaque marker; determining a position and orientation of the calibration plate relative to the X-ray device based on a visual representation of the at least one radiopaque marker in the X-ray image, the predetermined position of the at least one radiopaque marker within the calibration plate, and certain parameters of the X-ray device; receiving a camera image from the camera, the camera image including a visual representation of the at least one optical marker; determining a position and orientation of the camera relative to the calibration plate based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; A system configured to determine a position and orientation of the treatment tool relative to the X-ray device based on the determined position and orientation of the calibration plate relative to the X-ray device, the determined position and orientation of the camera within the calibration plate, and the specified position and orientation of the camera relative to the treatment tool.
2. The calibration plate comprises: at least one asymmetric radiopaque marker; a plurality of symmetrical radiopaque markers asymmetrically positioned within the calibration plate; The system of claim 1 , further comprising at least one of:
3. The system of claim 1 , wherein the calibration plate includes a plurality of optical markers.
4. The system of any one of claims 1 to 3, further comprising a display, wherein the processing unit is configured to present a visual indicator on the display that indicates the determined position and orientation of the treatment tool.
5. The processing unit includes: comparing the determined position and orientation of the treatment tool with respect to the X-ray device with a required position and orientation of the treatment tool with respect to the X-ray device to determine whether or not the position and orientation are within an allowable error range; The system of claim 4 , configured to change the visual indicator to indicate whether a position and orientation of the treatment tool relative to the X-ray device is within the tolerance range.
6. The system of claim 4 or 5, wherein the processing unit is configured to superimpose the visual indicator onto an X-ray image of a target area within a patient's body.
7. The processing unit includes: Detecting or marking the target area in the x-ray image; determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the X-ray image; The system of claim 6 , configured to indicate whether the treatment tool is aligned with the target area by altering the visual indicator.
8. The processing unit includes: generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool relative to the X-ray device; The system according to any one of claims 4 to 7, configured to display the treatment tool guidance instruction on the display.
9. The system according to claim 8 , wherein the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and an amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
10. The processing unit includes: treatment tool visual data including a visual representation of at least a portion of the treatment tool; system component visual data including a visual representation of at least one component of the system, the visual data indicating an actual position and orientation of each of the at least one component relative to the treatment tool; 10. The system according to claim 8 or 9, configured to display at least one of:
11. The system according to any one of claims 1 to 10, wherein the treatment tool is one of a focused ultrasound transducer and an interventional treatment tool.
12. 1. A method for aiming and aligning a treatment tool in an x-ray device environment, comprising: Attaching a calibration plate to an x-ray device, the calibration plate including at least one radiopaque marker and at least one optical marker positioned at a predetermined location within the calibration plate; Attaching a camera to the treatment tool at a predetermined position and orientation with respect to the treatment tool; acquiring, with the x-ray device, an x-ray image of the calibration plate including a visual representation of the at least one radiopaque marker; determining, by a processing unit, a position and orientation of the calibration plate relative to the X-ray device based on a visual representation of the at least one radiopaque marker in the X-ray image, the predetermined position of the at least one radiopaque marker within the calibration plate, and specific parameters of the X-ray device; acquiring, with the camera, a camera image including a visual representation of the at least one optical marker; determining a position and orientation of the camera relative to the calibration plate based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; determining a position and orientation of the treatment tool relative to the X-ray device based on the determined position and orientation of the calibration plate relative to the X-ray device, the determined position and orientation of the camera within the calibration plate, and a predetermined position and orientation of the camera relative to the treatment tool; The method includes:
13. The method of claim 12 , further comprising displaying a visual indicator on a display that indicates the determined position and orientation of the treatment tool.
14. 14. The method of claim 13, further comprising: comparing the determined position and orientation of the treatment tool relative to the X-ray device with a required position and orientation of the treatment tool relative to the X-ray device to determine whether it is within an acceptable error range.
15. 15. The method of claim 14, further comprising: modifying the visual indicator to indicate whether the position and orientation of the treatment tool relative to the x-ray device is within the tolerance range.
16. The method of any one of claims 12 to 15, further comprising superimposing the visual indicator on an X-ray image of a target area within a patient's body.
17. detecting or marking the target area in the x-ray image; determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the X-ray image; changing the visual indicator to indicate whether the treatment tool is aligned with the target area; 20. The method of claim 16, further comprising:
18. generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool with respect to the X-ray device; Displaying the treatment tool guidance instruction on the display; The method of any one of claims 13 to 17, further comprising:
19. The method according to claim 18 , wherein the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and an amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
20. treatment tool visual data including a visual representation of at least a portion of the treatment tool; system component visual data including a visual representation of at least one component of a system, the visual data indicating an actual position and orientation of each of the at least one component relative to the treatment tool; 20. The method of claim 18 or 19, further comprising displaying at least one of:
21. 21. The method of claim 13, further comprising: aiming and aligning the treatment tool according to the visual indicator displayed on the display without exposing a patient to X-ray imaging by the X-ray device.
22. 1. A system for aiming and aligning a treatment tool in an ultrasound imaging environment, comprising: a calibration plate attachable to an ultrasonic imaging probe of the ultrasonic imaging device, the calibration plate including at least one optical marker at a predetermined position within the calibration plate; a camera that can be attached to the treatment tool at a predetermined position and attitude with respect to the treatment tool; a processing unit in communication with the camera and an ultrasound imaging unit of the ultrasound imaging device; The processing unit includes: receiving a camera image from the camera, the camera image including a visual representation of the at least one optical marker; determining a position and orientation of the camera relative to the calibration plate based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; A system configured to determine a position and orientation of the treatment tool relative to the ultrasonic imaging probe based on a predetermined position and orientation of the calibration plate relative to the ultrasonic imaging probe, the determined position and orientation of the camera relative to the calibration plate, and a known position and orientation of the camera relative to the treatment tool.
23. The system of claim 22 , wherein the calibration plate includes a plurality of optical markers.
24. 24. The system of claim 22 or 23, further comprising a display, wherein the processing unit is configured to present a visual indicator on the display that is indicative of the determined position and orientation of the treatment tool.
25. The processing unit includes: comparing the determined position and orientation of the treatment tool with respect to the ultrasonic imaging probe with a required position and orientation of the treatment tool with respect to the ultrasonic imaging probe to determine whether or not the position and orientation are within an allowable error range; 25. The system of claim 24, configured to change the visual indicator to indicate whether a position and orientation of the treatment tool relative to the ultrasonic imaging probe is within the acceptable error range.
26. The system of any one of claims 22 to 25, wherein the processing unit is configured to superimpose the visual indicator onto an ultrasound image of a target area within a patient's body.
27. The processing unit includes: Detecting or marking the target area in the ultrasound image; determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the ultrasound image; 27. The system of claim 26, configured to indicate whether the treatment tool is aligned with the target area by altering the visual indicator.
28. The processing unit includes: generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe; The system according to any one of claims 24 to 27, configured to display the treatment tool guidance instruction on the display.
29. The system of claim 28, wherein the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and an amount of movement in each of the one or more directions in order to bring the treatment tool into a position and orientation within an acceptable error range.
30. The processing unit includes: treatment tool visual data including a visual representation of at least a portion of the treatment tool; system component visual data including a visual representation of at least one component of the system, the visual data indicating an actual position and orientation of each of the at least one component relative to the treatment tool; 30. The system of claim 28 or 29, configured to display at least one of
31. The system according to any one of claims 22 to 30, wherein the treatment tool is one of a focused ultrasound transducer and an interventional treatment tool.
32. 1. A method for aiming and aligning a treatment tool in an ultrasound imaging environment, comprising: Attaching a calibration plate to an ultrasound imaging probe of an ultrasound imaging device, the calibration plate including at least one optical marker positioned at a predetermined location within the calibration plate; Attaching a camera to the treatment tool at a predetermined position and orientation with respect to the treatment tool; acquiring, with the camera, a camera image including a visual representation of the at least one optical marker; determining a position and orientation of the camera relative to the calibration plate based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; determining a position and orientation of the treatment tool relative to the ultrasonic imaging probe based on the determined position and orientation of the calibration plate relative to the ultrasonic imaging probe, the determined position and orientation of the camera within the calibration plate, and a predetermined position and orientation of the camera relative to the treatment tool; The method includes:
33. 33. The method of claim 32, further comprising displaying a visual indicator on a display that indicates the determined position and orientation of the treatment tool.
34. 34. The method of claim 33, further comprising: comparing the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe with a required position and orientation of the treatment tool relative to the ultrasonic imaging probe to determine whether it is within an acceptable error range.
35. 35. The method of claim 34, further comprising: modifying the visual indicator to indicate whether the position and orientation of the treatment tool relative to the ultrasonic imaging probe is within the acceptable error range.
36. The method of any one of claims 32 to 35, further comprising superimposing the visual indicator on an ultrasound image of a target area within a patient's body.
37. Detecting or marking the target area in the ultrasound image; determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the ultrasound image; 37. The method of claim 36, further comprising: altering the visual indicator to indicate whether the treatment tool is aligned with respect to the target area.
38. generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool with respect to the ultrasonic imaging probe; The method according to any one of claims 32 to 37, further comprising: displaying the treatment tool guidance instruction on the display.
39. 39. The method of claim 38, wherein the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and an amount of movement in each of the one or more directions to bring the treatment tool into a position and orientation within an acceptable error range.
40. treatment tool visual data including a visual representation of at least a portion of the treatment tool; system component visual data including a visual representation of at least one component of a system, the visual data indicating an actual position and orientation of each of the at least one component relative to the treatment tool; 40. The method of claim 38 or 39, further comprising displaying at least one of:
41. The method of any one of claims 32 to 40, further comprising aiming and aligning the treatment tool according to the visual indicator displayed on the display.
42. 1. A system for aiming and aligning a treatment tool in an x-ray environment, comprising: a calibration plate attachable to a treatment tool, the calibration plate including at least one optical marker at a predetermined location within the calibration plate; a camera mountable to the X-ray device; a processing unit in communication with the camera and an X-ray imaging unit of the X-ray device; The processing unit includes: receiving a camera image from the camera, the camera image including a visual representation of the at least one optical marker; determining a position and orientation of the calibration plate relative to the camera based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; The system is configured to determine a position and orientation of the treatment tool relative to the X-ray device based on the determined position and orientation of the calibration plate relative to the camera and the position and orientation of the camera relative to the X-ray device.
43. The system of claim 42, wherein the position and orientation of the camera is known.
44. the camera includes at least one radiopaque marker disposed at least partially within a field of view of an x-ray source of the x-ray device; The processing unit includes: receiving an x-ray image including a visual representation of the at least one radiopaque marker; 43. The system of claim 42, configured to determine a position and orientation of the camera relative to the X-ray device based on the visual representation of the at least one radiopaque marker.
45. The system of any one of claims 42 to 44, wherein the calibration plate includes a plurality of optical markers.
46. The system of any one of claims 42 to 45, further comprising a display, wherein the processing unit is configured to present a visual indicator on the display that indicates the determined position and orientation of the treatment tool.
47. The processing unit includes: comparing the determined position and orientation of the treatment tool with respect to the X-ray device with a required position and orientation of the treatment tool with respect to the X-ray device to determine whether or not the position and orientation are within an allowable error range; 47. The system of claim 46, configured to change the visual indicator to indicate whether a position and orientation of the treatment tool relative to the x-ray device is within the tolerance range.
48. 48. The system of claim 46 or 47, wherein the processing unit is configured to superimpose the visual indicator onto an X-ray image of a target area within a patient's body.
49. The processing unit includes: Detecting or marking the target area in the x-ray image; determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the X-ray image; 49. The system of claim 48, configured to indicate whether the treatment tool is aligned with the target area by altering the visual indicator.
50. The processing unit includes: generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool relative to the X-ray device; The system according to any one of claims 47 to 49, configured to display the treatment tool guidance instruction on the display.
51. The system of claim 50, wherein the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and an amount of movement in each of the one or more directions to bring the treatment tool into a position and orientation within an acceptable error range.
52. The processing unit includes: treatment tool visual data including a visual representation of at least a portion of the treatment tool; system component visual data including a visual representation of at least one component of the system, the visual data indicating an actual position and orientation of each of the at least one component relative to the treatment tool; 52. The system of claim 50 or 51, configured to display at least one of:
53. The system according to any one of claims 42 to 52, wherein the treatment tool is one of a focused ultrasound transducer and an interventional treatment tool.
54. 1. A method for aiming and aligning a treatment tool in an x-ray device environment, comprising: Attaching a calibration plate to a treatment tool, the calibration plate including at least one optical marker positioned at a predetermined location within the calibration plate; Attaching a camera to the X-ray device; acquiring, with the camera, a camera image including a visual representation of the at least one optical marker; determining a position and orientation of the treatment tool relative to the camera based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; determining a position and orientation of the treatment tool relative to the X-ray device based on the determined position and orientation of the calibration plate relative to the camera and the position and orientation of the camera relative to the X-ray device.
55. 55. The method of claim 54, wherein the position and orientation of the camera are known.
56. mounting the camera to the x-ray device such that at least one radiopaque marker is located at least partially within a field of view of an x-ray source; acquiring an x-ray image including a visual representation of the at least one radiopaque marker; determining a position and orientation of the camera relative to the X-ray device based on the visual representation of the at least one radiopaque marker; 55. The method of claim 54, comprising:
57. The method according to any one of claims 54 to 56, further comprising displaying on a display a visual indicator indicating the determined position and orientation of the treatment tool.
58. 58. The method of claim 57, further comprising comparing the determined position and orientation of the treatment tool relative to the X-ray device with a required position and orientation of the treatment tool relative to the X-ray device to determine whether it is within an acceptable error range.
59. 60. The method of claim 58, further comprising: modifying the visual indicator to indicate whether the position and orientation of the treatment tool relative to the x-ray machine is within the acceptable error range.
60. 60. The method of any one of claims 54 to 59, further comprising superimposing the visual indicator on an x-ray image of a target area within a patient's body.
61. detecting or marking the target area in the x-ray image; determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the X-ray image; 61. The method of claim 60, further comprising: altering the visual indicator to indicate whether the treatment tool is aligned with respect to the target area.
62. generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool with respect to the X-ray device; The method according to any one of claims 57 to 61, further comprising: displaying the treatment tool guidance instruction on the display.
63. 63. The method of claim 62, wherein the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and an amount of movement in each of the one or more directions to bring the treatment tool into a position and orientation within an acceptable error range.
64. treatment tool visual data including a visual representation of at least a portion of the treatment tool; system component visual data including a visual representation of at least one component of a system, the visual data indicating an actual position and orientation of each of the at least one component relative to the treatment tool; 64. The method of claim 62 or 63, further comprising displaying at least one of:
65. 65. The method of any one of claims 57 to 64, further comprising aiming and aligning the treatment tool according to the visual indicator displayed on the display without exposing a patient to X-ray imaging by the X-ray device.
66. 1. A system for aiming and aligning a treatment tool in an ultrasound imaging environment, comprising: a calibration plate attachable to a treatment tool, the calibration plate including at least one optical marker at a predetermined location within the calibration plate; a camera that can be attached to the ultrasound imaging probe at a predetermined position and orientation relative to the ultrasound imaging probe; a processing unit in communication with the camera and an ultrasound imaging unit of the ultrasound imaging device; The processing unit includes: receiving a camera image from the camera, the camera image including a visual representation of the at least one optical marker; determining a position and orientation of the calibration plate relative to the camera based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; A system configured to determine a position and orientation of the treatment tool relative to the ultrasonic imaging probe based on a predetermined position and orientation of the calibration plate relative to the treatment tool, the determined position and orientation of the calibration plate relative to the camera, and a known position and orientation of the camera relative to the ultrasonic imaging probe.
67. 67. The system of claim 66, wherein the calibration plate includes a plurality of optical markers.
68. 68. The system of claim 66 or 67, further comprising a display, wherein the processing unit is configured to present a visual indicator on the display that is indicative of the determined position and orientation of the treatment tool.
69. The processing unit includes: comparing the determined position and orientation of the treatment tool with respect to the ultrasonic imaging probe with a required position and orientation of the treatment tool with respect to the ultrasonic imaging probe to determine whether or not the position and orientation are within an allowable error range; 69. The system of claim 68, configured to indicate whether a position and orientation of the treatment tool relative to the ultrasonic imaging probe is within the acceptable error range by modifying the visual indicator.
70. 70. The system of claim 68 or 69, wherein the processing unit is configured to superimpose the visual indicator onto an ultrasound image of a target area within a patient's body.
71. The processing unit includes: Detecting or marking the target area in the ultrasound image; determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the ultrasound image; 71. The system of claim 70, configured to indicate whether the treatment tool is aligned with the target area by altering the visual indicator.
72. The processing unit includes: generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe; 72. The system according to claim 70 or 71, configured to display the treatment tool guidance instructions on the display.
73. The system of claim 72, wherein the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and an amount of movement in each of the one or more directions to bring the treatment tool into a position and orientation within an acceptable error range.
74. The processing unit includes: treatment tool visual data including a visual representation of at least a portion of the treatment tool; system component visual data including a visual representation of at least one component of the system, the visual data indicating an actual position and orientation of each of the at least one component relative to the treatment tool; A system according to any one of claims 68 to 73, configured to display at least one of:
75. The system according to any one of claims 66 to 74, wherein the treatment tool is either a focused ultrasound transducer or an interventional treatment tool.
76. 1. A method for aiming and aligning a treatment tool in an ultrasound imaging environment, comprising: Attaching a calibration plate to a treatment tool, the calibration plate including at least one optical marker positioned at a predetermined location within the calibration plate; Attaching a camera to an ultrasonic imaging probe of an ultrasonic imaging device at a predetermined position and attitude relative to the treatment tool; acquiring, with the camera, a camera image including a visual representation of the at least one optical marker; determining a position and orientation of the calibration plate relative to the camera based on the visual representation of the at least one optical marker in the camera image and the predetermined position of the at least one optical marker within the calibration plate; determining a position and orientation of the treatment tool relative to the ultrasonic imaging probe based on a predetermined position and orientation of the calibration plate relative to the treatment tool, the determined position and orientation of the calibration plate relative to the camera, and a known position and orientation of the camera relative to the ultrasonic imaging probe.
77. 77. The method of claim 76, further comprising displaying a visual indicator on a display that indicates the determined position and orientation of the treatment tool.
78. 78. The method of claim 77, further comprising: comparing the determined position and orientation of the treatment tool relative to the ultrasonic imaging probe with a required position and orientation of the treatment tool relative to the ultrasonic imaging probe to determine whether it is within an acceptable error range.
79. 80. The method of claim 78, further comprising: varying the visual indicator to indicate whether the position and orientation of the treatment tool relative to the ultrasonic imaging probe is within the acceptable error range.
80. 80. The method of claim 77 or 78, further comprising superimposing the visual indicator on an ultrasound image of a target area within a patient's body.
81. Detecting or marking the target area in the ultrasound image; determining whether the treatment tool is aligned with respect to the target area based on the determined position and orientation of the treatment tool and the ultrasound image; 81. The method of claim 80, further comprising: altering the visual indicator to indicate whether the treatment tool is aligned with respect to the target area.
82. generating a treatment tool guidance instruction based on the determined position and orientation of the treatment tool with respect to the ultrasonic imaging probe; The method according to any one of claims 77 to 81, further comprising: displaying the treatment tool guidance instruction on the display.
83. 83. The method of claim 82, wherein the treatment tool guidance instruction indicates one or more directions in which the treatment tool should be moved and an amount of movement in each of the one or more directions to bring the treatment tool into a position and orientation within an acceptable error range.
84. treatment tool visual data including a visual representation of at least a portion of the treatment tool; system component visual data including a visual representation of at least one component of a system, the visual data indicating an actual position and orientation of each of the at least one component relative to the treatment tool; The method of any one of claims 77 to 83, further comprising displaying at least one of:
85. The method of any one of claims 77 to 84, further comprising aiming and aligning the treatment tool according to the visual indicator displayed on the display.
Citation Information
Patent Citations
Three-dimensional imaging and modeling of ultrasound image data
CN111655160A
Medical image diagnostic apparatus
JP2009207677A
Ultrasonic diagnostic device and medical image diagnostic device
JP2015198888A
Medical image display apparatus, control program therefor, and medical image display system
JP2017131433A
Ultrasonic observation apparatus, operation method of ultrasonic observation apparatus, and operation program of ultrasonic observation apparatus
JP2020044045A
