Systems and methods for automatically placing a medical device in an anatomic structure using a

Abstract

Systems and methods for placing a medical device in an anatomical structure using a locking mechanism are provided. The method includes performing one or more ultrasound image acquisitions; detecting and tracking one or more anatomical structures in the one or more ultrasound image acquisitions; causing a display system to present a volume of the region of interest within or around the one or more anatomical structures in the one or more ultrasound image acquisitions; processing the one or more ultrasound image acquisitions to identify a first location corresponding to a target placement location within the region of interest of a medical device; processing the one or more ultrasound image acquisitions to identify a second location corresponding to a location of a catheter including the medical device; and outputting a signal to deploy the medical device when the second location matches the first location.

Description

Technical Field

[0001] Certain embodiments relate to ultrasound imaging. More specifically, certain embodiments relate to a method and system for placing a medical device in an anatomical structure using a locking mechanism. Background Art

[0002] Ultrasound imaging is a medical imaging technique used to image anatomical structures in the human body, such as organs and soft tissues. Ultrasound imaging uses real-time, non-invasive, high-frequency sound waves to produce two-dimensional (2D), three-dimensional (3D), and / or four-dimensional (4D) (i.e., real-time / continuous 3D images) images.

[0003] Ultrasound imaging is a powerful tool for real-time visualization of cardiac (e.g., muscle and chamber) motion, and for placing medical devices in and around organs and soft tissues. However, current methods and ultrasound systems for placing medical devices do not adequately guide physicians when placing medical devices on actively moving organs and tissues, and do not automatically and accurately place medical devices in or around organs and tissues.

[0004] For example, a medical device may be placed in the left atrial appendage of the heart. However, the left atrial appendage is dynamic, and it may be difficult to provide accurate timing to place the probe in the correct position to properly place the medical device in the left atrial appendage.

[0005] Further limitations and disadvantages of conventional and traditional approaches will become apparent to those skilled in the art by comparing such systems with certain aspects of the present disclosure as set forth in the remainder of this application with reference to the accompanying figures. Summary of the invention

[0006] There is provided a system and / or method for automatically placing a medical device in an anatomical structure using a locking mechanism, substantially as shown and / or described in conjunction with at least one of the accompanying drawings, as more fully set forth in the claims.

[0007] These and other advantages, aspects and novel features of the present disclosure, as well as details of illustrated embodiments thereof, will be more fully understood from the following description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram of an exemplary ultrasound system operable to automatically place a medical device in an anatomical structure using a locking mechanism in accordance with various embodiments.

[0009] Figure 2 is an exemplary display 300 of a two-dimensional (2D) ultrasound image and a 3D / 4D ultrasound image of an anatomical structure prior to placement of a medical device in accordance with various embodiments.

[0010] Figure 3 is a second exemplary display 400 of a two-dimensional (2D) ultrasound image and a 3D / 4D ultrasound image of an anatomical structure prior to placement of a medical device in accordance with various embodiments.

[0011] Figure 4 is a third exemplary display 500 of a two-dimensional (2D) ultrasound image and a 3D / 4D ultrasound image of an anatomical structure prior to placement of a medical device in accordance with various embodiments.

[0012] Figure 5 is a graphical representation 600 including markings prior to placement of a medical device in an anatomical structure according to various embodiments.

[0013] Figure 6 is an exemplary display 700 of a 2D ultrasound image and a 3D / 4D ultrasound image during placement of a medical device in an anatomical structure, according to various embodiments.

[0014] Figure 7 is a graphical representation 800 during placement of a medical device in an anatomical structure, according to various embodiments.

[0015] Figure 8 is a flow chart 900 illustrating exemplary steps that may be used to automatically place a medical device in an anatomical structure using a locking mechanism in accordance with various embodiments. DETAILED DESCRIPTION

[0016] Certain embodiments may be found in methods and systems for automatically placing a medical device in an anatomical structure using a locking mechanism. Aspects of the present disclosure have the technical effect of automatically providing real-time feedback about the placement of the medical device in a target placement position of the anatomical structure and the position of the medical device relative to the target placement position in a live ultrasound scan. Various embodiments have the technical effect of processing the acquired ultrasound image to identify the target placement position of the medical device and the current position of the medical device in a live ultrasound scan. Certain embodiments have the technical effect of overlaying a live ultrasound scan with a first graphic mark and a second graphic mark, which represent the target placement position and the current position of the medical device in a live ultrasound scan. Aspects of the present disclosure have the technical effect of dynamically updating rendering, first marks and second marks over time. Various embodiments have the technical effect of automatically deploying a medical device in a target placement position using a locking mechanism.

[0017] When reading in conjunction with the accompanying drawings, the aforementioned summary of the invention and the following detailed description of certain embodiments will be better understood. In terms of the scope of the figures of the functional blocks of each embodiment shown in the accompanying drawings, these functional blocks do not necessarily represent the division between the hardware circuits. Therefore, for example, one or more functional blocks (e.g., processors or memories) can be implemented in a single piece of hardware (e.g., a general signal processor or random access memory block, a hard disk, etc.) or multiple pieces of hardware. Similarly, a program can be an independent program, can be included in an operating system as a subroutine, can be a function in an installed software package, etc. It should be understood that each embodiment is not limited to the arrangement and tools shown in the accompanying drawings. It should also be understood that embodiments can be combined, or other embodiments can be utilized, and structural, logical and electrical changes can be made without departing from the scope of various embodiments. Therefore, the following detailed description should not be considered as a restrictive meaning, and the scope of the present disclosure is limited by the attached claims and their equivalents.

[0018] As used herein, elements or steps listed in the singular and beginning with the word "one" or "an" should be understood as not excluding a plurality of said elements or steps, unless such exclusion is explicitly stated. In addition, references to "exemplary embodiments," "various embodiments," "certain embodiments," "representative embodiments," etc. are not intended to be interpreted as excluding the presence of additional embodiments that also include the features of the narration. In addition, unless explicitly stated to the contrary, embodiments that "comprise," "include," or "have" an element or multiple elements having a particular attribute may include additional elements that do not have that attribute.

[0019] In addition, as used herein, the term "image" refers to both visible images and data representing visible images in a broad sense. However, many embodiments generate (or are configured to generate) at least one visible image. In addition, as used herein, the phrase "image" is used to refer to an ultrasound mode, which can be one-dimensional (1D), two-dimensional (2D), three-dimensional (3D) or four-dimensional (4D), and includes brightness mode (B mode), motion mode (M mode), color motion mode (CM mode), color flow mode (CF mode), pulsed wave (PW) Doppler, continuous wave (CW) Doppler, contrast enhanced ultrasound (CEUS) and / or B mode and / or CF mode sub-mode, such as harmonic imaging, shear wave elastic imaging (SWEI), strain elastic imaging, tissue velocity imaging (TVI), power Doppler imaging (PDI), B flow, microangiography (MVI), ultrasound guided attenuation parameters (UGAP) etc.

[0020] Furthermore, as used herein, the term processor or processing unit refers to any type of processing unit that can perform the required computations required by various embodiments, such as a single-core or multi-core CPU, an accelerated processing unit (APU), a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a combination thereof.

[0021] It should be noted that various embodiments of generating or forming an image described herein may include a process for forming an image that includes beamforming in some embodiments and does not include beamforming in other embodiments. For example, an image may be formed without beamforming, such as by multiplying a matrix of demodulated data by a matrix of coefficients such that the product is an image and wherein the process does not form any "beams." Additionally, the formation of an image may be performed using a combination of channels that may originate from more than one transmit event (e.g., synthetic aperture techniques).

[0022] In various embodiments, ultrasound processing to form images, including ultrasound beamforming, such as receive beamforming, is performed, for example, in software, firmware, hardware, or a combination thereof. One specific implementation of an ultrasound system having a software beamformer architecture formed according to various embodiments is described in Figure 1 Shown in.

[0023] Figure 1 is a block diagram of an exemplary ultrasound system operable to automatically place a medical device in an anatomical structure using a locking mechanism in accordance with various embodiments. Figure 1 , an ultrasound system 100 and a training system 200 are shown. The ultrasound system 100 includes a transmitter 102, an ultrasound probe 104, a transmit beamformer 110, a receiver 118, a receive beamformer 120, an analog-to-digital (A / D) converter 122, a radio frequency (RF) processor 124, an RF quadrature (RF / IQ) buffer 126, a user input device 130, a signal processor 132, an image buffer 136, a display system 134, and an archive 138.

[0024] The transmitter 102 may include suitable logic components, circuits, interfaces and / or codes, which are operable to drive the ultrasound probe 104. The ultrasound probe 104 may include a two-dimensional (2D) array of piezoelectric elements. In certain embodiments, the ultrasound probe 104 is a transesophageal ultrasound probe. The ultrasound probe 104 may include a group of transmitting transducer elements 106 and a group of receiving transducer elements 108, which generally constitute the same element. The group of transmitting transducer elements 106 can emit ultrasound signals that pass through the oil and the probe cap and enter the target. In a representative embodiment, the ultrasound probe 104 is operable to collect ultrasound image data covering at least a substantial portion of an anatomical structure, such as a heart, an ovary, or any suitable anatomical structure. In an exemplary embodiment, the ultrasound probe 104 can be operated in a volume acquisition mode, wherein the transducer assembly of the ultrasound probe 104 collects a plurality of parallel 2D ultrasound slices that form an ultrasound volume.

[0025] The transmit beamformer 110 may include suitable logic, circuitry, interfaces, and / or code operable to control the transmitter 102 to drive the set of transmit transducer elements 106 via the transmit sub-aperture beamformer 114 to transmit ultrasound transmit signals into a region of interest (e.g., a human, an animal, an underground cavity, a physical structure, etc.). The transmitted ultrasound signals may be backscattered from structures (such as blood cells or tissue) in the object of interest to generate echoes. The echoes are received by the receive transducer elements 108.

[0026] The set of receive transducer elements 108 in the ultrasound probe 104 may be operable to convert the received echoes into analog signals, perform sub-aperture beamforming via the receive sub-aperture beamformer 116, and then transmit to the receiver 118. The receiver 118 may include suitable logic, circuitry, interfaces, and / or code that may be operable to receive the signals from the receive sub-aperture beamformer 116. The analog signals may be transmitted to one or more of the plurality of A / D converters 122.

[0027] The plurality of A / D converters 122 may include suitable logic components, circuits, and interfaces and / or codes that are operable to convert analog signals from the receiver 118 into corresponding digital signals. The plurality of A / D converters 122 are disposed between the receiver 118 and the RF processor 124. Nevertheless, the present disclosure is not limited in this regard. Therefore, in some embodiments, the plurality of A / D converters 122 may be integrated within the receiver 118.

[0028] The RF processor 124 may include suitable logic components, circuits, interfaces, and / or codes that are operable to demodulate the digital signals output by the plurality of A / D converters 122. According to an embodiment, the RF processor 124 may include a complex demodulator (not shown) that is operable to demodulate the digital signals to form I / Q data pairs representing corresponding echo signals. The RF or I / Q signal data may then be transferred to the RF / IQ buffer 126. The RF / IQ buffer 126 may include suitable logic components, circuits, interfaces, and / or codes that are operable to provide temporary storage of the RF or I / Q signal data generated by the RF processor 124.

[0029] The receive beamformer 120 may comprise suitable logic, circuitry, interfaces, and / or code operable to perform digital beamforming processing, e.g., to sum delayed channel signals received from the RF processor 124 via the RF / IQ buffer 126 and output a beam summed signal. The resulting processed information may be a beam summed signal output from the receive beamformer 120 and communicated to the signal processor 132. According to some embodiments, the receiver 118, the plurality of A / D converters 122, the RF processor 124, and the beamformer 120 may be integrated into a single beamformer, which may be a digital beamformer. In various embodiments, the ultrasound system 100 comprises a plurality of receive beamformers 120.

[0030] The user input device 130 may be used to: enter patient data; scan parameters, settings; select protocols and / or templates; select target structures for automatic detection and tracking; enter and / or select regions of interest; modify regions of interest; select regions of interest and / or activatable buttons corresponding to regions of interest for acquisition of a focus / zoom volume; and the like. Alternatively or additionally, the user input device 130 may be used to place graphical markers on structures, regions of interest, medical devices, and the like. In an exemplary embodiment, the user input device 130 may be operable to configure, manage, and / or control the operation of one or more components and / or modules in the ultrasound system 100. In this regard, the user input device 130 may be operable to configure, manage, and / or control the operation of the transmitter 102, ultrasound probe 104, transmit beamformer 110, receiver 118, receive beamformer 120, RF processor 124, RF / IQ buffer 126, user input device 130, signal processor 132, image buffer 136, display system 134, and / or archive 138. The user input device 130 may include buttons, rotary encoders, touch screens, motion tracking, voice recognition, a mouse device, a keyboard, a camera, and / or any other device capable of receiving user commands. In some embodiments, for example, one or more of the user input devices 130 may be integrated into other components such as the display system 134 or the ultrasound probe 104. For example, the user input device 130 may include a touch screen display.

[0031] The signal processor 132 may include suitable logic components, circuits, interfaces and / or codes that are operable to process ultrasound scan data (i.e., summed IQ signals) to generate ultrasound images for presentation on a display system 134. The signal processor 132 is operable to perform one or more processing operations based on a plurality of selectable ultrasound modalities on the acquired ultrasound scan data. In an exemplary embodiment, the signal processor 132 may be used to perform display processing and / or control processing, etc. The acquired ultrasound scan data may be processed in real time during a scanning session as echo signals are received. Additionally or alternatively, the ultrasound scan data may be temporarily stored in the RF / IQ buffer 126 during a scanning session and processed in a less real-time manner in an online operation or an offline operation. In various embodiments, the processed image data may be presented at the display system 134 and / or may be stored at the archive 138. The archive 138 may be a local archive, a picture archiving and communication system (PACS), or any suitable device for storing images and related information.

[0032] The signal processor 132 may be one or more central processing units, microprocessors, microcontrollers, etc. For example, the signal processor 132 may be an integrated component, or may be distributed in various locations. In an exemplary embodiment, the signal processor 132 may include a structure detection processor 140, a region of interest processor 150, a graphic mark processor 160, a structure tracking processor 170, a catheter tracking processor 180, and a lock processor 190. The signal processor 132 may be capable of receiving input information from the user input device 130 and / or the archive 138, generating output that may be displayed by the display system 134, and manipulating the output in response to the input information from the user input device 130, etc. For example, the signal processor 132, the structure detection processor 140, the region of interest processor 150, the graphic mark processor 160, the structure tracking processor 170, the catheter tracking processor 180, and the lock processor 190 may be capable of performing any of the methods and / or instruction sets discussed herein according to various embodiments.

[0033] The ultrasound system 100 is operable to continuously acquire ultrasound scan data at a frame rate suitable for the imaging situation under consideration. Typical frame rates are in the range of 20 to 120, but may be lower or higher. The acquired ultrasound scan data may be displayed on a display system 134 at a display rate that is the same as the frame rate, or at a display rate that is slower or faster than the frame rate. An image buffer 136 is included to store processed frames of acquired ultrasound scan data that are not scheduled for immediate display. Preferably, the image buffer 136 has sufficient capacity to store ultrasound scan data frames equivalent to at least several minutes. The frames of ultrasound scan data are stored in a manner that is easy to retrieve from them according to their acquisition order or time. The image buffer 136 may be embodied as any known data storage medium.

[0034] The signal processor 132 may include a structure detection processor 140 that includes suitable logic, circuitry, interfaces, and / or code that may be operable to analyze the acquired ultrasound images and / or volumes to detect the presence and location of anatomical structures in the ultrasound images and / or volumes. In this regard, the structure detection processor 140 may include, for example, an image analysis algorithm, one or more deep neural networks (e.g., convolutional neural networks, such as u-net), and / or may utilize any suitable form of image analysis technology, artificial intelligence, or machine learning processing functionality configured to detect and locate anatomical structures in the ultrasound images and / or volumes. Additionally and / or alternatively, the image analysis technology, artificial intelligence, or machine learning processing functionality configured to detect and locate anatomical structures in the ultrasound images and / or volumes may be provided by different processors, or distributed across multiple processors at the ultrasound system 100 and / or distributed across a remote processor communicatively coupled to the ultrasound system 100. For example, the structure detection and localization functionality may be provided as a deep neural network that may be comprised of, for example, an input layer, an output layer, and one or more hidden layers between the input layer and the output layer. Each layer may be composed of a plurality of processing nodes, which may be referred to as neurons. For example, a structure detection and localization function may include an input layer having a neuron for each pixel of an ultrasound image and / or a voxel of an ultrasound volume. An output layer may have neurons corresponding to each myocardium, ventricle, and / or any suitable anatomical structure. Each neuron of each layer may perform a processing function and pass the processed ultrasound image information to one of a plurality of neurons in a downstream layer for further processing. As an example, neurons in a first layer may learn to identify structural edges in an acquired ultrasound image and / or volume. Neurons in a second layer may learn to identify shapes based on detected edges from the first layer. Neurons in a third layer may learn the position of the identified shapes relative to landmarks in an acquired ultrasound image and / or volume. The processing performed by a deep neural network may identify anatomical structures and the positions of anatomical structures in an acquired ultrasound image and / or volume with a high probability.

[0035] In an exemplary embodiment, the structure detection processor 140 may be configured to analyze the ultrasound image and / or volume to detect and locate the target structure. For example, the structure detection processor 140 may be configured to receive user input selecting the target structure before performing an initial ultrasound image acquisition and analyzing the ultrasound image and / or volume of the initial ultrasound image acquisition to detect and locate the target structure.

[0036] Figure 2 FIG. 3 is an exemplary display 300 of two-dimensional (2D) ultrasound images 304, 308 and a 3D / 4D ultrasound image 312 of an anatomical structure prior to placement of a medical device according to various embodiments. Figure 2, the display 300 includes: a first image display portion 302 including a first ultrasound image 304, a second image display portion 306 including a second ultrasound image 308, and a third display portion 310 for displaying a third ultrasound image 310. Figure 2 As shown, display 300 may be provided on a main display of display system 134 having image display portions 302, 306, 310 and / or may be provided on a touch panel of display system 134. A third display portion may include a region of interest 314 that may be automatically determined or may be selected by a user via user input.

[0037] User input via the user input device 130 may trigger an initial ultrasound image acquisition by the ultrasound probe 104. The initial ultrasound image acquisition may be a 2D image acquisition, such as acquiring a 2D ultrasound image for analysis by the structure detection processor 140. Additionally and / or alternatively, the initial ultrasound image acquisition may be a volumetric acquisition. The structure detection processor 140 may be configured to analyze the acquired volume to detect and locate the structure 312. The structure detection processor 140 may provide the detected and located target structure 312 to the region of interest processor 150. Additionally and / or alternatively, the detected and located target structure 312 may be stored in the archive 138 and / or at any suitable computer-readable medium. Additional ultrasound image acquisitions may be acquired automatically and / or in response to user input. The additional ultrasound image acquisitions may be 2D images, 3D / 4D images, or volumetric acquisitions. For example, if no user input is received within a predetermined time period, the structure detection processor 140 may automatically initiate additional ultrasound image acquisitions by the ultrasound probe 104. As another example, the structure detection processor 140 may be configured to initiate additional ultrasound image acquisition by the ultrasound probe 104 in response to user input.

[0038] The first acquired image obtained through the first image may be placed in the first image display portion 302 as a 2D ultrasound image 304, the second acquired image may be placed in the second image display portion 306 as a 2D ultrasound image 308, and the third acquired image may be placed in the image display portion 310 as a 3D / 4D volume 312. Although the 2D images 304, 308, and the 3D / 4D image 312 are depicted in respective locations on the display, the locations of the 2D ultrasound images 304, 308, and the 3D / 4D image 312 may be customized and displayed in different locations on the display 400.

[0039] See again Figure 1, the structure detection processor 140 may be configured to analyze the ultrasound image and / or volume to detect and locate all anatomical structures present in the ultrasound image and / or volume. For example, the structure detection processor 140 may be configured to analyze the ultrasound image and / or volume of an initial ultrasound image acquisition to detect and locate all anatomical structures present in the ultrasound image and / or volume without receiving a target structure and / or before receiving a target structure. The structure detection processor 140 may provide the detected and located anatomical structures to the region of interest processor 150. Additionally or alternatively, the detected and located anatomical structures may be stored in the archive 138 and / or any suitable computer-readable medium, and the region of interest processor 150 may obtain the detected and located anatomical structures from the archive 138 and / or any suitable computer-readable medium.

[0040] The signal processor 132 may include a region of interest processor 150 including suitable logic, circuitry, interfaces, and / or code operable to cause the display system 134 to present an initial ultrasound image acquisition of a region of interest within, near, or around a structure detected and located by the structure detection processor 140. For example, the region of interest processor 150 may be configured to receive the identity and location of the target structure from the structure detection processor 140 or retrieve it from the archive 138 and / or any suitable data storage medium. The region of interest processor 150 may be configured to identify the local target structure by overlaying a bounding box, coloring the pixels, labeling, and / or any suitable identification technique. The initial ultrasound image acquisition with the identified region of interest may include a 2D image, a 2D biplane image, a 2D image slice extracted from a volume, a rendered volume, and / or any suitable ultrasound image and / or volume rendering (i.e., a 2D projection of 3D / 4D volumetric image data). The region of interest 314 may be determined automatically or in response to user input. The region of interest 314 may be modified by user input received via the user input device 130 and / or the touch screen display 130, 134 to adjust the position and / or size of the region of interest 314. The region of interest 314 may be provided by the region of interest processor 150 to the graphical indicia processor 160.

[0041] Figure 3 FIG. 4 is a second exemplary display 400 of two 2D ultrasound images 404, 408 and a 3D / 4D ultrasound image 412 of an anatomical structure during placement of a medical device according to various embodiments. Figure 3 , the display 400 includes: a first image display portion 402 including a 2D ultrasound image 404 and a second image display portion 406 including a second 2D ultrasound image 408. The detected and located target region of interest 414 may also be set on a display having a Figure 3 The image display portions 402, 406 are shown on a main display 410 of the display system 134 and / or may be provided on a touch panel of the display system 134. The graphic indicia 418 and the graphic indicia 420 may be overlaid on the 2D ultrasound images 404, 408 and / or on the 3D / 4D ultrasound image 412.

[0042] The user input device 130 can be used to initiate the acquisition of the 2D ultrasound images 404, 408 and / or the 3D / 4D ultrasound images 412, or the 2D ultrasound images 404, 408 and 3D / 4D ultrasound images 412 can be obtained automatically. The 2D images 404, 408 can additionally and / or alternatively be a single 2D image, a rendering of a volume (3D / 4D), a 2D image slice extracted from a volume (3D / 4D), and / or any suitable ultrasound image. Although the 2D images 404, 408 and 3D / 4D images 412 are depicted in various locations on the display, the positions of the 2D ultrasound images 404, 408 and 3D / 4D images 412 can be customized and displayed in different locations on the display 400.

[0043] The graphical marker 418 may be used to indicate the current position in space and time of a catheter that will be used to place a medical device at a target placement location within the region of interest 414. The graphical marker 420 may be used to indicate the target placement location of the medical device in space and time within the region of interest. The graphical marker 420 indicating the target placement location of the medical device may be automatically obtained using the structure detection processor 140 and / or the region of interest processor 150, or may be input by a user to visually indicate the location within the region of interest 414. If the target placement location is automatically obtained, the target placement location may be modified via user input. The 3D / 4D ultrasound 412 may be overlaid with the graphical marker 418 and the graphical marker 420 to indicate to the operator the proximity between the graphical marker 418 and the graphical marker 420. Additionally or alternatively, the target placement location may be provided to the lock processor 190 by the graphical marker processor 160.

[0044] See also Figure 1, the graphic mark processor 160 may include suitable logic, circuitry, interfaces, and / or code that may be configured to automatically overlay a graphic mark on, within, or near a target structure or region of interest in response to user input or based on information provided by the structure detection processor 140 and / or the region of interest processor 150. For example, the graphic mark processor 160 may place a graphic mark 418 on the 2D ultrasound image 404 and a graphic mark 420 on the 2D ultrasound image 408. The graphic mark 418 may be used to identify a first location that corresponds to a location in space and time of a catheter that will be used to place a medical device at a target location. The graphic mark processor 160 may overlay the 2D ultrasound image 404 and / or the 3D / 4D image 412 with the graphic mark 418 to indicate the current location of the catheter. The graphic mark processor 160 may place the graphic mark 420 on the 2D ultrasound image 408. The graphic mark processor 160 may overlay the 2D ultrasound image 408 and / or the 3D / 4D image 412 with a graphic mark 420 to identify a second location representing a target placement location in space and time of the medical device. The graphic mark processor 160 may place both the first mark 418 and the second mark 420 on the 3D / 4D image 412 to provide real-time location information for the target placement location and the catheter. As a non-limiting example, the graphic mark processor 160 may provide the following: Figure 3 The depicted circular visual indicator (see graphical indicia 418 and graphical indicia 420), provides visual indicators of different shapes, colors pixels, provides different graphical effects (e.g., blinking, flashing, pulsing, color changes, etc.), and can be customized by the user. The graphical indicia processor can also output an alert (such as a sound, a visual indicator, a text indicator), and can change the visual indication based on the proximity from the first indicia 418 to the second indicia 420 or based on the proximity from the first indicia 418 and / or the second indicia to additional indicia and / or to the area of ​​interest 414.

[0045] Although in Figure 3 4, but additional graphic markers may be included in the 2D ultrasound images 404, 408 and / or the 3D / 4D image 412. The graphic marker processor 160 may receive information from the structure tracking processor 170 and the catheter tracking processor 180 to update the placement of the graphic marker 418 and the graphic marker 420 on the 2D ultrasound images 404, 408 and / or on the 3D / 4D image 412, respectively.

[0046] Figure 4 FIG. 5 is a third exemplary display 500 of two 2D ultrasound images and a 3D / 4D ultrasound image of an anatomical structure during placement of a medical device according to various embodiments. Figure 4, the display 500 includes: a first image display portion 502 including a 2D ultrasound image 504 and a second image display portion 506 including a second 2D ultrasound image 508. The detected and located target region of interest 514 may also be set on a display having a Figure 4 The first image display portion 502 and the second image display portion 506 are shown on the main display of the display system 134 and / or can be set on the touch panel of the display system 134. The graphic mark 518 and the graphic mark 520 can be overlaid on the 2D ultrasound images 504, 508 and / or on the region of interest 514 of the 3D / 4D ultrasound image 512. The 3D / 4D ultrasound can be overlaid with the graphic mark 518 and the graphic mark 520 to indicate to the operator the location of the graphic mark 518 and the second graphic mark and / or the proximity between the graphic mark 518 and the graphic mark 520.

[0047] Similar to Figure 3 , the user input device 130 may be used to initiate acquisition of a 2D ultrasound image 504, 508 and / or a 3D / 4D ultrasound image 512, or the ultrasound image may be automatically acquired, and each of the ultrasound images 504, 508, 512 may be placed in a different arrangement on the display 500. The 2D images 404, 408 may additionally and / or alternatively be a single 2D image, a rendering of a volume (3D / 4D), a 2D image slice extracted from a volume (3D / 4D), and / or any suitable ultrasound image.

[0048] Figure 5 is a graphical representation 600 of a medical device prior to placement in an anatomical structure according to various embodiments. Figure 5 According to various embodiments, graphical marker 518 indicates a location corresponding to the location of catheter 610 in space and time, and graphical marker 520 designates a location representing a target placement location in space and time for catheter 610. The location of catheter 610 (e.g., graphical marker 518) is tracked as catheter 610 (indicated by graphical marker 518) moves toward the target placement location indicated by graphical marker 520. The target placement location indicated by graphical marker 520 may be tracked in space and time by structure tracking processor 170, and the location indicated by graphical marker 518 and representing the location of catheter 610 may be tracked in space and time by catheter tracking processor 180.

[0049] See also Figure 1, the structure tracking processor 170 includes suitable logic components, circuits, interfaces and / or codes that are operable to track the target structure and / or region of interest around the target placement location of the medical device. The catheter tracking processor 180 includes suitable logic components, circuits, interfaces and / or codes that are operable to track the spatial and temporal position corresponding to the catheter. For example, when the catheter (represented by graphic mark 518) is guided toward the target location for placement of the medical device (represented by graphic mark 520), the catheter tracking processor tracks the position of the catheter, and the structure tracking processor tracks the target placement location.

[0050] The structure tracking processor 170 can track the second position of the catheter (e.g., by using artificial intelligence and / or machine learning) Figure 5 ), and the catheter tracking processor 180 may track the first position of the catheter (e.g., indicated by the graphical marker 520 in FIG. 1 ). Figure 5 518 in the figure). In some examples, the structure tracking processor 170 and the catheter tracking processor 180 can track the second position and the first position, respectively, by performing atlas segmentation. Atlas segmentation involves the use of an atlas, which is a labeled map of anatomical structures (e.g., left atrial appendage (LAA) and / or LAA structures, mitral valve, gastrointestinal structures, urinary structures, reproductive structures, cardiac structures, lung structures, and / or any suitable anatomical structure). The atlas can be created by capturing an ultrasound image of the anatomical structure and then manually segmenting the ultrasound image. The atlas can then be used to segment new, unseen images using a process known as registration (e.g., template matching). For example, the structure tracking processor 170 can transform and deform the acquired ultrasound image so that structures on the acquired ultrasound image are aligned with corresponding structures in the manually segmented ultrasound image in the atlas. A mathematical scoring function can be used to evaluate how well one image is aligned with another image, which helps to locate and track anatomical structures and allows the structure tracking processor 170 to track the placement of targets within the region of interest in space and time. The structure tracking processor 170 may provide the target placement location to the graphical mark processor 160 so that the graphical mark processor 160 may update the graphical mark 520. The structure tracking processor 170 may also provide the target placement location to the lock processor 190. Similarly, the catheter tracking processor 180 may use registration to determine the current location of the catheter in space and time and provide the current location to the graphical mark processor 160 to update the graphical mark 518 and / or to the lock processor 190.

[0051] Additionally or alternatively, the structure tracking processor 170 can track the second position by using a machine learning method. In some examples, the machine learning method uses a segmentation mask to generate a mask reconstruction network, uses the original image to generate a segmentation network, and uses a potential space alignment, which performs feature alignment between the original image and the segmentation network with a constant learning loss to enhance feature alignment. The mask reconstruction network can be based on a ConvNeXt model to obtain the latent space features of the mask. A U-shaped structure consisting of an encoder, a decoder, and a jump connection layer can be used to construct a LAA segmentation network. The encoder enables the extraction of shallow features and deep features of the fused image to obtain a latent space, and then the decoder is used to restore the mask prediction result from the latent space. The jump connection layer fuses the feature map at each level of the encoder with the feature map obtained by the upsampling decoder layer, thereby allowing the decoder to access the high-level features learned by the encoder and helping the decoder to accurately preserve the details of the input image. Generative adversarial learning (GAN) can be used to align the latent space of the segmentation network with the latent space vector of the mask reconstruction network, thereby introducing prior knowledge of the LAA mask. The latent space alignment loss combining the adversarial-based loss and the contrastive learning loss aligns the latent space of the reconstructed network with the latent space of the segmentation network. The contrastive learning loss can be used to enhance the association between the previous latent features and the latent space features of the original image to improve the LAA segmentation accuracy. In this way, the structure tracking processor 170 can use the acquired ultrasound image to determine the spatial and temporal location of the target placement position within the region of interest within or near the anatomical structure. The structure tracking processor 170 can provide the location of the target placement position within the region of interest and / or structure to the graphic marking processor 160 and / or the locking processor 190. Although the GAN model is described above with respect to the LAA, the GAN model can utilize other structures, such as the mitral valve, gastrointestinal structures, urinary structures, reproductive structures, cardiac structures, lung structures and / or any suitable anatomical structure.

[0052] The structure tracking processor 170 and the catheter tracking processor 180 may each use different artificial intelligence and / or machine learning methods, and both may use the same methods, a combination of methods, or methods similar to those described above. The structure tracking processor 170 and the catheter tracking processor 180 may provide the first position and / or the second position to the graphic marking processor 160 and / or the locking processor 190, and may additionally or alternatively provide the first position and / or the second position to the archive 138 and / or any suitable computer-readable medium.

[0053] Figure 6FIG. 7 is an exemplary display 700 of two 2D ultrasound images and a 3D / 4D ultrasound image of an anatomical structure during placement of a medical device according to various embodiments. The display 700 includes a first image display portion 702 including a 2D ultrasound image 704 and a second image display portion 706 including a second 2D ultrasound image 708. The detected and located region of interest 714 may also be set on a display having a 2D ultrasound image 704. Figure 4 The first image display portion 702 and the second image display portion 706 are shown on the main display of the display system 134 and / or may be provided on the touch panel of the display system 134. The combined graphic mark 730 may be overlaid on the 2D ultrasound images 704, 708 and / or on the region of interest 714 of the 3D / 4D ultrasound image 708. The combined graphic mark 730 may be overlaid on the display by the graphic mark processor 160 and may indicate that two graphic marks (e.g., Figure 4 518, 520) overlap (for example, the first graphic mark has reached the second graphic mark).

[0054] See also Figure 1 The locking processor 190 includes suitable logic components, circuits, interfaces and / or codes, which are operable to provide a locking mechanism that specifies the target placement position of the medical device and guides the catheter to the target placement position. The locking processor 190 can be configured to receive current position information (e.g., the spatial and temporal position and / or structure of the target placement position in the region of interest and the spatial and temporal position of the catheter) from the structure tracking processor 170 and the catheter tracking processor 180, respectively. The locking processor 190 can use the locking mechanism to define the target placement position in the region of interest using the position information received from the structure tracking processor. The locking processor can then use the current spatial and temporal position of the catheter obtained from the catheter tracking processor 180 to determine the proximity of the catheter to the target placement position (e.g., the target for the locking mechanism), and when the catheter reaches the target placement position, the locking processor 190 can send an output signal. In some examples, the output signal is a signal for automatically releasing the catheter into the target placement position in the region of interest and / or structure. In some examples, the output signal can be a notification, such as a visual indicator (e.g., an alert on display system 134), an auditory indicator (e.g., a sound), and / or a physical indicator (e.g., vibration of an ultrasound probe), so that the operator can release the catheter to place the medical device in the target placement location.

[0055] Figure 7 is a graphical representation 800 of a medical device during placement in an anatomical structure according to various embodiments. Figure 7According to various embodiments, graphical representation 800 includes catheter 610, medical device 720, and overlaying graphical marker 740 during placement of the medical device in an anatomical structure. As catheter 610 moves toward a target placement location, the location of catheter 610 is tracked by catheter tracking processor 180. The target placement location is also tracked in space and time by structure tracking processor 170. Lock processor 190 may obtain the current target placement location from structure tracking processor 170 and the catheter location from catheter tracking processor 180. As the catheter moves toward the target placement location, lock processor 190 may track the current location of the catheter. Once catheter 610 reaches the target placement location (as determined by structure tracking processor 170), lock processor 190 may track the current location of the catheter. Figure 7 The locking processor 190 determines that the position of the catheter matches the target placement position of the medical device 720 and can output a signal indicating the match. Additionally or alternatively, the graphic mark processor 160 may determine that the first mark 518 overlaps the second mark 520. Once the determination is made, the locking processor sends an output signal. The output signal can be a signal to automatically release the medical device 720 into the target placement position within the structure. In some examples, the output signal can be a notification (such as a visual indicator or an audible indicator) so that the operator can manually release the catheter to be placed in the target position.

[0056] See again Figure 1 The display system 134 may be any device capable of conveying visual information to a user. For example, the display system 134 may include a liquid crystal display, a light emitting diode display, and / or any suitable one or more displays. The display system 134 may be operable to present the 2D ultrasound images 304, 308, 404, 408, 504, 508, 704, 708, 804, 806, the rendered 3D / 4D volumes 312, 412, 512, 712, and / or any suitable information.

[0057] The archive 138 may be one or more computer-readable memories, such as a picture archiving and communication system (PACS), a server, a hard disk, a floppy disk, a CD, a CD-ROM, a DVD, a compressed storage device, a flash memory, a random access memory, a read-only memory, an electrically erasable and programmable read-only memory, and / or any suitable memory, that are integrated with the ultrasound system 100 and / or communicatively coupled to the ultrasound system 100 (e.g., via a network). The archive 138 may include, for example, a database, a library, a collection of information, or other memory that is accessed by and / or associated with the signal processor 132. For example, the archive 138 may be capable of storing data temporarily or permanently. The archive 138 may be capable of storing medical image data, data generated by the signal processor 132, and / or instructions readable by the signal processor 132, etc. In various embodiments, the archive 138 stores 2D ultrasound images 304, 308, 404, 408, 504, 508, 704, 708, 804, 806; rendered 3D / 4D volumes 312, 412, 512, 712; instructions for automatically detecting and tracking target structures and other anatomical structures; instructions for causing the display system 134 to present regions of interest 314, 414, 514, 714 within or near the target structures and other anatomical structures; instructions for triggering additional ultrasound image acquisitions (e.g., 3D / 4D volume acquisitions); instructions for identifying and tracking target placement locations of medical devices within the regions of interest; instructions for identifying and tracking catheters within anatomical structures; instructions for overlaying ultrasound image acquisitions with graphic markers and dynamically updating the graphic markers about the ultrasound image acquisitions over time; and instructions for outputting signals when the graphic markers overlap and / or when the position of the catheter matches the target placement location.

[0058] The components of the ultrasound system 100 may be implemented in software, hardware, firmware, etc. The various components of the ultrasound system 100 may be communicatively connected. The components of the ultrasound system 100 may be implemented separately and / or integrated in various forms. For example, the display system 134 and the user input device 130 may be integrated into a touch screen display.

[0059] Still see Figure 1, the training system 200 may include a training engine 210 and a training database 220. The training engine 210 may include suitable logic components, circuits, interfaces, and / or code that are operable to train neurons of a deep neural network (e.g., an artificial intelligence model) inferred (i.e., deployed) by the structure detection processor 140, the region of interest processor 150, the graphic marking processor 160, the structure tracking processor 170, and / or the catheter tracking processor 180. For example, the artificial intelligence model inferred by the structure detection processor 140 may be trained to automatically identify anatomical structures depicted in the ultrasound image and / or volume using the database 220 of classified ultrasound images and / or volumes of anatomical structures. For another example, the artificial intelligence model inferred by the structure detection processor 140 and / or the region of interest processor 150 may be trained to automatically identify a target structure, surrounding structures, target structure shape, major axis / minor axis of a target structure, etc. depicted in the ultrasound volume using the database 220 of classified ultrasound volumes of possible target structures.

[0060] In various embodiments, the database 220 of training images may be a picture archiving and communication system (PACS) or any suitable data storage medium. In certain embodiments, the training engine 210 and / or the training image database 220 may be a remote system communicatively coupled to the ultrasound system 100 via a wired or wireless connection, such as Figure 1 Additionally and / or alternatively, components or all of the training system 200 may be integrated with the ultrasound system 100 in various forms. In some examples, the training image database may include an atlas (e.g., a labeled map of an anatomical structure, a raw image, a segmentation mask as a non-limiting example).

[0061] Figure 8 900 is a flowchart illustrating exemplary steps 902 to 922 that may be used to automatically place a medical device in an anatomical structure using a locking mechanism according to various embodiments. Certain embodiments may omit one or more steps, and / or perform the steps in a different order than listed, and / or combine certain steps discussed below. For example, some steps may not be performed in certain embodiments. For another example, certain steps may be performed in a different time order than listed below, including simultaneously.

[0062] At step 902, the signal processor 132 of the ultrasound system 100 may be configured to receive user input selecting a target structure 312. For example, the structure detection processor 140 may be configured to receive user input selecting a target structure 312 (e.g., as non-limiting examples, a left atrial appendage (LAA), a mitral valve, a gastrointestinal structure, a urinary structure, a reproductive structure, a cardiac structure, a lung structure, and / or any suitable anatomical structure) via the user input device 130. The structure detection processor 140 may initiate an initial ultrasound image acquisition in response to the selection of the target structure 312.

[0063] At step 904, the ultrasound probe 104 of the ultrasound system 100 performs an initial ultrasound image acquisition. For example, the ultrasound probe 104 may be operable to acquire 2D ultrasound images 304, 308, 404, 408, 504, 508, 704, 708, 3D / 4D volumes 312, 412, 512, 712, and / or any suitable ultrasound images. The acquired ultrasound images 304, 308, 404, 408, 504, 508, 704, 708, and / or volumes 312, 412, 512, 712 of the initial ultrasound image acquisition may be provided to the structure detection processor 140 and / or stored at the archive 138 and / or any suitable computer-readable medium.

[0064] At step 906, the signal processor 132 of the ultrasound system 100 automatically detects and tracks the target structure 312 in the initial ultrasound image acquisition. For example, the structure detection processor 140 may be configured to analyze the acquired ultrasound images 304, 308, 404, 408, 504, 508, 704, 708, and / or volumes to detect the presence and location of the selected target structure 312 in the ultrasound images 304, 308, 404, 408, 504, 508, 704, 708, and / or volumes 312, 412, 512, 712. The structure detection processor 140 may include, for example, an image analysis algorithm, one or more deep neural networks (e.g., convolutional neural networks such as u-net), and / or may utilize any suitable form of image analysis technology, artificial intelligence, or machine learning processing functionality configured to detect and locate anatomical structures in ultrasound images and / or volumes. The structure detection processor 140 may provide the detected and located target structure 312 to the region of interest processor 150. Additionally and / or alternatively, the detected and located target structure 312 may be stored at the archive 138 and / or any suitable computer-readable medium.

[0065] At step 908, the signal processor 132 of the ultrasound system 100 presents the initial ultrasound image acquisition 304, 308, 404, 408, 504, 508, 704, 708 to the region of interest 314, 414, 514, 714 within or near the target structure 312, 412, 512, 712 at the display system 134. For example, the region of interest processor 150 of the signal processor 132 may be configured to cause the display system 134 to present the initial ultrasound image acquisition 304, 308 of the region of interest 408 within or near the selected target structure 312 detected and located by the structure detection processor 140. The region of interest processor 150 may be configured to receive from the structure detection processor 140 or retrieve from the archive 138 and / or any suitable data storage medium the identity and location of the selected target structure 312. The region of interest processor 150 may be configured to identify the locally selected target structure by overlaying a marker, a bounding box, coloring the pixels, and / or any suitable identification technique. The region of interest 314, 414, 514, 714 may be automatically detected or may be input by a user.

[0066] At step 910, the signal processor 13 of the ultrasound system 100 can process the acquired ultrasound image to identify an anatomical structure location that is a target placement location for a medical device in the region of interest. For example, the graphic mark processor 160 can be configured to indicate the anatomical structure location on the display using a graphic mark 420, 520. The anatomical structure location can be automatically determined by the graphic mark processor 160 using information obtained from the region of interest processor 150, or can be input by a user. The graphic mark processor 160 can provide the anatomical structure location to the structure tracking processor 170 and / or the lock processor, and can provide real-time updates of the anatomical structure location at the display system 134.

[0067] At step 912, the signal processor may process the acquired second ultrasound image to identify a current medical device position corresponding to the current position of the medical device to be placed in the anatomical structure location. For example, the graphic mark processor 160 may be configured to indicate the current medical device position on the display using the graphic mark 418, 518. The current medical device position may be automatically determined by the graphic mark processor 160 using information obtained from the region of interest processor 150 and / or the structure processor. The graphic mark processor 160 may provide the current medical device position to the catheter tracking processor 180 and / or the locking processor 190, and may provide real-time updates of the current medical device position at the display system 134.

[0068] At step 914, the signal processor may process the first ultrasound image and the second ultrasound image to overlay the acquired ultrasound image with a first graphical marker representing the location of the anatomical structure and / or a second graphical marker representing the current medical device location. The graphical marker processor 160 may provide the location of the anatomical structure and the current medical device location to the structure tracking processor 170 and the catheter tracking processor 180, respectively, and may provide real-time updates of the location of the anatomical structure at the display system 134. The 3D / 4D ultrasound 412, 512, 712 may be overlaid with the graphical markers 418, 518 and the graphical markers 420, 520. Additionally or alternatively, the location of the anatomical structure and / or the current medical device location may be provided by the graphical marker processor 160 and / or the lock processor 190.

[0069] At step 916, the signal processor tracks the structure and the anatomical structure position over time and updates the overlaid ultrasound image. For example, the structure tracking processor 170 and the catheter tracking processor 180 can obtain the anatomical structure position and the current medical device position, respectively, and can provide real-time updates of the anatomical structure position and the current medical device position to the graphical marker processor 160. 3D / 4D ultrasound 412, 512, 712 with overlaid graphical markers 418, 518, 420, 520. Additionally or alternatively, the structure tracking processor 170 and the catheter tracking processor 180 can provide real-time updates of the anatomical structure position and / or the current medical device position to the graphical marker processor 160 to dynamically update the display system 134 over time and / or provide real-time updates of the anatomical structure position and / or the current medical device position to the lock processor 190.

[0070] At step 918, the signal processor tracks the anatomical structure position and the current medical device position in space and time, and when the anatomical structure position and the current medical device position match, the signal processor outputs a signal. For example, the structure tracking processor 170 and the catheter tracking processor 180 can provide real-time updates of the anatomical structure position and / or the current medical device position to the locking processor 190, and when the locking processor 190 detects a match, the locking processor 190 can output a signal. In some examples, the output signal is a signal for automatically releasing the catheter into the target placement position within the region of interest and / or structure. In some examples, the output signal can be a notification, such as a visual indicator (e.g., an alert on the display system 134), an auditory indicator (e.g., a sound) and / or a physical indicator (e.g., a vibration of an ultrasound probe), so that the operator can release the catheter to place the medical device in the anatomical structure position (i.e., the target placement position).

[0071] At step 920, the output signal is used to deploy the medical device. In some examples, the output signal is a signal for automatically releasing the catheter into a target placement location within the region and / or structure of interest. In some examples, the output signal can be a notification, such as a visual indicator (e.g., an alert on the display system 134), an auditory indicator (e.g., a sound), and / or a physical indicator (e.g., a vibration of the ultrasound probe), so that the operator can release the catheter to place the medical device in the target placement location.

[0072] At step 922, the signal processor determines whether additional medical devices are to be deployed. If there are additional medical devices to be placed in or around the anatomical structure, the method will start again at step 902. In some examples, a medical device (such as a wristwatch device) may be placed, and then a mitral valve clip may be placed. In some other examples, a mitral valve clip is placed and then the wristwatch device is placed. In some examples, one or more medical devices (such as gastroenterology devices, urology devices, cardiac devices, pulmonary devices, reproductive devices, etc.) may be placed via process 900, and additional medical devices may be placed by repeating process 900 at step 922. If no additional medical devices are to be placed in or around the anatomical structure, process 900 ends at step 924.

[0073] Aspects of the present disclosure provide a method 900 and system 100 for placing a medical device 720 in an anatomical structure, the method comprising: performing 902 one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 by an ultrasound probe 104 of an ultrasound system 100. The method 900 may include: detecting and tracking, by at least one processor 132, 140, 150, 170 of the ultrasound system, one or more anatomical structures in the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712. The method 900 may include causing 908, by the at least one processor 132, 140, 150, a display system to present a volume 412, 512, 712 of a region of interest 314, 414, 514, 714 within or around one or more anatomical structures in the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712. The method 900 may include processing 910, by the at least one processor 132, 140, 170, the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 to identify a first location corresponding to a target placement location within the region of interest 314, 414, 514, 714 of the medical device 720. The method 900 may include processing 912, by the at least one processor 132, 180, one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 to identify a second location corresponding to the location of the catheter 610 including the medical device 720. The method 900 may include outputting 916, by the at least one processor 132, 170, 180, 190, a signal to deploy 920 the medical device 720 when the second location corresponding to the location of the catheter 610 matches the first location corresponding to the target placement location 420, 520 of the medical device 720.

[0074] In an exemplary embodiment, the method 900 includes rendering 914, by at least one processor 132, 140, 150, 160, one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712, the one or more ultrasound image acquisitions overlaid with a first graphical marker 418, 518 corresponding to a first location and a second graphical marker 420, 520 corresponding to a second location. In a representative embodiment, the method 900 also includes updating 916, by at least one processor 132, 160, the first graphical marker 418, 518 corresponding to the target placement location when the first location is tracked by the at least one processor 132, 170, and updating 916, by the at least one processor 132, 180, the second graphical marker 420, 520 corresponding to the catheter 610 when the second location is tracked by the at least one processor 132, 180. In various embodiments, the method 900 includes: the matching of the first position and the second position is indicated by the first graphical mark 418, 518 overlapping 730, 740 with the second graphical mark 420, 520. In some exemplary embodiments, the method 900 includes: automatically deploying the medical device 720 when the signal is output. In an exemplary embodiment, the method 900 includes: transmitting a notification to an operator to deploy the medical device 720 when the signal is output. In an exemplary embodiment, the one or more anatomical structures are a left atrial appendage (LAA).

[0075] Various embodiments provide an ultrasound system 100 comprising an ultrasound probe 104 configured to perform one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712; and at least one processor 132, 140, 170 configured to detect and track one or more anatomical structures in the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712. The at least one processor may be configured to cause the display system to present a volume of the region of interest 314, 414, 514, 714 within or around one or more anatomical structures in the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712. The at least one processor may be configured to process the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 to identify a first position corresponding to a target placement position within the region of interest 314, 414, 514, 714 of the medical device 720. The at least one processor may be configured to process the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 to identify a second position corresponding to the position of the catheter 610 including the medical device 720. The at least one processor may be configured to output a signal to deploy the medical device 720 when the second position corresponding to the position of the catheter 610 matches the first position corresponding to the target placement position of the medical device 720.

[0076] In a representative embodiment, the at least one processor is configured to render one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 overlaid with a first graphical marker 418, 518 corresponding to the first location and a second graphical marker 420, 520 corresponding to the second location. In various embodiments, the at least one processor 132, 160 is further configured to update the first graphical marker 418, 518 corresponding to the target placement location as the first location is tracked by the at least one processor 132, 170, and update the second graphical marker 420, 520 corresponding to the catheter 610 as the second location is tracked by the at least one processor 132, 180. In certain embodiments, the at least one processor is further configured such that a match between the first location and the second location is indicated by the first graphical marker 418, 518 overlapping the second graphical marker 420, 520. In various embodiments, the at least one processor is further configured to automatically deploy the medical device 720 when the signal is output. In certain embodiments, the at least one processor 132, 190 is further configured to transmit a notification to an operator to deploy the medical device 720 when the signal is output. In a representative embodiment, the one or more anatomical structures are a left atrial appendage (LAA).

[0077] Various embodiments provide an ultrasound system 100 for placing a medical device 720 in an anatomical structure, the ultrasound system comprising an ultrasound probe 104 configured to perform one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712. The at least one processor 132 may be configured to detect and track a left atrial appendage (LAA) in the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712. The at least one processor may be configured to cause the display system to present a volume of the region of interest 314, 414, 514, 714 within or around the LAA in the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712. The at least one processor may be configured to process the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 to identify a first position within the region of interest 314, 414, 514, 714 corresponding to a target placement position of the medical device 720. The at least one processor may be configured to process the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 to identify a second location corresponding to the location of the catheter 610 including the medical device 720. The at least one processor 132, 140, 150, 160 may be configured to render the one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 with an indication of the first location and an indication of the second location. The at least one processor 132, 190 may be configured to output a signal to automatically deploy the medical device 720 when the second location corresponding to the location of the catheter 610 matches the first location corresponding to the target placement location of the medical device 720.

[0078] In a representative embodiment, the at least one processor 132, 160 may be further configured to overlay one or more ultrasound image acquisitions 304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712 with a first graphical marker 418, 518 corresponding to the first location and a second graphical marker 420, 520 corresponding to the second location. In certain embodiments, the at least one processor 132, 160, 170, 180 may be further configured to update the first graphical marker 418, 518 corresponding to the target placement location as the first location is tracked by the at least one processor, and update the second graphical marker 420, 520 corresponding to the catheter 610 as the second location is tracked by the at least one processor. In certain embodiments, the first indication is the first graphical marker 418, 518 and the second indication is the second graphical marker 420, 520. In a representative embodiment, the at least one processor 132, 160, 170, 180 is also configured to dynamically update the first graphical indicia 418, 518 as the target placement location is updated, and dynamically update the second graphical indicia 420, 520 as the catheter 610 moves toward the target placement location. In certain embodiments, the at least one processor 132, 190 can be configured to provide a visual or audible notification when the second location matches the first location.

[0079] As used herein, the term "circuit" refers to physical electronic components (e.g., hardware) and any software and / or firmware ("code") that can configure hardware, is executed by hardware, and / or is otherwise associated with hardware. For example, as used herein, when executing one or more first codes, a specific processor and memory may include a first "circuit", and when executing one or more second codes, a specific processor and memory may include a second "circuit". As used herein, "and / or" represents any one or more of the items in the list connected by "and / or". For example, "x and / or y" represents any element in the three-element set {(x), (y), (x, y)}. As another example, "x, y and / or z" represents any element in the seven-element set {(x), (y), (z), (x, y), (x, z), (y, z), (x, y, z)}. As used herein, the term "exemplary" means used as a non-limiting example, instance, or illustration. As used herein, the terms "for example" and "such as" lead to a list of one or more non-limiting examples, instances, or illustrations. As used herein, a circuit is “operable to” and / or “configured to” perform a function whenever the circuit includes the necessary hardware and code (if necessary) to perform the function, regardless of whether execution of the function is disabled or not enabled by some user-configurable setting.

[0080] Other embodiments may provide a computer-readable device and / or a non-transitory computer-readable medium, and / or a machine-readable device and / or a non-transitory machine-readable medium, on which are stored machine codes and / or a computer program having at least one code segment that can be executed by the machine and / or computer, thereby causing the machine and / or computer to perform the steps for automatically acquiring and rotating an ultrasound volume based on a local target structure as described herein.

[0081] Therefore, the present disclosure may be implemented in hardware, software, or a combination of hardware and software. The present disclosure may be implemented in a centralized manner in at least one computer system, or in a distributed manner, where different elements are distributed across several interconnected computer systems. Any type of computer system or other device suitable for executing the methods described herein is suitable.

[0082] The various embodiments may also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein and which, when loaded into a computer system, is capable of carrying out these methods. A computer program herein refers to any expression of a set of instructions in any language, code or notation, which is intended to cause a system with information processing capabilities to perform specific functions directly or after either or both of the following: a) conversion into another language, code or notation; b) reproduction in a different material form.

[0083] Although the present disclosure has been described with reference to certain embodiments, it will be appreciated by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the present disclosure. In addition, many modifications may be made to adapt specific circumstances or materials to the teachings of the present disclosure without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the specific embodiments disclosed, but the present disclosure will include all embodiments falling within the scope of the appended claims.

Claims

1. A method (900) for placing a medical device in an anatomical structure, the method comprising: performing (902) one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712) by an ultrasound probe (104) of an ultrasound system (100); detecting and tracking (906), by at least one processor (132, 140, 150, 170) of the ultrasound system (100), one or more anatomical structures in the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712); causing (908) a display system (134) to present, by the at least one processor (132, 140, 150), a volume (412, 512, 712) of a region of interest (314, 414, 514, 714) within or around the one or more anatomical structures in the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712); processing (910) the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712) by the at least one processor (132, 140, 170) to identify a first location corresponding to a target placement location within the region of interest (314, 414, 514, 714) of a medical device (720); processing (912), by the at least one processor, the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712) to identify a second location corresponding to a location of a catheter (610) including the medical device (720); as well as When the second position corresponding to the position of the catheter matches the first position corresponding to the target placement position (420, 520) of the medical device (720), a signal (918) is output by the at least one processor (132, 170, 180, 190) to deploy (920) the medical device (720).

2. The method (900) of claim 1, further comprising: The one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712) are rendered (914) overlaid with a first graphical marker (418, 518) corresponding to the first location and a second graphical marker (420, 520) corresponding to the second location.

3. The method (900) of claim 2, further comprising: The first graphic mark (418, 518) corresponding to the target placement position is updated (916) as the first position is tracked by the at least one processor (132, 170), and the second graphic mark (420, 520) corresponding to the catheter (620) is updated (916) as the second position is tracked by the at least one processor (132, 180).

4. The method (900) of claim 2, wherein the matching of the first position and the second position is indicated by the first graphical mark (418, 518) overlapping (730, 740) with the second graphical mark (420, 520).

5. The method (900) of claim 1, further comprising: The medical device is automatically deployed when the signal is output (720).

6. The method (900) of claim 1, transmitting a notification to an operator to deploy the medical device (720) when the signal is output.

7. The method (900) of claim 1, wherein the one or more anatomical structures is a left atrial appendage (LAA).

8. An ultrasound system (100), comprising: an ultrasound probe (104) configured to perform one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712); and At least one processor (132, 140, 170), the at least one processor configured to: detecting and tracking (906) one or more anatomical structures in the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712); causing (908) a display system (134) to present a volume (412, 512, 712) of a region of interest (314, 414, 514, 714) within or around the one or more anatomical structures in the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712); processing (910) the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712) to identify a first location corresponding to a target placement location within the region of interest (314, 414, 514, 714) of a medical device; processing (912) the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712) to identify a second location corresponding to a location of a catheter including the medical device; and When the second position corresponding to the position of the catheter (610) matches the first position corresponding to the target placement position of the medical device (720), a signal (918) is output to deploy the medical device.

9. The system (100) of claim 8, wherein the at least one processor is further configured to render (914) the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712), the one or more ultrasound image acquisitions being overlaid with a first graphic marker (418, 518) corresponding to the first location and a second graphic marker (420, 520) corresponding to the second location.

10. According to the system (100) of claim 9, the at least one processor (132, 160) is also configured to update (916) the first graphic mark (418, 518) corresponding to the target placement position as the first position is tracked by the at least one processor (132, 160), and to update (916) the second graphic mark (420, 520) corresponding to the catheter (610) as the second position is tracked by the at least one processor (132, 180).

11. The system (100) of claim 9, wherein the at least one processor (132, 160) is further configured such that a match between the first position and the second position is indicated by the first graphic mark (418, 518) overlapping the second graphic mark (420, 520).

12. The system (100) of claim 8, the at least one processor (132, 160) further configured to automatically deploy the medical device (720) when the signal is output (918).

13. The system (100) of claim 8, the at least one processor (132, 160) further configured to transmit a notification to an operator to deploy the medical device (720) when the signal is output (918).

14. The system (100) of claim 8, wherein the one or more anatomical structures is a left atrial appendage (LAA).

15. An ultrasound system (100) for placing a medical device in an anatomical structure, the ultrasound system comprising: an ultrasound probe (104) configured to perform one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712); and At least one processor (132, 140, 170), the at least one processor configured to: detecting and tracking (906) a left atrial appendage (LAA) in the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712); causing (908) a display system (134) to present a volume (412, 512, 712) of a region of interest (314, 414, 514, 714) within or around the LAA in the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712); processing (910) the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712) to identify a first location corresponding to a target placement location within the region of interest (314, 414, 514, 714) of a medical device; processing (912) the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712) to identify a second location corresponding to a location of a catheter (610) including the medical device (720); rendering (914) the one or more ultrasound image acquisitions (304, 308, 404, 408, 312, 412, 504, 508, 512, 704, 708, 712) using a first indication of the first location and a second indication of the second location; and When the second position corresponding to the position of the catheter matches the first position corresponding to the target placement position of the medical device (720), a signal is output (918) to automatically deploy the medical device.

Images