Dual-plane and three-dimensional ultrasound image acquisition for generating route mapping images, and related systems and equipment.

Abstract

An ultrasound route mapping image generation system is disclosed. The system includes a processor configured to communicate with an ultrasound imaging device movable relative to a patient. The processor receives a first biplane or 3D image representing a first volume within the patient and a second biplane or 3D image representing a second volume within the patient. The processor then registers the first biplane or 3D image with the second biplane or 3D image to determine motion between the two images. Based on the determined motion, the processor then generates a 2D route mapping image of a region of interest by combining the first biplane or 3D image with the second biplane or 3D image; and outputs a screen display including the 2D route mapping image.

Description

Technical Field

[0001] The subject matter described herein generally relates to the acquisition and display of ultrasound images. In particular, this disclosure describes systems and methods for generating panoramic ultrasound images. The described devices, systems, and methods have specific, but not exclusive, uses for diagnostic medical imaging. Background Technology

[0002] Ultrasound imaging is frequently used to obtain images of a patient's internal anatomy. An ultrasound system typically includes an ultrasound transducer probe comprising an array of transducers coupled to a probe housing. The transducer array is activated to vibrate at ultrasonic frequencies, thereby emitting ultrasonic energy into the patient's anatomy, and then receiving the ultrasonic echoes reflected or backscattered by the patient's anatomy to create an image. Such a transducer array may include various layers, including layers with piezoelectric materials that vibrate in response to an applied voltage, thereby generating desired pressure waves. These transducers can be used to sequentially emit and receive several ultrasonic pressure waves through various tissues of the body. The various ultrasonic responses can be further processed by the ultrasound imaging system to visualize various structures and tissues of the body.

[0003] Ultrasound imaging is limited by a small field of view, especially compared to other imaging modalities such as X-ray, CT, and MRI. This limited field of view leads to difficult workflows, increased surgical times, and reduced clinical outcomes. One approach to synthesizing extended ultrasound images based on a finite number of individual frames is to stitch these frames together to create a panoramic “roadmap” image that reduces the reliance on mental imagery and memory for features appearing outside the current field of view.

[0004] Creating roadmap images in ultrasound can present several challenges. For example, random speckles and noise present in ultrasound images can easily introduce uncertainty and errors into roadmap creation. Second, because roadmap images often rely on a series of ultrasound images acquired as the sonographer manually moves the ultrasound probe across the patient's anatomy, determining whether sequential ultrasound images are coplanar is challenging. At this point, the movement of the ultrasound probe may not follow a linear path, and the planes of sequential images may not be parallel to each other. If successive two-dimensional (2D) frames are not coplanar, the number of significant image features between frames is limited, resulting in poor inter-frame registration accuracy.

[0005] Therefore, while there is great interest in using image-based registration techniques for 2D ultrasound frame alignment to create extended views, the clinical and commercial translation of these techniques has been limited.

[0006] The information included in the background section of this specification (including any references cited herein and any descriptions or discussions thereof) is included for technical reference purposes only and is not intended to define the scope of this disclosure. Summary of the Invention

[0007] A novel image acquisition system and method are disclosed, comprising constructing an extended field-of-view panoramic roadmap using multiple biplane and / or 3D ultrasound images. Compared to conventional 2D image-based methods, using biplane and / or 3D ultrasound imaging significantly improves the accuracy of road mapping because, unlike 2D images, these image types contain information that can be used to determine the relative pose of each image (e.g., the position and orientation of the ultrasound scanner), thereby allowing for improved landmark registration. Therefore, multiple images can be translated and / or rotated to a common coordinate system, assembled, and smoothed to form a panoramic roadmap image that covers a larger area of ​​the patient's anatomy than a single image might cover. This system may be referred to below as an ultrasound roadmap generation system.

[0008] The ultrasound route mapping generation system disclosed herein has specific, but not exclusive, applications for diagnostic medical imaging. According to embodiments of this disclosure, an ultrasound route mapping generation system includes processor circuitry configured to communicate with an ultrasound imaging device movable relative to a patient. The processor circuitry is configured to: receive a first biplane or 3D image representing a first volume within the patient and a second biplane or 3D image representing a second volume within the patient; register the first biplane or 3D image with the second biplane or 3D image to determine a first motion between the two images; based on the determined first motion, generate a 2D route map image of a region of interest by combining the first biplane or 3D image with the second biplane or 3D image; and output a screen display including the 2D route map image to a display communicating with the processor circuitry. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0009] In some embodiments, the processor circuitry is configured to combine the first dual-plane or 3D image and the second dual-plane or 3D image by transforming the first dual-plane or 3D image relative to the second dual-plane or 3D image such that the first dual-plane or 3D image and the second dual-plane or 3D image are in the same coordinate system. The processor circuitry may be configured to: identify image landmarks in each of the first dual-plane or 3D image and the second dual-plane or 3D image; and combine the first dual-plane or 3D image and the second dual-plane or 3D image by: transforming at least one of the first dual-plane or 3D image or the second dual-plane or 3D image based on a determined first motion such that the image landmarks in the first dual-plane or 3D image are aligned with the image landmarks in the second dual-plane or 3D image. The processor circuitry may also be configured to: receive a third biplane or 3D image representing a third volume within the patient; register the third biplane or 3D image to at least one of a first biplane or 3D image or a second biplane or 3D image to determine a second motion between the third biplane or 3D image and at least one of the first biplane or 3D image or the second biplane or 3D image; and generate the 2D route map image of the region of interest by combining the first biplane or 3D image, the second biplane or 3D image, and the third biplane or 3D image based on the determined first and second motions. Two of the first, second, and third dual-plane or 3D images are 3D images, wherein one of the first, second, and third dual-plane or 3D images is a dual-plane image, and wherein the processor circuitry is configured to: register the two 3D images to each other; register the dual-plane image to the nearest 3D image among the two 3D images; extract image data from the dual-plane image; and generate the 2D roadmap image based on the extracted image data. The resolution of the 3D image may be lower than the resolution of the dual-plane image. Each of the three dual-plane or 3D images may be a 3D image. The processor circuitry may be configured to determine the first motion by detecting out-of-plane translation or rotation between adjacent dual-plane images using blob-based decorrelation. Implementations of the described techniques may include hardware, methods, or processes, or computer software on a computer-accessible medium.

[0010] According to another embodiment, a method includes: receiving, at a processor circuitry, a first biplane or 3D image of a first volume within a patient and a second biplane or 3D image of an overlapping second volume within the patient, the processor circuitry configured to communicate with an ultrasound imaging device movable relative to the patient; registering the first biplane or 3D image with the second biplane or 3D image to determine a first motion between the first biplane or 3D image and the second biplane or 3D image; using the processor circuitry, based on the determined first motion, generating a 2D route map image of a region of interest by combining the first biplane or 3D image with the second biplane or 3D image; and outputting a screen display including the 2D route map image to a display communicating with the processor circuitry. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0011] In some embodiments, generating the route map image may include: transforming the first dual-plane or 3D image relative to the second dual-plane or 3D image such that the first dual-plane or 3D image and the second dual-plane or 3D image are in the same coordinate system. Generating the route map image may include: identifying image landmarks common to the first dual-plane or 3D image and the second dual-plane or 3D image; and combining the first dual-plane or 3D image and the second dual-plane or 3D image by: transforming at least one of the first dual-plane or 3D image or the second dual-plane or 3D image based on a determined first motion such that the image landmarks in the first dual-plane or 3D image are aligned with the image landmarks in the second dual-plane or 3D image. The method may further include: receiving a third biplane or 3D image representing a third volume within the patient; registering the third biplane or 3D image to at least one of a first biplane or 3D image or a second biplane or 3D image to determine a second motion between the third biplane or 3D image and at least one of the first biplane or 3D image or the second biplane or 3D image; and generating the 2D route map image of the region of interest by combining the first biplane or 3D image, the second biplane or 3D image, and the third biplane or 3D image based on the determined first and second motions. Two of the first, second, and third dual-plane or 3D images may be 3D images, wherein one of the first, second, and third dual-plane or 3D images is a dual-plane image, and wherein the method further includes: registering the two 3D images to each other; registering the dual-plane image to the nearest 3D image among the two 3D images; extracting image data from the dual-plane image; and generating the 2D roadmap image based on the extracted image data. The resolution of the dual-plane image may be higher than the resolution of the 3D image. Each of the three dual-plane or 3D images may be a 3D image. Combining the three dual-plane images may involve using blob-based decorrelation to detect out-of-plane translations or rotations between adjacent dual-plane images. Implementations of the described techniques may include hardware, methods, or processes, or computer software on a computer-accessible medium.

[0012] The purpose of this invention summary is to present, in a simplified form, some alternative concepts that will be further described in the detailed description below. This invention summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. A broader presentation of the features, details, uses, and advantages of the ultrasonic route mapping system is provided in the following written description of various embodiments of this disclosure, and these presentations are illustrated in the accompanying drawings. Attached Figure Description

[0013] Illustrative embodiments of this disclosure will be described with reference to the accompanying drawings, in which:

[0014] Figure 1 This is a block diagram of an example ultrasound system based on various aspects of this disclosure.

[0015] Figure 2A Biplane ultrasound images of perivascular tissue are shown according to various aspects of this disclosure.

[0016] Figure 2B The various aspects of this disclosure are shown. Figure 2A Cross-section of a biplane image.

[0017] Figure 3A 3D ultrasound images, including blood vessels, are shown according to various aspects of this disclosure.

[0018] Figure 3B The following are shown from various aspects of this disclosure: Figure 3A A 2D longitudinal view extracted from a 3D image.

[0019] Figure 4A A first synthetic longitudinal view of a first perivascular tissue according to various aspects of this disclosure is shown.

[0020] Figure 4B A second synthetic longitudinal view of the second perivascular tissue according to various aspects of this disclosure is shown.

[0021] Figure 5 This is a schematic diagram of an ultrasound probe that acquires multiple individual image frames during continuous scanning of a patient's anatomy, in accordance with various aspects of this disclosure.

[0022] Figure 6 This is a schematic diagram of an ultrasound probe, according to various aspects of this disclosure, acquiring multiple individual image frames as the probe scans the patient's anatomy in an incremental or stepwise manner.

[0023] Figure 7This is a schematic representation of probe movements occurring during a dual-plane image registration process, based on various aspects of this disclosure.

[0024] Figure 8 Various hybrid imaging workflows that can be used to improve the accuracy and resolution of roadmap images are described in accordance with various aspects of this disclosure.

[0025] Figure 9 A flowchart illustrating an example ultrasound route map generation method according to various aspects of this disclosure is shown.

[0026] Figure 10 This is a simplified diagram of the processor circuitry according to various aspects of this disclosure.

[0027] Figure 11A This is a schematic diagram of a narrowed blood vessel being imaged using longitudinal scanning according to at least one embodiment of the present disclosure.

[0028] Figure 11B At least one embodiment of the present disclosure is shown. Figure 11A The flattened longitudinal image is derived from the curved longitudinal image.

[0029] Figure 12 This is a schematic representation of an example two-pass scanning imaging technique for roadmap generation according to at least one embodiment of the present disclosure. Detailed Implementation

[0030] As mentioned above, the limited field of view of ultrasound images leads to difficult workflows, increased surgical times, and reduced clinical outcomes. This disclosure describes novel apparatus, systems, and methods for generating roadmap images by utilizing information provided from multiplanar images (e.g., biplanar or 3D imaging). In some aspects, this disclosure describes methods that combine biplanar and 3D imaging to create roadmaps. In this regard, external tracking systems or sensors are often unavailable in ultrasound imaging systems, and roadmaps are generated using image processing of ultrasound images. However, the roadmap imaging workflow can be improved if pose information (the position and orientation of the ultrasound transducer at the time of image capture) is available for individual images from which roadmap images are constructed.

[0031] According to at least one embodiment of this disclosure, an ultrasound roadmap generation system is provided that combines multiple biplane and / or 3D ultrasound images into a single roadmap image, the single roadmap image representing a wider or larger field of view than any of the individual ultrasound images used to create the roadmap. Using biplane and / or 3D ultrasound imaging can significantly improve the accuracy of roadmap stitching compared to methods using only single-plane 2D images, because biplane and 3D ultrasound images contain multiplane information that can be used to determine the relative pose of each image (e.g., the position and orientation of the ultrasound scanner), thus allowing improved landmark registration from one image to the next, even if pose changes (whether accidental or intentional) occur when consecutively generated images are captured. Therefore, multiple images can be translated and / or rotated to a common coordinate system, combined, and smoothed to form a panoramic roadmap image that accurately represents the patient's anatomy and covers a larger area of ​​the patient's anatomy than a single image might cover. This system will be referred to below as the ultrasonic route map generation system.

[0032] The various aspects of this disclosure include: (1) a workflow involving using continuous or incremental probe movement to sweep an ultrasound probe across an extended area to create an extended field-of-view 3D image; (2) a hardware configuration in which a matrix probe is used for scanning; and (3) an image acquisition mode utilizing biplane imaging, 3D imaging, or a combination of biplane and 3D imaging. The ultrasound route mapping system of this disclosure has the potential to improve the accuracy of ultrasound route mapping techniques. This ultrasound route mapping system can be applied to any type of ultrasound imaging, including external ultrasound, intracatheter or intravascular ultrasound, and transesophageal echo, to improve the accuracy of route mapping.

[0033] This disclosure substantially aids in the acquisition and interpretation of diagnostic ultrasound images by improving the speed, accuracy, and confidence of assembling panoramic roadmap images. When implemented on a computing system communicating with an ultrasound imaging probe, the ultrasound roadmap generation system disclosed herein provides a real improvement in the quality and accuracy of ultrasound roadmap images and reduces the user workload involved in constructing and employing roadmap images. This improved workflow transforms fragmented processes into a fully automated road mapping system that reveals anatomical features outside the current field of view of the ultrasound scanner or interprets image artifacts and other inaccuracies in the current 2D roadmap without relying solely on memory and mental visualization. Furthermore, the 3D ultrasound roadmap generated by this system can be integrated into clinical reports and presented to clinicians for disease diagnosis and treatment planning. This unconventional approach improves the functional operation of ultrasound imaging systems by automatically providing accurate roadmap images during routine imaging processes.

[0034] This ultrasound route mapping system can be implemented as an ultrasound image combiner that can be viewed on a display and is operated by a control process running on a processor that accepts user input from a keyboard, mouse, or touchscreen interface and communicates with one or more ultrasound imaging probes or imaging arrays. At this point, the control process performs specific operations in response to different inputs, selections, or probe movements at different times.

[0035] To facilitate an understanding of the principles of this disclosure, reference will now be made to embodiments illustrated in the accompanying drawings, and these embodiments will be described using specific language. Nevertheless, it should be understood that this disclosure is not intended to limit its scope. Any changes and further modifications to the described devices, systems, and methods, as well as any further applications of the principles of this disclosure, are fully contemplated and included within this disclosure, as are commonly conceived by those skilled in the art to which this disclosure pertains. In particular, it is fully contemplated that features, components, and / or steps described with respect to one embodiment may be combined with features, components, and / or steps described with respect to other embodiments of this disclosure. However, for the sake of brevity, numerous iterations of these combinations will not be described separately.

[0036] Figure 1 A block diagram of an example ultrasound system 100 according to various aspects of this disclosure is shown. The ultrasound probe 10 has a transducer array 12 comprising a plurality of ultrasound transducer elements or acoustic elements. In some instances, the array 12 may include any number of acoustic elements. For example, the array 12 is capable of including from 1 acoustic element to 100,000 acoustic elements, including, for example, 2 acoustic elements, 4 acoustic elements, 36 acoustic elements, 64 acoustic elements, 128 acoustic elements, 300 acoustic elements, 812 acoustic elements, 3,000 acoustic elements, 9,000 acoustic elements, 30,000 acoustic elements, 65,000 acoustic elements, and / or more or fewer other numbers of acoustic elements. In some instances, the acoustic elements of array 12 can be arranged in any suitable configuration, such as a linear array, planar array, curved array, zigzag array, circular array, ring array, phased array, matrix array, one-dimensional (1D) array, 1.X-dimensional array (e.g., 1.5D array), or two-dimensional (2D) array. The array of acoustic elements (e.g., one or more rows, one or more columns, and / or one or more orientations) can be controlled and activated uniformly or independently. Array 12 can be configured to acquire one-dimensional, two-dimensional, biplane, and / or three-dimensional images of the patient's anatomy.

[0037] While this disclosure relates to the use of external ultrasound probes for synthetic aperture external ultrasound imaging, it should be understood that one or more aspects of this disclosure can also be implemented in any suitable ultrasound imaging probe or system, including external ultrasound probes and intraluminal ultrasound probes. For example, various aspects of this disclosure can be implemented in ultrasound imaging systems using mechanically scanned external ultrasound imaging probes, intracardiac (ICE) echocardiography catheters and / or transesophageal echocardiography (TEE) probes, rotating intravascular ultrasound (IVUS) imaging catheters, phased array IVUS imaging catheters, transthoracic echocardiography (TTE) imaging devices, or any other suitable type of ultrasound imaging device.

[0038] Refer again Figure 1 The acoustic elements of array 12 may include one or more piezoelectric / piezoresistive elements, lead zirconate titanate (PZT), piezoelectric micromechanical ultrasonic transducer (PMUT) elements, capacitive micromechanical ultrasonic transducer (CMUT) elements, and / or any other suitable type of acoustic element. One or more acoustic elements of array 12 communicate with (e.g., are electrically coupled to) electronic circuitry 14. In some embodiments (e.g., Figure 1 In one embodiment, electronic circuitry 14 may include a microwave beamformer (μBF). In other embodiments, electronic circuitry includes a multiplexer circuit (MUX). Electronic circuitry 14 is located in probe 10 and is communicatively coupled to transducer array 12. In some embodiments, one or more components of electronic circuitry 14 may be located in probe 10. In some embodiments, one or more components of electronic circuitry 14 may be located in computing device or processing system 28. Computing device 28 may be a processor or include processors, such as one or more processors communicating with memory. As further described below, computing device 28 may include, for example,... Figure 10 The processor circuitry is shown. In some aspects, some components of electronic circuitry 14 are located in probe 10, and other components of electronic circuitry 14 are located in computing device 28. Electronic circuitry 14 may include one or more electrical switches, transistors, programmable logic devices, or other electronic components configured to combine and / or continuously switch among multiple inputs to transmit signals from each of the multiple inputs on one or more common communication channels. Electronic circuitry 14 may be coupled to elements of array 12 via multiple communication channels. Electronic circuitry 14 is coupled to cable 16, which transmits signals including ultrasound imaging data to computing device 28.

[0039] In computing device 28, signals are digitized and coupled to channels of system beamformer 22, which appropriately delays each signal. The delayed signals are then combined to form a coherent, steered, and focused receiving beam. The system beamformer may include electronic hardware components, software-controlled hardware, or a microprocessor running a beamforming algorithm. In this respect, beamformer 22 may be referred to as electronic circuitry. In some embodiments, beamformer 22 can be a system beamformer, for example, Figure 1 The system beamformer 22, or beamformer 22, may be a beamformer implemented by circuitry within the ultrasonic probe 10. In some embodiments, the system beamformer 22 works in conjunction with a microwave beamformer (e.g., electronic circuitry 14) disposed within the probe 10. In some embodiments, beamformer 22 may be an analog beamformer, or in some embodiments, beamformer 22 may be a digital beamformer. In the case of a digital beamformer, the system includes an A / D converter that converts the analog signal from array 12 into sampled digital echo data. Beamformer 22 will typically include one or more microprocessors, shift registers, and / or digital or analog memories to process the echo data into coherent echo signal data. Delay is affected by various means, such as the sampling time of the received signal, the write / read interval of data temporarily stored in memory, or the length or clock rate of the shift register, as described in U.S. Patent 4,173,007 to McKeighen et al., which is incorporated herein by reference in its entirety. Additionally, in some embodiments, the beamformer is capable of applying appropriate weights to each signal in the signals generated by array 12. Signal and image processor 24 processes the beamforming signals from the image field to produce a 2D or 3D image for display on image display 30. Signal and image processor 24 may include electronic hardware components, software-controlled hardware, or a microprocessor running image processing algorithms. Signal and image processor 24 typically also includes dedicated hardware or software, such as a scan converter, for processing received echo data into image data for a desired display format. In some embodiments, beamforming functions can be partitioned among different beamforming components. For example, in some embodiments, system 100 may include a microwave beamformer located within probe 10 and communicating with system beamformer 22. This microwave beamformer can perform preliminary beamforming and / or signal processing, which can reduce the number of communication channels used to send received signals to computing device 28.

[0040] Under the control of the system controller 26, which is coupled to various modules of system 100, control is performed on ultrasound system parameters (e.g., scanning mode (e.g., B-mode, M-mode), probe selection, beam steering and focusing, and signal and image processing). The system controller 26 may consist of application-specific integrated circuits (ASICs) or microprocessor circuitry and software data storage devices (e.g., RAM, ROM, or disk drives). In the case of probe 10, some of this control information can be provided from computing device 28 to electronic circuitry 14 via cable 16, thereby regulating electronic circuitry 14 for array operation according to a specific scanning procedure. The user inputs these operating parameters via user interface device 20.

[0041] In some embodiments, image processor 24 is configured to generate images in different modes for further analysis or output to display 30. For example, in some embodiments, the image processor can be configured to compile B-mode images of the patient's anatomy, such as live B-mode images. In other embodiments, image processor 24 is configured to generate or compile M-mode images. M-mode images can be described as images showing the temporal changes of the imaged anatomy along a single scan line.

[0042] It should be understood that computing device 28 may include hardware circuitry, such as a computer processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), capacitors, resistors and / or other electronic devices, software, or a combination of hardware and software. In some embodiments, computing device 28 is a single computing device. In other embodiments, computing device 28 includes separate computer devices communicating with each other.

[0043] The computing device 28 may also include a route map generation system 25 for generating route map images based on multiple dual-plane and / or 3D ultrasound images. The route map generation system 25 may be configured to receive various inputs from the system, including inputs from the interface device 20, ultrasound imaging data from the ultrasound probe 10, the system beamformer 22, and / or the signal and image processor 24.

[0044] Figure 2A A biplane ultrasound image 200 of tissue surrounding a blood vessel 230 according to various aspects of this disclosure is shown. The image includes a longitudinal plane 210 and a cross-sectional plane 220. Depending on the implementation, the biplane image may be captured by a biplane ultrasound imaging sensor, by a 2D array ultrasound sensor in biplane imaging mode, or by a linear (1D) sensor that is manually rotated to capture two separate images in two different imaging planes (e.g., orthogonal or approximately orthogonal to each other).

[0045] By combining multiple biplane images 200, individual longitudinal images or 2D cross-sections of a 3D model constructed based on individual biplane images can be stitched together to form a synthetic longitudinal image 210.

[0046] Figure 2B It shows Figure 2A A cross-section 220 of a dual-plane image. Blood vessels 230 can be seen in this image. Because the blood vessels 230 appear in two orthogonal planes, it is possible to use the two planes to detect different types of movement. For example, longitudinal displacement can be considered as a change in the cross-sectional image, while lateral movement can be considered as a change in the longitudinal image, and diagonal movement can be considered as a change in both images. In some embodiments, the planes of the dual-plane image are orthogonal to each other. In other embodiments, the two planes may intersect at an acute angle or an oblique angle.

[0047] Figure 3A A 3D ultrasound image 300 including blood vessels 230 is shown. The volumetric image can be captured by a 3D ultrasound imaging system with an ultrasound sensor matrix, or by a device that electrically or mechanically manipulates a 2D sensor array to create a 3D image. By combining multiple 3D images, the ultrasound route mapping generation system is able to stitch individual 3D frames together to form a synthetic 3D route map.

[0048] Figure 3B It shows from Figure 3A The 2D longitudinal view 310 is extracted from the 3D route map 300 (e.g., as a longitudinal cross-section of the 3D image). When multiple 3D images are stitched together to create a synthetic 3D route map 300, the ultrasound route map generation system can extract the synthetic longitudinal view 310, for example, by acquiring a longitudinal cross-section of the synthetic 3D image 300. Multiple 3D route maps 300 can be registered with each other and combined to generate a larger synthetic 3D route map 300. The synthetic 2D longitudinal view 310 can then be extracted from the larger synthetic 3D route map 300.

[0049] Figure 4A and Figure 4BTwo distinct composite longitudinal views 400 are shown of the tissue surrounding two different blood vessels 230, 231. Both composite longitudinal views 400 and 401 are formed by stitching together multiple individual image frames 410, 411 (whether biplane or 3D image frames) and are suitable for use as roadmap images by clinicians or other users. At this point, each composite longitudinal view 400 includes annotations 420, 421 added by clinicians or other users to highlight certain dimensions or other features of the image. Such roadmap images, with or without annotations, can be used by clinicians and other users in clinical decision-making without having to review multiple image frames 410 or 411 of different regions of the tissue or mentally visualize features outside the field of view of individual image frames 410 or 411.

[0050] The clinical value of panoramic ultrasound road mapping is immense. For example, the expanded field of view allows clinicians to make more accurate measurements, such as determining the size of long vessels and devices. This disclosure advantageously enables the use of ultrasound to create anatomical maps with an expanded field of view. In some embodiments, the roadmaps can be comparable to current modalities such as X-ray angiography, CT, and MRI in terms of resolution, accuracy, and readability / interpretability. The expanded field of view enables many important secondary uses, including but not limited to the ability to make more accurate measurements, such as determining the size of long vessels and devices in peripheral vascular disease (PVD), tracking IVUS location in vivo, measuring TEE cannulation distance, etc. Typically, ultrasound only provides a small window of anatomical information for performing multimodal fusion and registration. Improving the accuracy of image-based fusion, making it comparable to other imaging modalities (e.g., X-ray / CT / MRI), allows for the creation of accurate roadmaps with added anatomical background (e.g., Figure 4A and Figure 4B (As shown), this can improve registration fidelity. This also makes it possible to create ultrasound-based anatomical atlases to, for example, examine anatomical variability between patients.

[0051] Figure 5The illustration shows an ultrasound probe 10 collecting multiple individual image frames 410 while the probe 10 continuously scans the patient's anatomy. In this modality, ultrasound frames 410 are acquired sequentially (e.g., biplane image frames or 3D image frames), and registration is performed to calculate relative pose transformations between frames. These transformations are converted from image space to physical space to obtain the relative probe motion between each pair of consecutively generated frames. Finally, the transformations are "stacked" (synthesized) to obtain the total probe motion, from which a complete 3D roadmap can be reconstructed. Note that transformations that associate one image with another can include not only rotation or translation, but also other variations, including but not limited to magnification, distortion, or changes in brightness or color palette. Note that the accuracy of the roadmap depends on the specific software algorithm used for inter-frame registration. One exemplary method could be to apply intensity-based or feature-based rigid registration algorithms to find the optimal inter-frame alignment.

[0052] It can perform image acquisition according to several different workflows. Figure 5 Continuous acquisition based on dual-plane imaging or continuous acquisition based on 3D imaging is described. However, it should be understood that images can also be acquired through incremental (stepping) acquisition workflows based on 3D imaging or by combining hybrid image acquisition modes of dual-plane and 3D, staggered dual-plane and 3D acquisition, simultaneous dual-plane and 3D acquisition, and multiple dual-plane acquisition.

[0053] In some embodiments, position sensors and / or orientation sensors are present in the ultrasound probe 10, and each image has an associated transformation matrix that encodes the relative or absolute position and / or orientation of the ultrasound probe 10 at the time of image capture. The initial coarse geometric stitching step involves simply constructing a 3D model containing each component image 410 at a location matching its known 3D position and / or orientation. Each new individual image 410 added to the 3D model can simply cover the portion of the position that overlaps with the previously added image 410. A route map image 400 (e.g., a 2D longitudinal route map image as shown in FIG. 4) can then be extracted from the 3D model (e.g., by acquiring a longitudinal cross-section of the 3D model).

[0054] In some embodiments, image recognition and image stitching techniques are used instead of, or in addition to, the geometric stitching step to perform a more refined image stitching step. Such algorithms may rely on the identification and matching of anatomical landmarks (e.g., branches or intersections in blood vessels 230) from one image to the next. These landmark locations can be used to register and align multiple images to a single image or coordinate system. Algorithms for stitching multiple images together include random sample consensus (RANSAC) methods and may include steps such as keypoint or landmark detection, keypoint or landmark registration, image calibration, image alignment, synthesis, motion compensation, ghosting removal, color blending, and suture removal. Such algorithms can operate near real-time and may be able to operate in real-time, but may be limited by hardware. In some embodiments, geometric stitching is used exclusively without an additional image stitching step. In other embodiments, image stitching is used exclusively without a prior geometric stitching step. In other embodiments, both geometric stitching and image stitching may be used. However, in each of these cases, the position of each pixel in the route map image 400 is known and can be represented, for example, in a coordinate system centered on the patient or the workbench.

[0055] In some embodiments, instead of stitching together grayscale or color pixels of images acquired by an ultrasound system, data extracted from the images is stitched together to present a 3D image representation of the roadmap. Algorithms for extracting data from the images include object segmentation, image compression, image-to-point cloud conversion, object detection, and object tracking. In some embodiments, vascular contours can be segmented from ultrasound images and stitched together to create a 3D vascular mesh representation from the roadmap generation system.

[0056] Alternatively, data extraction can be applied to the reconstructed 3D roadmap itself. As before, algorithms for extracting data from 3D roadmaps include object segmentation, image compression, image-to-point cloud conversion, object detection, and object tracking. For example, blood vessel outlines can be segmented from a 3D roadmap and displayed to the user.

[0057] In the example, a smaller number of frames 410 are selected from multiple biplane or 3D frames 410 captured by the ultrasound imaging system 100 for stitching. The number of frames selected can be two, three, four, five, or more. In the example, the selected frames 410 have sufficient overlap so that the landmark recognition algorithm can match any given landmark between at least two images. If there is sufficient overlap to register landmarks between images, but the overlap is minimized, this reduces the number of images to be stitched to cover the full length of the anatomical region of interest, thus reducing the time, memory, and computational cost required to perform the algorithm. Minimizing overlap also increases the amount of additional information added to the 3D model per frame, thus increasing the amount of additional information added to the roadmap image 400 extracted from the 3D model per frame. In the example, the roadmap image is extracted as image data located along a single 2D cross-sectional plane (e.g., a longitudinal plane) within the 3D model.

[0058] exist Figure 5 In the example shown, dual-plane or 3D images are acquired sequentially as the user scans the transducer across the anatomical region of interest. An advantage of sequential acquisition workflows based on dual-plane imaging is the ability to acquire dual-plane images at high frame rates (20–40 Hz) and high image resolution relative to 3D imaging using matrix transducers. As the transducer moves, both longitudinal and minor axis image data can be used to calculate inter-frame registration, which in turn allows for the reverse calculation of probe movement. For example, probe movement along the primary direction of motion can be estimated first by determining motion in the major axis image (i.e., “in-plane” motion with respect to the longitudinal image). “Out-of-plane” motion can be determined in a second step based on the minor axis image data. Alternatively, motion can be calculated in reverse order, where “out-of-plane” motion from the minor axis is calculated first, and “in-plane” motion from the major axis is determined last. Some embodiments combine image-based methods with tracking sensors (e.g., inertial measurement sensors) that can provide the amplitude and direction of these motions with or without additional computation.

[0059] For embodiments employing a continuous acquisition workflow based on 3D imaging, panoramic route maps can be created from 3D volume rather than biplane images. One advantage of this approach is that out-of-plane translation and rotation are no longer problematic because the images are volumetric and inherently contain spatial information that can be used to infer differences in position or orientation from one image to the next. Therefore, a full 6-DOF transformation from one 3D frame to the next can be computed with little or no ambiguity. However, compared to biplane methods, frame rates and / or image resolutions may be lower when using this approach.

[0060] In some embodiments, the parameters of 3D acquisition are optimized to minimize the aforementioned drawbacks. For example, some embodiments may not use the full 3D volume that can be acquired by the imaging array for registration. Instead, the algorithm can acquire pseudo-3D “thick slices” or “bread slices,” for example, 3D depth and volume information that is just sufficient to allow full 6DOF registration while limiting the amount of data used for each image and thus maintaining 2D images with frame rates and resolutions similar to dual-plane imaging.

[0061] Figure 6 The diagram illustrates how an ultrasound probe 10 collects multiple individual image frames 410 while scanning the patient's anatomy in an incremental or stepwise manner. Utilizing this incremental (stepwise) acquisition workflow based on 3D imaging instead of sequentially acquiring 3D volumes, the system or method stitches volumes together in an incremental or stepwise manner. Here, the user's workflow is slightly altered; instead of scanning the anatomy sequentially, the user places the probe at a starting position, acquires a 3D image, moves the probe a distance to another acquisition location (e.g., approximately half the distance of the 3D imaging field), acquires a second 3D image, and repeats this process until the user reaches the endpoint of the anatomical region of interest. One advantage of this approach is the ability to improve image resolution. However, the workflow may be slightly more challenging for clinicians or other users who must ensure sufficient overlap between consecutive 3D acquisitions to allow for accurate registration.

[0062] Figure 7 This is a schematic representation of challenging probe movements for biplane-based image registration. As the ultrasound probe 10 continuously or incrementally scans the patient's anatomy while acquiring biplane images 200, certain probe movements pose additional challenges to the registration process. The desired translational movement of the ultrasound probe 10 during imaging sweeps is typically along the direction of the long axis (longitudinal) image plane. For out-of-plane translation 710, the probe 10 moves in a direction diagonally to both the long axis (longitudinal) and short axis (cross-sectional) image planes (e.g., any direction not parallel to either plane). For out-of-plane rotational movement, the orientation angle or clock angle about the vertical axis of the probe 10 rotates (e.g., unintentionally) to a different orientation with respect to the patient's body, such that the long axis and short axis image planes of a given biplane image are not parallel to the long axis and short axis image planes of previously captured biplane images. Some degree of out-of-plane translation 710 and out-of-plane rotation 720 typically occurs when a clinician or other user manually performs probe sweeps.

[0063] When significant out-of-plane motion occurs on both axes, out-of-plane translation 710 becomes problematic, causing image alignment to be distorted in both image planes. Some embodiments of this disclosure use speckle-based decorrelation to estimate out-of-plane motion in each plane, which is a challenging open research problem for 2D ultrasound imaging, but significantly easier for biplane images.

[0064] Another challenging scenario is rotation 720° about the probe axis, where there is significant “out-of-plane” motion on both axes. Some embodiments of this disclosure use blob-based decorrelation to estimate out-of-plane motion in each plane. Other embodiments use small image sub-regions instead of the entire image to estimate out-of-plane rotation, thereby reducing the computational cost involved. Still other embodiments automatically learn out-of-plane decorrelation for different tissues from training data using deep learning algorithms or other learning artificial intelligence. For example, image stitching is discussed in U.S. Application US 62 / 931693, filed November 6, 2019, entitled “CO-REGISTRATION OF INTRAVASCULAR DATA AND MULTI-SEGMENT VASCULATURE, AND ASSOCIATED DEVICES, SYSTEMS, AND METHODS,” which is incorporated herein by reference in its entirety.

[0065] Other embodiments involve estimating in-plane and out-of-plane motion separately while applying known constraints. Specifically, since the two image planes are perpendicular to each other, any out-of-plane motion on the major axis should be the same as the in-plane motion on the minor axis, and vice versa. Applying this constraint to motion estimation can be used to reduce errors in out-of-plane motion estimation. Some embodiments not only detect out-of-plane translation or rotation but also attempt to correct for them. Based on inter-frame decorrelation after in-plane motion correction, or based on the displacement ratio of the major axis to the minor axis of the dual-plane images, the system can adaptively manipulate the rotational orientation of the imaging plane relative to the probe in real time, such that the longitudinal plane is more closely aligned with the orientation of the probe translation, rather than with the orientation of the probe itself.

[0066] Figure 8 Various hybrid imaging workflows (AEs) are described, which can be used to improve the accuracy and resolution of roadmap images. By using 3D images for coarse 6DOF registration and biplane images for fine in-plane correction and actual imaging details, biplane and 3D imaging can be combined in novel ways to maximize registration accuracy. These modes can be combined by performing biplane and 3D imaging alternately or by simultaneously acquiring images in both modes.

[0067] Workflow A illustrates interleaved or alternating dual-plane image acquisition and 3D image acquisition. Here, the transducer alternates between dual-plane and 3D imaging according to a set schedule, either manually or during probe translation. The alternation mode between dual-plane and 3D imaging can be adjusted, and it represents a trade-off between frame rate, image resolution, processing power requirements, and route accuracy.

[0068] For example, a mode such as simply alternating between 3D imaging and biplane imaging will result in high roadmap accuracy because 3D volumes are acquired frequently. However, the imaging frame rate is reduced because 3D volumes take up more time and have not yet been used to generate actual anatomical images within the roadmap. Each biplane is registered to the latest 3D volume, and each 3D volume is registered to the 3D volume immediately preceding it.

[0069] One repeating pattern involves a 3D image followed by three biplane images. This allows for faster updates to image information, at the cost of a reduced frequency of obtaining 3D information for roadmap construction. It may be desirable that the image resolution of the 3D volume not exceed a resolution sufficient for accurate road mapping, as increased spatial resolution could lead to decreased temporal resolution.

[0070] Workflow B illustrates how biplane and 3D images alternate according to an adaptive schedule rather than a fixed schedule. Instead of setting a fixed pattern, the interleaving is altered so that 3D volume is acquired only when the estimated probe has moved approximately 1 / 3 or 1 / 2 of its volume. This minimizes the number of volumes acquired for accurate registration. Roadmap construction is based on biplane registration, using intermittent 3D acquisition / registration to prevent loss of accuracy.

[0071] Workflow C illustrates another embodiment where motion for each frame is calculated in the registration from biplane to 3D. Each time a 3D volume is acquired, all subsequent biplane acquisitions are downsampled to a resolution similar to the 3D volume, and then that biplane acquisition is registered to that volume. This allows for full 6DOF registration between frames without waiting for the next 3D volume. When a new 3D volume is acquired, additional 3D-3D registration steps can be performed to update the roadmap. Therefore, the roadmap is built based on a series of biplane-to-3D registrations. This is similar to the embodiment shown in Workflow A, except that multiple biplanes are acquired for each 3D volume to speed up the frame rate.

[0072] Workflow D describes the simultaneous acquisition of dual-plane and 3D images. Instead of interleaved dual-plane and 3D acquisition, a special imaging modality is introduced to acquire both types of images simultaneously. In one example, sparse (low-resolution) 3D volumetric and dual-plane images are acquired simultaneously. The sparse 3D is used for inter-frame registration, while the dual-plane is used for image reconstruction.

[0073] Workflow E describes multi-plane acquisition. In this embodiment, instead of combining the dual-plane images with the 3D image, multiple dual planes are fired simultaneously or rapidly in succession (i.e., significantly faster than probe movement) to provide volumetric information for registration and image formation.

[0074] For each of the embodiments described above, multiple scans can be performed on the region, and the images obtained from each scan can be combined (e.g., via weighted averaging or joint optimization) to produce a route map with higher accuracy than that that could be obtained with only one scan.

[0075] Similarly, a scan can be performed using one acquisition method (or a combination of methods), followed by subsequent scans using different methods (or combinations of methods). In other examples, besides interleaving, simultaneous acquisition, or other combinations of biplane and 3D acquisition, 2D and 3D acquisition can be combined in similar or identical ways. For example, in Figure 8 In this approach, 2D acquisition can be used instead of biplane acquisition for each instance to achieve a similar effect in roadmap creation. The advantage of using 2D acquisition is that it can increase frame rate or line density (image resolution) compared to biplane acquisition. The disadvantage is that 2D frame-to-3D volume registration may not be as accurate as biplane-to-3D registration. Other workflows are anticipated, including but not limited to combinations of the above.

[0076] Figure 9 A flowchart of an example ultrasound route map generation method 900 according to at least one embodiment of the present disclosure is shown. It should be understood that the steps of method 900 may differ from... Figure 9 The sequential execution shown can also include additional steps before, during, and after these steps, and / or, in other embodiments, the ability to replace or eliminate some of the described steps. Steps can be subdivided into sub-steps, which can also be executed in different orders. One or more steps in method 900 can be performed by one or more devices and / or systems described herein (e.g., Figure 1 Imaging system 100 and / or Figure 10 The processor circuitry of the 1050 is used to execute the commands.

[0077] In step 910, method 900 includes capturing two or more biplane and / or 3D images, such as... Figure 5-8 As described in the text, the captured image dataset can include various combinations of biplane and 3D images. These combinations are selected within the speed, memory, and bandwidth limitations of the processor performing the method to provide sufficient 3D registration information and adequate detail and resolution for the resulting roadmap images. As the imaging probe moves between capturing one image and capturing the next, each image may represent a slightly different tissue volume within the patient.

[0078] In step 920, the method identifies common landmarks or key points between images. Such anatomical landmarks may include, for example, branches or intersections in blood vessels, as their positions may change from one image to the next. Alternatively, common landmarks may be the raw intensity of the image pixels themselves. A combination of landmarks, key points, and raw image intensity may also be used.

[0079] In step 930, the method uses common landmarks to determine the relative pose of the images. This can be accomplished, for example, by 3D localization of landmarks in two different images and calculating the translations and rotations involved in producing the localization changes observed from one image to the next. This can be done with a single landmark, but the 3D accuracy is greatly improved if two or more landmarks are used.

[0080] In step 940, the method, for example, registers the images by rotating and / or translating them into a common coordinate system, thereby creating a 3D model of the anatomical region of interest, which combines image data from two or more biplane and / or 3D images. This can be accomplished, for example, using the RANSAC method as described above. If the original image intensity is used as a common landmark, image-based registration techniques known in the art can be used.

[0081] In step 950, the method constructs a roadmap image, for example by obtaining a suitable longitudinal cross-section of a 3D model that illustrates the desired anatomical features of the region of interest. This can then be displayed (e.g., on a screen). Figure 1 (on the display 30) or stored (e.g., stored in the storage) Figure 10 (In memory 1064) This roadmap.

[0082] In step 960, the method accepts user input via a user interface, which allows text or graphic annotations to be placed on the roadmap image, thereby assisting in reporting, clinical decision-making, and record keeping. For example... Figure 4A and Figure 4B Examples of such annotations can be seen in the text.

[0083] Figure 10 This is a schematic diagram of a processor circuit 1050 according to an embodiment of the present disclosure. The processor circuit 1050 can be implemented in... Figure 1 The method may be implemented in an ultrasound imaging system 100 or other device or workstation (e.g., a third-party workstation, network router, etc.), or on a cloud processor or other remote processing unit, to implement the method when necessary. As shown, the processor circuitry 1050 may include a processor 1060, a memory 1064, and a communication module 1068. These components may communicate directly or indirectly with each other (e.g., via one or more buses).

[0084] Processor 1060 may include a central processing unit (CPU), a digital signal processor (DSP), an ASIC, a controller, or any combination of general-purpose computing devices, reduced instruction set computing (RISC) devices, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other associated logic devices (including mechanical computers and quantum computers). Processor 1060 may also include another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 1060 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0085] Memory 1064 may include cache memory (e.g., cache memory of processor 1060), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state storage device, hard disk drive, other forms of volatile and non-volatile memory, or combinations of different types of memory. In embodiments, memory 1064 includes a non-transient computer-readable medium. Memory 1064 may store instructions 1066. Instructions 1066 may include instructions that, when executed by processor 1060, cause processor 1060 to perform the operations described herein. Instructions 1066 may also be referred to as code. The terms “instruction” and “code” should be interpreted broadly to include any type of computer-readable statement(s). For example, the terms “instruction” and “code” may refer to one or more programs, routines, subroutines, functions, procedures, etc. “Instruction” and “code” may include a single computer-readable statement or multiple computer-readable statements.

[0086] The communication module 1068 can include any electronic and / or logic circuitry to facilitate direct or indirect data communication between the processor circuitry 1050 and other processors or devices. In this respect, the communication module 1068 can be an input / output (I / O) device. In some instances, the communication module 1068 facilitates direct or indirect data communication between the processor circuitry 1050 and / or other processors or devices. Figure 1 The ultrasound imaging system 100 allows for direct or indirect communication between various components. The communication module 968 can communicate within the processor circuitry 950 using various methods or protocols. Serial communication protocols may include, but are not limited to, US SPI, I... 2 C. Serial communication may be transmitted via RS-232, RS-485, CAN, Ethernet, ARINC429, MODBUS, MIL-STD-1553, or any other suitable method or protocol. Parallel protocols include, but are not limited to, ISA, ATA, SCSI, PCI, IEEE-488, IEEE-1284, and other suitable protocols. Where appropriate, serial and parallel communication may be bridged via UART, USART, or other suitable subsystems.

[0087] External communication (including, but not limited to, software updates, firmware updates, or readouts from ultrasound equipment) can be achieved using any suitable wireless or wired communication technology. These technologies include cable interfaces such as USB, micro USB, Lightning, or FireWire, Bluetooth, Wi-Fi, ZigBee, Li-Fi, or cellular data connections such as 2G / GSM, 3G / UMTS, 4G / LTE / WiMax, or 5G. For example, Bluetooth Low Energy (BLE) radios can be used to establish connections to cloud services for sending data and receiving software patches. The controller can be configured to communicate with remote servers or local devices (e.g., laptops, tablets, or handheld devices) or may include a display capable of showing status variables and other information. Information can also be transferred on physical media such as USB flash drives or memory sticks.

[0088] Figure 11A This is a perspective view of a blood vessel 230 including a stenosis 1110, imaged using longitudinal scanning 200 according to at least one embodiment of this disclosure. Whether data is extracted from images captured from an individual or from a roadmap, this data can be further used to provide additional information and visualization. Figure 11A In the example shown, the outline of the blood vessel 230 segmented from the route map is then used to generate curved longitudinal slices that traverse the route map, following the trajectory 1112 of the blood vessel (e.g., the centerline). (Similar to...) Figure 3A and Figure 3B Compared to the straight longitudinal slices shown, Figure 11A The curved slices shown increase the likelihood of displaying blood vessels in each frame, even when the vessels are curved or tortuous. The curved slices that cross the route map can then be displayed on the screen as a flattened image.

[0089] Figure 11B At least one embodiment of the present disclosure is shown in accordance with the provisions of this disclosure. Figure 11A The trajectory 1112 acquires a curved longitudinal image 200, from which a flattened longitudinal image 200 is derived. Blood vessels 230 and stenosis 1110 are visible. Although Figure 11A The diagram shows the vertical axis of the curved longitudinal slice oriented along the axial direction (i.e., aligned with the ultrasound imaging beam), but in some embodiments, the vertical axis of the curved longitudinal slice may, for example, be orthogonal to... Figure 3A The longitudinal slice is shown. In some embodiments, multiple curved slices are extracted to be displayed side by side simultaneously (e.g., one slice is axial-longitudinal and one slice is elevation-longitudinal). These multiple curved slices can help to better assess plaque load along the length and the roundness of the vessel lumen along the length of the vessel.

[0090] Methods for accumulating pairwise (frame-to-frame) registrations of multiple individuals into a single stitched roadmap can be applied. For example, techniques for accumulating pairwise registrations of multiple individuals into a single stitched roadmap may include optimizing multiple pairwise registrations to impose global constraints on the reconstructed roadmap. These constraints include consistency between multiple sets of overlapping images or volumes. For example, in sequential sweeps, the added rotation and translation estimated by registering the first frame to the second frame and then the second frame to the third frame should be consistent with the rotation and translation estimated by directly registering the first frame to the third frame (assuming overlap). Methods such as bundle adjustment, Simultaneous Localization and Mapping (SLAM), Kalman estimation, and other global optimization techniques can be applied to address multi-frame stitching problems.

[0091] Other prior constraints (e.g., known dimensions or lengths of the anatomical structures being mapped by the road, or information from secondary sensors or imaging) can be used to further improve the accuracy of the final roadmap reconstruction. Similarly, if structures within an image frame (e.g., vessels of interest) are segmented during or after roadmap sweeping, this information can be used to optimize the roadmap.

[0092] Figure 12 This is a schematic representation of an example two-pass scanning imaging technique for roadmap generation according to at least one embodiment of this disclosure. Figure 12 As shown, a roadmap can be constructed based on multiple scans. Furthermore, intermediate roadmaps constructed from each scan can be registered with each other to create a complete, expanded roadmap or resolve inaccuracies arising from any single roadmap. Figure 12 In the example shown, a series of 3D images are collected and registered in the first scan in the first direction, and then a series of biplane images are collected in the second scan in the opposite direction. Each of the biplane images is then able to be registered to the nearest 3D image.

[0093] If multiple scans or sweeps are performed, these scans or sweeps can be performed using different acquisition workflows (e.g., sequential acquisition workflows followed by incremental acquisition workflows, or vice versa) or different acquisition modes (e.g., 3D followed by dual-plane, or vice versa). For example, as Figure 12 The diagram illustrates that a first scan can be performed using 3D acquisition at a lower resolution and frame rate, while a second scan can be performed using dual-plane acquisition at a higher resolution and frame rate. By registering each dual-plane acquisition to the 3D acquisition acquired in the first scan, a high-resolution final roadmap can be produced, providing geometric accuracy through 3D volumetric acquisition.

[0094] In this disclosure, an ultrasound transducer sweeps across an extended area of ​​the anatomical structure to create an extended panoramic roadmap image. Typically, instead of a standard 2D probe, a matrix probe is used, and dual-plane and / or 3D, or combinations thereof, are the primary imaging modes during sweep. Out-of-plane probe motion from the user is automatically accounted for in a manner impossible with existing methods based on standard 2D image registration. The ultrasound roadmap generation system can be applied to any ultrasound imaging system, including external ultrasound, intracatheter or intravascular ultrasound, and transesophageal echo. The ultrasound roadmap generation system has the potential to significantly improve the accuracy of road mapping techniques and become an important new feature of future ultrasound platforms, particularly (but not limited to) those supporting matrix transducer technology.

[0095] The examples and embodiments described above may have many variations. For example, specialized ultrasound probes, beamformers, or processor circuits may be developed, optimized to work with ultrasound route mapping systems. The techniques described herein can be applied to fields other than human medicine, including veterinary medicine, materials testing, and manufacturing. The logical operations constituting embodiments of the techniques described herein are referred to differently as operations, steps, objects, elements, components, or modules. It should be understood that these items may occur, be performed, or be arranged in any order, and unless expressly stated otherwise, the language of the claims inherently forms a particular order.

[0096] All directional references (e.g., up, down, inside, outside, upward, downward, left, right, lateral, front, back, top, bottom, above, below, vertical, horizontal, clockwise, counterclockwise, proximal, and distal) are used for illustrative purposes only to aid the reader's understanding of the claimed subject matter and do not impose limitations, particularly regarding the location, orientation, or use of the ultrasonic route mapping system. Unless otherwise stated, connection references (e.g., attachment, coupling, connection, and joining) should be interpreted broadly and may include intermediate members between element sets and relative movement between elements. Therefore, a connection reference does not necessarily mean that two elements are directly connected and fixed to each other. The term "or" should be interpreted as "and / or," not "exclusive or." Unless otherwise stated in the claims, the values ​​recorded should be interpreted as illustrative only and not as limiting.

[0097] The foregoing description, examples, and data provide a complete description of the structure and use of exemplary embodiments of the ultrasonic route mapping system. While various embodiments of the claimed subject matter have been described above with a degree of specificity or with reference to one or more individual examples, those skilled in the art can make various changes to the disclosed embodiments without departing from the spirit or scope of the claimed subject matter.

[0098] Other embodiments are also contemplated. All content contained in the above description and shown in the accompanying drawings should be interpreted as illustrative of particular embodiments only, and not as limiting. Changes in detail or structure may be made without departing from the essential elements of the subject matter as defined in the claims.

Claims

1. A system for acquiring and displaying ultrasound images, comprising: A processor circuit configured to communicate with an ultrasound imaging device movable relative to a patient, wherein the processor circuit is configured to: Receive a first biplane or 3D image representing a first volume within the patient, a second biplane or 3D image representing a second volume within the patient, and a third biplane or 3D image representing a third volume within the patient; Register the first dual-plane or 3D image with the second dual-plane or 3D image to determine a first motion between the first dual-plane or 3D image and the second dual-plane or 3D image; The third dual-plane or 3D image is registered to at least one of the first dual-plane or 3D image or the second dual-plane or 3D image to determine a second motion between the third dual-plane or 3D image and at least one of the first dual-plane or 3D image or the second dual-plane or 3D image; Based on the determined first and second motions, a 2D route map image of the region of interest is generated by combining the first biplane or 3D image, the second biplane or 3D image, and the third biplane or 3D image; and Output a screen display including the 2D route map image to a display that communicates with the processor circuitry. Among them, two of the first dual-plane or 3D image, the second dual-plane or 3D image, and the third dual-plane or 3D image are 3D images. Wherein, one of the first dual-plane or 3D image, the second dual-plane or 3D image, and the third dual-plane or 3D image is a dual-plane image, and The processor circuit is further configured as follows: Register the two 3D images together; Register the dual-plane image to the nearest 3D image among the two 3D images; Extract image data from the dual-plane image; and The 2D route map image is generated based on the extracted image data.

2. The system according to claim 1, wherein, The processor circuit is configured to combine the first dual-plane or 3D image with the second dual-plane or 3D image by transforming the first dual-plane or 3D image relative to the second dual-plane or 3D image such that the first dual-plane or 3D image and the second dual-plane or 3D image are in the same coordinate system.

3. The system according to claim 1, wherein, The processor circuit is configured as follows: Identify image landmarks in each of the first biplane or 3D image and the second biplane or 3D image; and The first biplane or 3D image and the second biplane or 3D image are combined by the following operation: based on a determined first motion, at least one of the first biplane or 3D image or the second biplane or 3D image is transformed such that the image markers in the first biplane or 3D image are aligned with the image markers in the second biplane or 3D image.

4. The system according to claim 1, wherein, The resolution of the 3D image is lower than that of the dual-plane image.

5. A method for acquiring and displaying ultrasound images, comprising: The processor circuit receives a first biplane or 3D image of a first volume within a patient, a second biplane or 3D image of an overlapping second volume within the patient, and a third biplane or 3D image representing a third volume within the patient. The processor circuit is configured to communicate with an ultrasound imaging device that can move relative to the patient. Register the first dual-plane or 3D image with the second dual-plane or 3D image to determine a first motion between the first dual-plane or 3D image and the second dual-plane or 3D image; The third dual-plane or 3D image is registered to at least one of the first dual-plane or 3D image or the second dual-plane or 3D image to determine a second motion between the third dual-plane or 3D image and at least one of the first dual-plane or 3D image or the second dual-plane or 3D image; Using the processor circuitry, based on the determined first and second motions, a 2D route map image of the region of interest is generated by combining the first dual-plane or 3D image, the second dual-plane or 3D image, and the third dual-plane or 3D image; and Output a screen display including the 2D route map image to a display that communicates with the processor circuitry. Among them, two of the first dual-plane or 3D image, the second dual-plane or 3D image, and the third dual-plane or 3D image are 3D images. Wherein, one of the first dual-plane or 3D image, the second dual-plane or 3D image, and the third dual-plane or 3D image is a dual-plane image, and The method further includes: Register the two 3D images together; Register the dual-plane image to the nearest 3D image among the two 3D images; Extract image data from the dual-plane image; and The 2D route map image is generated based on the extracted image data.

6. The method according to claim 5, wherein, Generating the route map image includes: transforming the first dual-plane or 3D image relative to the second dual-plane or 3D image, such that the first dual-plane or 3D image and the second dual-plane or 3D image are in the same coordinate system.

7. The method according to claim 5, wherein, Generating the route map image includes: Identify image landmarks shared by the first dual-plane or 3D image and the second dual-plane or 3D image; and The first biplane or 3D image and the second biplane or 3D image are combined by the following operation: based on a determined first motion, at least one of the first biplane or 3D image or the second biplane or 3D image is transformed such that the image markers in the first biplane or 3D image are aligned with the image markers in the second biplane or 3D image.

8. The method according to claim 5, wherein, The resolution of the dual-plane image is higher than that of the 3D image.

9. A non-transient computer-readable medium storing instructions that, when executed by a processor included in the processor circuitry of a system according to any one of claims 1-4, cause the processor to perform the method according to any one of claims 5-8.

Images