System and method for image fusion rendering for registered multi-volumetric analysis

Abstract

A system is configured to perform actions including obtaining a first 3D medical image from a first medical imaging volume acquisition and obtaining a second 3D medical image from a second medical imaging volume acquisition, where the first medical imaging volume acquisition and the second medical imaging volume acquisition are integrally registered with each other. The actions include: receiving a selection of a region of interest in the second 3D medical image; and performing gradient domain fusion with blending of respective pixel intensities between the selected region of interest in the second 3D medical image and a region corresponding to the region of interest in the first 3D medical image to generate a blended region of interest. The actions include displaying the first 3D medical image in a first viewport on a user interface and displaying the blended region of interest in a second viewport located at the region in the first 3D medical image.

Description

Background Technology

[0001] The topics disclosed in this paper relate to image processing, and more specifically to systems and methods for image fusion rendering for registration multi-volume analysis.

[0002] Clinical decisions can be derived from the analysis of any number of datasets. In the field of radiology, this can involve the analysis of regions of interest from medical image data, which may include 2D or 3D medical images such as images of organs (kidneys, liver, spleen, etc.), blood vessels, bones, etc. In some examples, medical image analysis may be performed at the request of a clinician for a specific purpose, which may include detecting, assessing, and / or monitoring the progression of anatomical abnormalities (such as lesions, tumors, aneurysms, atrophy, and arterial stenosis, etc.).

[0003] Visualization tools enable access to regions of interest in medical image data and the performance of desired analyses. Rendering processes can be employed to separate the rendering of regions of interest, thereby improving the user interface for visualizing, detecting, evaluating, and monitoring various anatomical abnormalities. Summary of the Invention

[0004] The following provides an overview of some of the embodiments disclosed herein. It should be understood that these aspects are provided merely to give the reader a brief overview of these specific embodiments, and are not intended to limit the scope of this disclosure. In fact, this disclosure may cover various aspects that may not be set forth below.

[0005] In one embodiment, a system is provided. The system includes a memory that encodes processor-executable routines. The system also includes a processing system comprising one or more processors and configured to access the memory and execute the processor-executable routines, wherein the processor-executable routines, when executed by the processing system, cause the processing system to perform actions. These actions include: acquiring a first three-dimensional (3D) medical image from a first medical imaging volume and acquiring a second 3D medical image from a second medical imaging volume, wherein the first and second medical imaging volume acquisitions are globally registered with each other. The actions also include: receiving a selection of a region of interest (ROI) in the second 3D medical image. The actions further include: performing gradient-domain fusion using the blending of corresponding pixel intensities between the selected ROI in the second 3D medical image and a corresponding region in the first 3D medical image to generate a blended ROI. The actions further include: displaying the first 3D medical image in a first viewport on a user interface and displaying the blended ROI in a second viewport located at the region in the first 3D medical image corresponding to the ROI.

[0006] In another embodiment, a computer-implemented method is provided. This computer-implemented method includes: acquiring a first three-dimensional (3D) medical image from a first medical imaging volume via a processing system including one or more processors, and acquiring a second 3D medical image from a second medical imaging volume, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are globally registered with each other. The computer-implemented method further includes: receiving, at the processing system, a selection of a region of interest (ROI) in the second 3D medical image. The computer-implemented method further includes: performing gradient domain fusion via the processing system using the blending of corresponding pixel intensities between the selected ROI in the second 3D medical image and a corresponding region in the first 3D medical image to generate a blended ROI. The computer-implemented method further includes: displaying the first 3D medical image in a first viewport on a user interface via the processing system, and displaying the blended ROI in a second viewport located at the region in the first 3D medical image corresponding to the ROI.

[0007] In another embodiment, a non-transitory computer-readable medium is provided, comprising processor-executable code that, when executed by a processing system comprising one or more processors, causes the processing system to perform actions. These actions include: acquiring a first three-dimensional (3D) medical image from a first medical imaging volume and acquiring a second 3D medical image from a second medical imaging volume, wherein the first and second medical imaging volume acquisitions are globally registered with each other. The actions also include: receiving a selection of a region of interest (ROI) in the second 3D medical image. The actions further include: performing gradient-domain fusion to generate a blended ROI by blending corresponding pixel intensities between the selected ROI in the second 3D medical image and a corresponding region in the first 3D medical image. The actions further include: displaying the first 3D medical image in a first viewport on a user interface and displaying the blended ROI in a second viewport located at the region in the first 3D medical image corresponding to the ROI. Attached Figure Description

[0008] These and other features, aspects, and advantages of the invention will be better understood when reading the following detailed description with reference to the accompanying drawings, in which the same reference numerals denote the same parts throughout the drawings, wherein:

[0009] Figure 1 This is a schematic diagram of an example imaging system for generating 3D multi-volume imaging data according to various aspects of this disclosure;

[0010] Figure 2It is a block diagram of a computing device according to various aspects of this disclosure;

[0011] Figure 3 This is a flowchart of an image fusion rendering method for registration based on various aspects of this disclosure;

[0012] Figure 4 This is a flowchart of a method for performing gradient domain fusion using blending, according to various aspects of this disclosure;

[0013] Figure 5 This is a flowchart of a method for viewing different frames or slices of a region of interest, according to various aspects of this disclosure;

[0014] Figure 6 This is a flowchart of a method for changing the position of a second viewport according to various aspects of this disclosure;

[0015] Figure 7 This is a flowchart of a method for changing a region of interest according to various aspects of this disclosure;

[0016] Figure 8 Images illustrating the comparison of arterial lesions in the portal vein phase and delayed phase using naive solutions and gradient domain fusion with blending, according to various aspects of this disclosure, are depicted.

[0017] Figure 9 Images illustrating the comparison of diffuse lesions in images acquired from different sequences using naive solutions and gradient domain fusion with blending, according to various aspects of this disclosure;

[0018] Figure 10 This is a schematic diagram illustrating image fusion rendering for registration using multi-volume analysis according to various aspects of this disclosure; and

[0019] Figure 11 This is an example of a graphical user interface on a display having an integrated viewport on another viewport, according to various aspects of this disclosure. Detailed Implementation

[0020] One or more specific implementations will be described below. To provide a concise description of these implementations, not all features of the actual implementation will be described in this specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions must be made to achieve the developer's specific objectives, such as complying with system-related and business-related constraints that may differ from implementation to implementation. Furthermore, it should be understood that such development efforts may be complex and time-consuming, but will in any case remain routine tasks of design, fabrication, and manufacturing for those skilled in the art who benefit from this disclosure.

[0021] When describing elements of various embodiments of the subject matter of this invention, the articles “a,” “an,” “the,” and “described” are intended to indicate the presence of one or more elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to the listed elements. Furthermore, any numerical examples in the following discussion are intended to be non-limiting, and therefore the additional values, ranges, and percentages are within the scope of the disclosed embodiments.

[0022] Some general information is provided to offer general background to the various aspects of this disclosure and to facilitate understanding and interpretation of certain technical concepts described herein.

[0023] As used herein, the terms “processor,” “processing system,” or “processing unit” refer to any type of processing unit capable of performing the required computations for various implementation schemes, such as single-core or multi-core: CPU, Accelerated Processing Unit (APU), graphics board, DSP, FPGA, ASIC, or combinations thereof.

[0024] As used herein, the term "computing system" refers to an electronic computing device, such as, but not limited to, a single computer, virtual machine, virtual container, host, server, laptop computer, and / or mobile device, or multiple electronic computing devices working together to perform functions described as being performed on or by a computing system. As used herein, the terms "application," "application module" (or "module"), "engine," or "program" or "plugin" refer to one or more sets of computer software instructions (e.g., computer programs and / or scripts) that can be executed by one or more processors of a computing system to provide a specific function. Computer software instructions may be written in any suitable programming language, such as C, C++, C#, Pascal, Fortran, Perl, MATLAB, SAS, SPSS, JavaScript, AJAX, and JAVA. Such computer software instructions may include standalone applications with data input and data display aspects (e.g., modules). Alternatively, the disclosed computer software instructions may be classes instantiated as distributed objects. The disclosed computer software instructions may also be component software, such as JAVABEANS or ENTERPRISE JAVABEANS. Additionally, the disclosed application or engine may be implemented in computer software, computer hardware, or a combination thereof.

[0025] As used herein, the terms “automatic” and “automatically” refer to actions performed by a computing device or computing system (e.g., in one or more computing devices) without human intervention. For example, an automatically executed function may be performed by a computing device or system solely based on data stored on and / or received by that computing device or system, even without prompting from a human user. As a non-limiting example only, a computing device or system may make decisions and / or initiate other functions solely based on decisions made by the computing device or system, regardless of any other input relating to the decision.

[0026] Image registration is crucial for multiphase, multimodal, and subsequent analysis in medical imaging. Image registration allows the establishment of spatial and temporal correspondences between images. In multiphase analysis, alignment enables accurate comparisons of changes over time, facilitating the assessment of disease progression, treatment response, and the identification of potential biomarkers. In subsequent analysis, image registration enables the precise overlay of images acquired at different time points (such as pre-treatment and post-treatment scans). This allows clinicians to accurately track changes in patient condition over time, assess the effectiveness of interventions, and make informed decisions regarding further treatment strategies. Multimodal fusion involves integrating information from different imaging modalities, such as magnetic resonance (MR), computed tomography (CT), and positron emission tomography (PET) scans. By aligning these modalities, clinicians benefit from a more comprehensive and complementary view of underlying anatomy and pathology, improving diagnostic accuracy and treatment planning. Image registration plays a key role in achieving accurate spatial and temporal alignment of medical images. This capability is essential for enhancing diagnostic accuracy, monitoring disease progression, and assessing treatment response across various clinical applications.

[0027] Current strategies for comparing registration results may offer static views that lack easy user interaction, red-green-blue fusion views that transform the true grayscale values ​​for clinical interpretation of damage, checkerboard methods, or multiple viewports requiring a high level of attention to focus on the same point. Integrative registration provides the ability to align and fuse two volumetric acquisitions from the same or different modalities. This enables easy comparison of three-dimensional (3D) anatomical images from CT, MR & PET, SPECT, and X-ray angiography for comprehensive analysis. The view of the fused image is crucial for evaluating registration results in medical imaging, as it allows for visual assessment of the alignment and integration of information from different scanning modalities into a single synthetic image.

[0028] This disclosure provides systems and methods for image fusion rendering using multi-volume analysis for registration. In the disclosed embodiments, the system and method include: acquiring a first three-dimensional (3D) medical image from a first medical imaging volume and acquiring a second 3D medical image from a second medical imaging volume, wherein the first and second medical imaging volume acquisitions are globally registered with each other. The system and method also include: receiving a region of interest (ROI) selection in the second 3D medical image. The system and method further include: performing gradient domain fusion to generate a blended ROI by blending or integrating corresponding pixel intensities (e.g., signal intensities) between the selected ROI in the second 3D medical image and a corresponding region in the first 3D medical image. Blending may occur using Poisson blending, Laplacian pyramid blending, gradient domain fusion, or deep learning-based blending techniques. The system and method also include: displaying the first 3D medical image in a first viewport on a user interface and displaying the blended ROI in a second viewport located at the region in the first 3D medical image corresponding to the ROI. In some embodiments, the first medical imaging volume and the second medical imaging volume are acquired from the same imaging modality. In some embodiments, the first medical imaging volume and the second medical imaging volume are acquired from different imaging modalities.

[0029] The disclosed embodiments improve efficiency by providing a user interface that enables easy comparison of registered images. The disclosed embodiments also improve efficiency by providing a user interface that enables local analysis of two image sequences of different types. The disclosed embodiments enable comparison of registered images with less user interaction, where the region of interest can be easily modified for comparison with the registered image. The disclosed embodiments provide an embedded viewport that can be navigated while within another viewport. The disclosed embodiments improve the evaluation of registration results in medical imaging by enabling easy visual assessment of alignment and integration of information. Therefore, the disclosed embodiments improve diagnostic accuracy, monitoring of disease progression, and evaluation of treatment response across a variety of clinical applications.

[0030] Turn now Figure 1This illustrates an exemplary imaging system that can be used to generate 3D multi-volume imaging data. As an example, the MRI system 10 includes a static magnetic field magnet unit 12, a gradient coil unit 13, an RF coil unit 14, an RF volume or volumetric coil unit 15, a transmit / receive (T / R) switch 20, an RF driver unit 22, a gradient coil driver unit 23, a data acquisition unit 24, a controller unit 25, a patient bed or examination table 26, an image processing unit 31, an operation console unit 32, and a display device 33. In some examples, the RF coil unit 14 is a surface coil, which is a local coil typically placed near the anatomical structures of interest of the subject 16. In this document, the RF volumetric coil unit 15 is the transmitting coil that transmits RF signals, and the local surface RF coil unit 14 receives MR signals. Therefore, the transmitting volumetric coil (e.g., the RF volumetric coil unit 15) and the surface receiving coil (e.g., the RF coil unit 14) are independent but electromagnetically coupled components. The MRI system 10 sends electromagnetic pulse signals to a subject 16 placed in an imaging space 18, where a static magnetic field is formed to perform a scan to obtain magnetic resonance signals from the subject 16. One or more images of the subject 16 can be reconstructed based on the magnetic resonance signals obtained by the scan.

[0031] The static magnetic field magnet unit 12 includes, for example, a toroidal superconducting magnet mounted within a toroidal vacuum container. The magnet defines a cylindrical space surrounding the subject 16 and generates a constant main static magnetic field B0.

[0032] The MRI system 10 also includes a gradient coil unit 13 that forms a gradient magnetic field in the imaging space 18 to provide three-dimensional positional information for magnetic resonance signals received by the RF coil array. The gradient coil unit 13 includes three gradient coil systems, each generating a gradient magnetic field along one of three spatial axes perpendicular to each other, and generating gradient fields in each of the frequency encoding direction, phase encoding direction, and slice selection direction, depending on the imaging conditions. More specifically, the gradient coil unit 13 applies a gradient field in the slice selection direction (or scan direction) of the subject 16 to select slices; and the RF body coil unit 15 or the local RF coil array can send RF pulses to the selected slices of the subject 16. The gradient coil unit 13 also applies a gradient field in the phase encoding direction of the subject 16 to perform phase encoding of the magnetic resonance signals from the RF pulse-excited slices. Then, the gradient coil unit 13 applies a gradient field in the frequency encoding direction of the subject 16 to perform frequency encoding of the magnetic resonance signals from the RF pulse-excited slices.

[0033] RF coil unit 14 is configured, for example, to surround the imaging region of subject 16. In some examples, RF coil unit 14 may be referred to as a surface coil or receiving coil. In the static magnetic field space or imaging space 18, where a static magnetic field B0 is formed by static magnetic field magnet unit 12, RF coil unit 15 sends RF pulses as electromagnetic waves to subject 16 based on control signals from controller unit 25, thereby generating a high-frequency magnetic field B1. This excites proton spins in the slice of subject 16 to be imaged. RF coil unit 14 receives electromagnetic waves generated as magnetic resonance signals when the proton spins thus excited in the slice of subject 16 to be imaged return to alignment with the initial magnetization vector. In some examples, RF coil unit 14 may transmit RF pulses and receive MR signals. In other examples, RF coil unit 14 may be used only to receive MR signals without transmitting RF pulses.

[0034] RF body coil unit 15 is configured, for example, to surround imaging space 18 and generate RF magnetic field pulses within imaging space 18 that are orthogonal to the main magnetic field B0 generated by static magnetic field magnet unit 12 to excite the nucleus. RF coil unit 14 can be detached from MRI system 10 and replaced with another RF coil unit, whereas RF body coil unit 15 is fixedly attached to and connected to MRI system 10. Furthermore, while local coils (such as RF coil unit 14) can only send or receive signals to or from a local area of ​​subject 16, RF body coil unit 15 typically has a larger coverage area. For example, RF body coil unit 15 can be used to send or receive signals to or from the whole body of subject 16. Using only receiving local coils and transmitting body coils provides uniform RF excitation and good image homogeneity, at the cost of higher RF power deposited in the subject. For transmit-receive local coils, the local coil provides RF excitation to the region of interest and receives MR signals, thereby reducing the RF power deposited in the subject. It should be understood that the specific use of RF coil unit 14 and / or RF body coil unit 15 depends on the imaging application.

[0035] When operating in receive mode, T / R switch 20 selectively connects RF body coil unit 15 to data acquisition unit 24, and when operating in transmit mode, it selectively connects the RF body coil unit to RF driver unit 22. Similarly, when RF coil unit 14 operates in receive mode, T / R switch 20 selectively connects RF coil unit 14 to data acquisition unit 24, and when the RF coil unit operates in transmit mode, it selectively connects the RF coil unit to RF driver unit 22. When both RF coil unit 14 and RF body coil unit 15 are used for a single scan, for example, if RF coil unit 14 is configured to receive MR signals and RF body coil unit 15 is configured to transmit RF signals, T / R switch 20 can direct control signals from RF driver unit 22 to RF body coil unit 15 while simultaneously directing the received MR signals from RF coil unit 14 to data acquisition unit 24. The coil of RF body coil unit 15 can be configured to operate in transmit-only mode or transmit-receive mode. The coil of local RF coil unit 14 can be configured to operate in transmit-receive mode or receive-only mode.

[0036] RF driver unit 22 includes a gate modulator (not shown), an RF power amplifier (not shown), and an RF oscillator (not shown), which are used to drive an RF coil (e.g., RF coil unit 15) and generate a high-frequency magnetic field in the imaging space 18. Based on a control signal from controller unit 25 and using the gate modulator, RF driver unit 22 modulates the RF signal received from the RF oscillator into a signal with a predetermined timing and a predetermined envelope. The RF signal modulated by the gate modulator is amplified by the RF power amplifier and then output to RF coil unit 15.

[0037] The gradient coil driver unit 23 drives the gradient coil unit 13 based on control signals from the controller unit 25, thereby generating a gradient magnetic field in the imaging space 18. The gradient coil driver unit 23 includes three systems (not shown) of driver circuits corresponding to the three gradient coil systems included in the gradient coil unit 13.

[0038] The data acquisition unit 24 includes a preamplifier (not shown), a phase detector (not shown), and an analog-to-digital converter (not shown) for acquiring the magnetic resonance signal received by the RF coil unit 14. In the data acquisition unit 24, the phase detector uses the output of the RF oscillator from the RF driver unit 22 as a reference signal to perform phase detection on the magnetic resonance signal received from the RF coil unit 14 and amplified by the preamplifier. The phase-detected analog magnetic resonance signal is then output to the analog-to-digital converter for conversion into a digital signal. The resulting digital signal is then output to the image processing unit 31.

[0039] The MRI apparatus 10 includes an examination table 26 for placing a subject 16 thereon. The subject 16 can be moved inside and outside the imaging space 18 by moving the examination table 26 based on control signals from the controller unit 25.

[0040] The controller unit 25 includes a computer and a recording medium on which a program to be executed by the computer is stored. When executed by the computer, the program causes various parts of the device to perform operations corresponding to a predetermined scan. The recording medium may include, for example, a ROM, floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, or non-volatile memory card. The controller unit 25 is connected to the operation console unit 32 and processes operation signals input to the operation console unit 32, and also controls the examination table 26, RF driver unit 22, gradient coil driver unit 23, and data acquisition unit 24 by outputting control signals to them. The controller unit 25 also controls the image processing unit 31 and the display device 33 based on operation signals received from the operation console unit 32 to obtain a desired image.

[0041] The operation console unit 32 includes user input devices such as a touchscreen, keyboard, and mouse. The operation console unit 32 is used by the operator to input data, for example, as an imaging protocol, and to set the area where the imaging sequence will be executed. Data regarding the imaging protocol and the area where the imaging sequence will be executed is output to the controller unit 25.

[0042] The image processing unit 31 includes a computing device and a recording medium on which a program to be executed by the computing device to perform predetermined data processing is recorded. The image processing unit 31 is connected to the controller unit 25 and performs data processing based on control signals received from the controller unit 25. The image processing unit 31 is also connected to the data acquisition unit 24 and generates spectral data by applying various image processing operations to the magnetic resonance signal output from the data acquisition unit 24.

[0043] Display device 33 can display one or more images within the GUI on its display screen based on control signals received from controller unit 25. Display device 33 displays, for example, images of input items about which the operator inputs operational data from operation console unit 32. Display device 33 also displays two-dimensional (2D) slice images or three-dimensional (3D) images of subject 16 generated by image processing unit 31.

[0044] The MRI system 10 can be configured for multi-volume imaging, such as multi-parameter and / or multi-phase imaging, wherein multiple imaging sequences and / or phases are imaged during a single imaging session. The resulting MRI imaging data can include images from each of the imaged sequences and / or phases, wherein the MRI imaging data is subdivided into specified sequences and / or phases. Each of the specified sequences and / or phases can define multiple 2D slices, each of which is specific to the z-coordinate of the MRI imaging data. The z-coordinate can therefore define multiple 2D slices, each slice originating from each of the specified sequences and / or phases.

[0045] Although the MRI system has been described by way of example, it should be understood that this technique can be applied to images acquired using other imaging systems (such as CT, tomography, PET, ultrasound, etc.) capable of multi-parameter, multi-phase, or other types of multi-volume imaging. The present invention discussion of the MRI imaging modality is provided only as an example of a suitable imaging modality.

[0046] Figure 2 This is a block diagram of an example computing device 200 capable of rendering medical imaging data. The computing device 200 can be, for example, a medical imaging system, such as a medical imaging system 10, a CT scanner, a PET scanner, an ultrasound scanner, a hospital monitor, a laptop computer, a desktop computer, a tablet computer, or a mobile phone. The computing device 200 may include a processor 202 adapted to execute stored instructions, and a memory device 204 storing instructions executable by the processor 202. The processor 202 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory device 204 may include random access memory, read-only memory, flash memory, or any other suitable memory system. (The following text is about...) Figures 3 to 4 In more detail, the instructions executed by processor 202 can be used to implement methods for rendering medical imaging data.

[0047] The processor 202 can also be linked via system interconnect 206 (e.g., PCI, PCI-Express, NuBus, etc.) to a display interface 208 suitable for connecting the computing device 200 to the display device 210. The display device 210 may include a display screen as a built-in component of the computing device 200. The display device 210 may also include externally connected computer monitors, televisions, or projectors connected to the computing device 200. The display device 210 may include light-emitting diodes (LEDs) and micro-LEDs, organic light-emitting diode (OLED) displays, etc.

[0048] The processor 202 can be connected to an input / output (I / O) device interface 212 via a system interconnect 206, which is adapted to connect the computing device 200 to one or more I / O devices 214. For example, the I / O device 214 may include a keyboard and a pointing device, wherein the pointing device may include a touchpad or a touch screen, etc. The I / O device 214 may be a built-in component of the computing device 200 or may be an external device connected to the computing device 200.

[0049] In some examples, processor 202 may also be linked to storage device 216 via system interconnect 206, which may include a hard disk drive, optical drive, USB flash drive, drive array, or any combination thereof. In some examples, storage device 216 may include any suitable application. In some examples, storage device 216 may include a region of interest (ROI) manager 218. In some examples, ROI manager 218 may use a 3D cursor to obtain selection of a region of interest from a 3D medical image. In some examples, storage device 216 may also include a 3D cursor manager 220 that detects 3D cursor settings for the region of interest. These 3D cursor settings may indicate at least rendering settings for the region of interest. As mentioned herein, rendering settings may include maximum intensity projection, minimum intensity projection, or average intensity projection of pixels or voxels within a region of medical imaging data. In some examples, storage device 216 may also include a user interface manager 222 that may modify the user interface including the 3D medical image using the 3D cursor settings applied within the region of interest. User interface manager 222 can modify the user interface to display a corresponding 3D medical image acquired from a corresponding medical imaging volume. Furthermore, user interface manager 222 can modify the user interface to display a selected region of interest (ROI) within a 3D medical image on a viewport integrated onto another 3D medical image (in another viewport) in a region corresponding to the selected ROI. Specifically, the viewport integrated onto the other 3D medical image displays a blended ROI generated by gradient domain fusion using blending of the selected ROI and a corresponding ROI in the other 3D medical image. Storage device 216 includes a blending manager 223 configured to perform gradient domain fusion using blending to generate a blended ROI between a selected ROI in a source image and a corresponding region in a target image. Blending can occur using Poisson blending, Laplacian pyramid blending, gradient domain fusion, or deep learning-based blending techniques.

[0050] In some examples, a network interface controller (also referred to herein as a NIC) 224 may be adapted to connect computing device 200 to network 226 via system interconnect 206. Network 226 may be a cellular network, radio network, wide area network (WAN), local area network (LAN), or the Internet, etc. Network 226 may enable data such as alarms, as well as other data, to be transmitted from computing device 200 to remote computing devices, remote display devices, etc. For example, network 226 may enable remote devices (e.g., imaging archive 228, etc.) to generate or modify user interfaces by rendering any number of regions of interest in a medical imaging dataset using different rendering settings and other features.

[0051] It should be understood that Figure 2 The block diagram is not intended to indicate that computing device 200 will include Figure 2 All components shown in the diagram. Instead, computing device 200 may include... Figure 2 Fewer or additional components not illustrated herein (e.g., additional memory components, embedded controllers, additional modules, additional network interfaces, etc.). Furthermore, any functionality of the ROI Manager 218, 3D Cursor Manager 220, or User Interface Manager 222 may be implemented, partially or wholly, in the hardware and / or processor 202. For example, this functionality may be implemented using an application-specific integrated circuit, logic implemented in an embedded controller, or logic implemented in the processor 202. In some examples, the functionality of the ROI Manager 218, 3D Cursor Manager 220, or User Interface Manager 222 may be implemented using logic components, wherein logic components as mentioned herein may include any suitable hardware (e.g., processors, etc.), software (e.g., applications, etc.), firmware, or any suitable combination of hardware, software, and firmware.

[0052] In some examples, computing device 200 may be incorporated into an imaging system such as MRI system 10. For example, computing device 200 may be the image processing unit 31 of MRI system 10. However, in other examples, computing device 200 may be located at a device (e.g., a server, edge device, etc.) communicatively coupled to the imaging system via wired and / or wireless connections. In some examples, at least a portion of computing device 200 may be located at a separate device (e.g., a workstation) that can receive images from the imaging system or from a storage device storing images generated by the imaging system and / or other additional imaging systems.

[0053] In addition to images directly provided by computing device 200, images may further originate from imaging archives 228 communicatively coupled to computing device 200. Imaging archives 228 may include, for example, a Picture Archiving and Communication System (PACS), a Vendor Neutral Archive (VNA), or other suitable medical image database. Medical imaging archives may be hosted on a remote server configured to allow computing device 200 to access multiple medical images and patient data hosted thereon. In some examples, the multiple medical images stored in imaging archives 228 may be of different types, such as MRI images, CT images, or ultrasound images, and these medical images may be stored in imaging archives 228 for one or more patients.

[0054] Figure 3 This is a flowchart of a method 300 for image fusion rendering using multi-volume analysis for registration. One or more steps of method 300 may be performed by... Figure 2One or more components of the computing device 200 in the computer perform the operation.

[0055] Method 300 includes: acquiring a first medical imaging volume and a second imaging volume (box 302). In some embodiments, the first and second medical imaging volume acquisitions are acquired using different medical imaging modalities. In some embodiments, the first and second medical imaging volume acquisitions are acquired using the same medical imaging modality. In some embodiments, the first and second medical imaging volume acquisitions may have been acquired for multiphase analysis (e.g., an example using the same medical imaging modality). For example, multiphase analysis may be a CT study using an injected contrast agent. Multiphase imaging data includes multiple images of a target anatomical structure typically taken at different time points as an intravenous contrast agent moves through the circulatory system. As an example, a multiphase liver CT may include non-contrast phase images, arterial phase images (e.g., late arterial phase), portal vein phase images, and delayed phase images, wherein each image is acquired at different times when the contrast agent is providing specific enhancement to a designated region. For example, for the arterial phase, peak aortic attenuation with minimal liver enhancement may be seen, while for the portal vein phase, peak liver parenchyma enhancement as well as portal and hepatic vein enhancement may be seen. In some implementations, a first medical imaging volume acquisition and a second medical imaging volume acquisition may have been acquired for multiparameter analysis (e.g., an example using the same medical imaging modality). Multiparameter imaging data can be combined for multiple imaging parameters (e.g., sequences) of a 3D medical imaging dataset. For example, multiparameter MRI may include data from multiple sequences, such as T1-weighted sequences, T2-weighted sequences, T1-contrast enhancement (T1CE) sequences, fluid attenuation inversion recovery (FLAIR) sequences, diffusion-weighted imaging (DWI) sequences, etc.

[0056] Method 300 further includes: overall registration of the first medical imaging volume acquisition with the second medical imaging volume acquisition (box 304). Overall registration enables alignment and fusion of two volume acquisitions from the same or different medical imaging modalities. In some embodiments, the registration may be rigid. In some embodiments, the registration may be non-rigid (e.g., deformable).

[0057] Method 300 further includes: acquiring a first three-dimensional (3D) medical image (e.g., a slice) from a first medical imaging volume, and acquiring a second 3D medical image (e.g., a slice) from a second medical imaging volume (box 306). Method 300 even further includes: (e.g., via...) Figure 2The I / O device 214 in the second 3D medical image receives a selection of a region of interest (ROI) in the second 3D medical image (box 308). Method 300 further includes performing gradient domain fusion by blending the corresponding pixel intensities between the selected ROI in the second 3D medical image and a corresponding region in the first 3D medical image to generate a blended ROI (box 310). Gradient domain fusion seamlessly blends objects or textures from a source image (e.g., the second 3D medical imaging volume) into a target image (e.g., the first medical imaging volume). In gradient domain fusion, the source image is modified such that the gradients of the source image are preserved to the greatest extent possible, while the overall intensity matches the target image. Gradient domain fusion is superior to a naive solution (i.e., copying and pasting the source image into the target image), which results in an unnatural appearance. Therefore, gradient domain fusion provides improved image quality. Figure 4 The text describes gradient domain fusion using blending in more detail. Blending can occur using Poisson blending, Laplacian pyramid blending, gradient domain fusion, or deep learning-based blending techniques.

[0058] Method 300 includes: displaying the first 3D medical image (box 312) in a first viewport on a user interface (e.g., a graphical user interface). Method 300 also includes: (e.g., via...) Figure 2 The I / O device 214 receives first user input to display a blended region of interest in a second viewport located (e.g., embedded in) a region in the first 3D medical image corresponding to the region of interest (box 314). In some embodiments, method 300 further includes displaying a second 3D medical image in a third viewport adjacent to the first viewport (box 316). In some embodiments, the second viewport is a dynamic cursor that can be utilized as described in U.S. Patent Application No. 18 / 496,782, filed October 27, 2023, entitled “METHODS AND SYSTEMS FOR MEDICAL IMAGE RENDERING,” the entire contents of which are incorporated herein by reference. Specifically, the second viewport may be a three-dimensional dynamic cursor having three-dimensional cursor settings that can indicate at least rendering settings for the region of interest shown in the second viewport. As mentioned herein, rendering settings may include maximum intensity projection, minimum intensity projection, or average intensity projection of pixels or voxels within a region of medical imaging data. In some embodiments, method 300 includes: (e.g., via...) Figure 2The I / O device 214 in the first 3D medical image receives a second user input to hide the second viewport and, alternatively, to display the region in the first 3D medical image corresponding to the region of interest (box 318). Boxes 312 and 318 may occur at different points in method 300 and whenever it is necessary to turn the display of the second viewport in the user interface on / off.

[0059] Figure 4 This is a flowchart of method 320 for performing gradient domain fusion using co-mixing. One or more steps of method 320 may be performed by... Figure 2 One or more components of the computing device 200 in the computer perform the operation.

[0060] Method 320 includes: obtaining a first signal (pixel intensity or value) from a first 3D medical image corresponding to a region of interest selected in a second 3D medical image (box 322). Method 320 also includes: obtaining a second signal (pixel intensity or value) from the selected region of interest in the second 3D medical image (box 324). Boxes 322 and 324 can be performed simultaneously. Method 320 also includes: calculating the corresponding derivatives of the first and second signals (box 326). The average value of the corresponding derivatives is centered at zero to provide a smoother blend in the derivative space. Method 320 further includes: combining or blending the corresponding derivatives to generate a blended derivative (box 328). Method 320 also includes: (via integration) reconstructing the blended signal based on the blended derivative (box 330). Method 320 further includes: generating a blended region of interest (i.e., a blended region of interest image) based on the blended signal (box 332).

[0061] Figure 5 This is a flowchart of method 334 for viewing different frames or slices of a region of interest. One or more steps of method 334 may be derived from... Figure 2 The method is executed by one or more components of the computing device 200. Method 334 may be combined with... Figure 3 Method 320 in the middle or executed after that method.

[0062] Method 334 includes: (e.g., via) Figure 2The I / O device 214 receives user input to change to different 3D medical images acquired from a second medical imaging volume, wherein the region of interest is the same in the different 3D medical images (box 336). In some embodiments, the user input may be a toggle on a mouse. This enables scrolling between different slices or frames (within a second viewport) of the region of interest in the second medical imaging volume acquisition, independent of the first 3D medical image (and the first medical imaging volume acquisition). This enables analysis of potential shifts in registration in the direction of the navigation axis. Method 334 further includes performing gradient domain fusion by blending the corresponding pixel intensities between the region of interest in the different 3D medical images and the corresponding region in the first 3D medical image to generate different blended regions of interest (box 338). Blending may occur using Poisson blending, Laplacian pyramid blending, gradient domain fusion, or deep learning-based blending techniques. Method 334 further includes: displaying a first 3D medical image in a first viewport on a user interface (e.g., a graphical user interface) and displaying different blended regions of interest in a second viewport located in the region of the first 3D medical image corresponding to the region of interest (box 340).

[0063] Figure 6 This is a flowchart of method 342 for changing the position of the second viewport. One or more steps of method 342 may be derived by... Figure 2 One or more components of the computing device 200 in the process are executed. Method 342 may be combined with Figure 3 Method 320 in the middle or executed after that method.

[0064] Method 342 includes: (e.g., via) Figure 2 The I / O device 214 in the first 3D medical image receives user input that changes the position of the second viewport on the first 3D medical image (box 344). Method 342 further includes obtaining different regions of interest (ROIs) in the second 3D medical image, which correspond to the position of the second viewport on the first 3D medical image (box 346). Method 342 further includes performing gradient domain fusion to generate different blended ROIs by blending corresponding pixel intensities between the different ROIs in the second 3D medical image and different regions in the first 3D medical image corresponding to the different ROIs (box 348). Blending may occur using Poisson blending, Laplacian pyramid blending, gradient domain fusion, or deep learning-based blending techniques. Method 342 even includes displaying the first 3D medical image in the first viewport on a user interface (e.g., a graphical user interface) and displaying the different blended ROIs in the second viewport, which is located at the different regions in the first 3D medical image corresponding to the different ROIs (box 350).

[0065] Figure 7 This is a flowchart of method 352 for changing the region of interest. One or more steps of method 352 may be derived from... Figure 2 One or more components of the computing device 200 in the process are executed. Method 352 may be combined with Figure 3 Method 320 in the middle or executed after that method.

[0066] Method 352 includes receiving another selection of different regions of interest in a second 3D medical image (box 354). Method 352 further includes performing gradient domain fusion by blending the corresponding pixel intensities between the different regions of interest selected in the second 3D medical image and different regions in the first 3D medical image corresponding to the different regions of interest, to generate different blended regions of interest (box 355). Blending may occur using Poisson blending, Laplacian pyramid blending, gradient domain fusion, or deep learning-based blending techniques. Method 352 further includes changing the display position of the different blended regions in a second viewport of the first 3D medical image on a user interface (e.g., a graphical user interface) to correspond to the different regions of interest (box 356).

[0067] Figure 8 Images illustrating the comparison of arterial lesions in the portal vein phase and delayed phase using naive fusion and gradient domain fusion with blending (e.g., Poisson blending) are depicted. The images are derived from multiphase liver CT scans of the subject, where each image was acquired at a different time when the contrast dye was applied to a specific region with particular enhancement. Top row 358 includes an arterial phase image 360 ​​(e.g., lateral arterial phase). A box 362 on image 360 ​​marks a region of interest (ROI) with high arterial enhancement, which is copied and pasted onto the corresponding region on the portal vein phase and delayed phase images via naive fusion (e.g., conventional blending). Top row 358 also includes a portal vein phase image 364 having a ROI fused (via naive fusion) in the corresponding ROI of image 360. The corresponding ROI in image 364 without fused ROI is removed. Top row 358 also includes a delayed phase image 366 having a ROI fused (via naive fusion) in the corresponding ROI of image 360. In both images 364 and 366, the fused region of interest from image 360 ​​looks unnatural on both images 364 and 366.

[0068] Bottom row 368 includes an arterial phase image 370 (which is identical to image 360) with an unlabeled region of interest. The region of interest in box 362 on image 360 ​​is also fused into the corresponding region on the portal vein phase image and the delayed phase image via gradient domain fusion using Poisson blending (i.e., high-level fusion). Bottom row 368 also includes a portal vein phase image 372, which has a region of interest fused into the corresponding region of interest in image 360 ​​(via gradient domain fusion using Poisson blending). The corresponding region of interest in image 372 without fused regions of interest is removed. Bottom row 368 also includes a delayed phase image 374, which has a region of interest fused into the corresponding region of interest in image 360 ​​(via gradient domain fusion using Poisson blending). Of both images 372 and 374, the fused region of interest from image 360 ​​appears more natural on images 372 and 374.

[0069] Figure 9 Images illustrating the comparison of diffuse lesions in images acquired with different sequences using naive solutions and gradient domain fusion employing blending (e.g., Poisson blending). The images are from MR multiparameter acquisitions of a subject's prostate (e.g., using different sequences). Top row 375 includes a T2-weighted image 376 (e.g., acquired with a T2-weighted sequence), an arterial contrast agent (ABC) image 378 (e.g., acquired with an ABC sequence), and a diffusion-weighted image 380 (e.g., acquired with a diffusion-weighted sequence). Boundary 381 in image 380 includes the region of interest (ROI) of the diffuse lesion. Bottom row 382 includes a diffusion-weighted image 384 (same as image 378) of the unlabeled ROI. The ROI (diffuse lesion) in image 380 is fused to the corresponding region on the T2-weighted image (e.g., image 376) and the ABC image (e.g., image 378) via gradient domain fusion employing Poisson blending (i.e., high-level fusion). Bottom row 382 includes a T2-weighted image 386 having a region of interest fused in the corresponding region of interest in image 380 (via gradient domain fusion using Poisson blending). Bottom row 382 also includes an ABC image 388 having a region of interest fused in the corresponding region of interest in image 380 (via gradient domain fusion using Poisson blending).

[0070] Figure 10This is a schematic diagram illustrating image fusion rendering for registration using multi-volume analysis. Two medical imaging volumes of the subject, namely volumes A and B, are acquired and globally registered with each other. Images 390 and 392 are displayed for volumes A and B respectively (e.g., on a user interface). Cursor 394 marks the region of interest selected in image 392. The region of interest selection performed in image 392 undergoes gradient domain fusion using blending and is fused into the corresponding region in image 392 indicated by another cursor 396. Blending can occur using Poisson blending, Laplacian pyramid blending, gradient domain fusion, or deep learning-based blending techniques. Cursors 394 and 396 can be dynamic 3D cursors as described above.

[0071] Figure 11 This is an example of a graphical user interface 398 on display 210 with an integrated viewport 400 on another viewport 402. Two medical imaging volumes of a subject are acquired and (e.g., via non-rigid registration) they are globally registered to each other. Images 404 and 406 are displayed adjacent to each other on the graphical user interface 398 for the acquisition of the two medical imaging volumes in viewports 402 and 408, respectively. A cursor marks a region of interest selected in image 406. The region of interest in image 406 undergoes gradient domain fusion using blending and is blended into the corresponding region in image 404 shown within viewport 400. Blending may occur using Poisson blending, Laplacian pyramid blending, gradient domain fusion, or deep learning-based blending techniques. Viewport 400 is a dynamic 3D cursor as described above. In some embodiments, this is achieved via user input (e.g., via...). Figure 2 In the I / O device 214), viewport 400 (and the blended regions of interest therein) can be hidden (i.e., rendering features are turned off), and the corresponding regions in image 404 are instead shown. In some embodiments, via user input (e.g., via...) Figure 2 (I / O device 214 in the image), viewport 400 can be shown again on viewport 402 (i.e., on image 404) (i.e., rendering features are enabled). In some embodiments, a region of interest within viewport 400 is selected or viewport 400 is selected and (e.g., via...) Figure 2 The I / O device 214 in the image is scrolled so that the region of interest can be shown in another slice from the medical imaging volume acquisition (from which the region of interest is derived) to be shown in viewport 400, and the slice (i.e., image) of the region of interest shown in viewport 400 is shown in viewport 408. In some embodiments, the position of viewport 402 may be (e.g., via...) Figure 2The I / O device 214 in the image 404 changes position on viewport 400 (i.e., on image 406), and the region of interest in image 404 corresponding to the new position in viewport 400 will be displayed within viewport 400 (while cursor 410 also changes position to the corresponding region of interest in image 406). In some embodiments, the position of cursor 410 on image 406 can be (e.g., via...) Figure 2 If the I / O device 214 in the viewport changes, the region of interest shown in the viewport 400 (and the position of the viewport 400 on the image 404) will change to correspond to the new position of the cursor 410 in the image 406.

[0072] The technical effects of the disclosed embodiments include improved efficiency by providing a user interface that allows for easy comparison of registered images. The technical effects of the disclosed embodiments include improved effectiveness by providing a user interface that allows for local analysis of two image sequences of different types. The technical effects of the disclosed embodiments include enabling the comparison of registered images with less user interaction, wherein the region of interest can be easily modified for comparison with the registered image. The technical effects of the disclosed embodiments include providing an embedded viewport that can be navigated when within another viewport. The technical effects of the disclosed embodiments include improved evaluation of registration results in medical imaging by enabling easy visual assessment of alignment and integration of information. The technical effects of the disclosed embodiments include making it easier to evaluate structures of interest. The technical effects of the disclosed embodiments include improved diagnostic accuracy, monitoring of disease progression, and evaluation of treatment responses across a variety of clinical applications.

[0073] Referring to the technology presented herein and protected by the claims, and applying it to physical objects and concrete examples of practical nature, which explicitly improves the present art, and therefore is not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to the end of this specification contains one or more elements designated as “component for [performing]…” or “step for [performing]…”, such elements are intended to be interpreted according to 35U.SC112(f). However, for any claim containing elements designated in any other manner, such elements are not intended to be interpreted according to 35U.SC112(f).

[0074] This disclosure also provides support for a system comprising: a memory encoding processor-executable routines; and a processing system including one or more processors and configured to access the memory and execute the processor-executable routines, wherein the processor-executable routines, when executed by the processing system, cause the processing system to: acquire a first three-dimensional (3D) medical image from a first medical imaging volume and acquire a second 3D medical image from a second medical imaging volume, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are globally registered with each other; receive a selection of a region of interest in the second 3D medical image; perform gradient domain fusion by mixing corresponding pixel intensities between the selected region of interest in the second 3D medical image and a region in the first 3D medical image corresponding to the region of interest to generate a mixed region of interest; and display the first 3D medical image in a first viewport and the mixed region of interest in a second viewport located at the region in the first 3D medical image corresponding to the region of interest on a user interface. In a first example of the system, the processor-executable routine, when executed by the processing system, causes the processing system to: holistically register the first medical imaging volume acquisition with the second medical imaging volume acquisition. In a second example of the system, the second example optionally includes the first example, wherein the first and second medical imaging volume acquisitions are acquired using different medical imaging modalities. In a third example of the system, the third example optionally includes one or both of the first and second examples, wherein the first and second medical imaging volume acquisitions are acquired using the same medical imaging modality. In a fourth example of the system, the fourth example optionally includes one or more, or each of, the first to the third examples, wherein the processor-executable routine, when executed by the processing system, causes the processing system to: display the second 3D medical image in a third viewport adjacent to the first viewport. In a fifth example of the system, optionally including one or more of the first to fourth examples, the processor-executable routine, when executed by the processing system, causes the processing system to: receive a first user input to display the blended region of interest in the second viewport located at the region in the first 3D medical image corresponding to the region of interest. In a sixth example of the system, optionally including one or more of the first to fifth examples, the processor-executable routine, when executed by the processing system, causes the processing system to: receive a second user input to hide the second viewport and, alternatively, display the region in the first 3D medical image corresponding to the region of interest.In a seventh example of the system, which optionally includes one or more or each of the first to sixth examples, the processor-executable routine, when executed by the processing system, causes the processing system to: receive user input to change to different 3D medical images acquired from the second medical imaging volume, wherein the region of interest is the same in the different 3D medical images; perform gradient domain fusion by mixing the corresponding pixel intensities between the region of interest in the different 3D medical images and the region corresponding to the region of interest in the first 3D medical image to generate different mixed regions of interest; and display the first 3D medical image on the user interface in a first viewport and display the different mixed regions of interest in a second viewport located at the region corresponding to the region of interest in the first 3D medical image. In an eighth example of the system, the eighth example optionally includes one or more or each of the first to seventh examples, wherein the processor executable routine, when executed by the processing system, causes the processing system to: receive user input that changes the position of the second viewport on the first 3D medical image; obtain different regions of interest in the second 3D medical image, the different regions of interest corresponding to the position of the second viewport on the first 3D medical image; perform gradient domain fusion using the blending of the corresponding pixel intensities between the different regions of interest in the second 3D medical image and different regions in the first 3D medical image corresponding to the different regions of interest, to generate different blended regions of interest; and display the first 3D medical image in the first viewport on the user interface, and display the different blended regions of interest in the second viewport located at the different regions in the first 3D medical image corresponding to the different regions of interest. In a ninth example of the system, optionally including one or more, or each of the first to eighth examples, the processor-executable routine, when executed by the processing system, causes the processing system to: receive another selection of different regions of interest in the second 3D medical image; perform gradient domain fusion by mixing the corresponding pixel intensities between the different regions of interest selected in the second 3D medical image and different regions in the first 3D medical image corresponding to the different regions of interest, to generate different mixed regions of interest; and change the display position of the different mixed regions in the second viewport of the first 3D medical image on the user interface to correspond to the different regions of interest. In a tenth example of the system, optionally including one or more, or each of the first to ninth examples, the second viewport includes a 3D cursor having 3D cursor settings for rendering the regions of interest.

[0075] This disclosure also provides support for a computer-implemented method comprising: acquiring a first three-dimensional (3D) medical image from a first medical imaging volume via a processing system including one or more processors, and acquiring a second 3D medical image from a second medical imaging volume, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are globally registered with each other; receiving at the processing system a selection of a region of interest (ROI) in the second 3D medical image; performing gradient domain fusion via the processing system using the blending of corresponding pixel intensities between the selected ROI in the second 3D medical image and a corresponding region in the first 3D medical image to generate a blended ROI; and displaying the first 3D medical image in a first viewport and the blended ROI in a second viewport on a user interface via the processing system, the second viewport being located at the region in the first 3D medical image corresponding to the ROI. In a first example of this computer-implemented method, the first medical imaging volume acquisition and the second medical imaging volume acquisition are acquired using different medical imaging modalities. In a second example of the computer-implemented method, the second example optionally includes the first example, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are acquired using different medical imaging modalities. In a third example of the computer-implemented method, the third example optionally includes one or both of the first and second examples, and the computer-implemented method further includes: receiving a first user input at the processing system to display the blended region of interest in the second viewport, the second viewport being located at the region in the first 3D medical image corresponding to the region of interest. In a fourth example of the computer-implemented method, the fourth example optionally includes one or more, or each of, the first to the third examples, and the computer-implemented method further includes: receiving a second user input at the processing system to hide the second viewport and, alternatively, display the region in the first 3D medical image corresponding to the region of interest.In a fifth example of the computer-implemented method, the fifth example optionally includes one or more or each of the first to fourth examples, the computer-implemented method further includes: receiving user input at the processing system to change to different 3D medical images acquired from the second medical imaging volume, wherein the region of interest is the same in the different 3D medical images; performing gradient domain fusion via the processing system using the mixing of the corresponding pixel intensities between the region of interest in the different 3D medical images and the region corresponding to the region of interest in the first 3D medical image to generate different mixed regions of interest; and displaying the first 3D medical image in a first viewport on the user interface via the processing system, and displaying the different mixed regions of interest in a second viewport located at the region corresponding to the region of interest in the first 3D medical image. In a sixth example of the computer-implemented method, which optionally includes one or more of the first to fifth examples, the computer-implemented method further includes: receiving user input at the processing system to change the position of the second viewport on the first 3D medical image; obtaining via the processing system different regions of interest in the second 3D medical image, the different regions of interest corresponding to the position of the second viewport on the first 3D medical image; performing gradient domain fusion via the processing system using the mixing of corresponding pixel intensities between the different regions of interest in the second 3D medical image and different regions in the first 3D medical image corresponding to the different regions of interest, to generate different mixed regions of interest; and displaying via the processing system the first 3D medical image in the first viewport on the user interface and the different mixed regions of interest in the second viewport located at the different regions in the first 3D medical image corresponding to the different regions of interest. In a seventh example of the computer-implemented method, which optionally includes one or more or each of the first to sixth examples, the computer-implemented method further includes: receiving at the processing system another selection of different regions of interest in the second 3D medical image; performing gradient domain fusion via the processing system using the blending of corresponding pixel intensities between the different regions of interest selected in the second 3D medical image and different regions in the first 3D medical image corresponding to the different regions of interest, to generate different blended regions of interest; and changing the display position of the different blended regions in the second viewport of the first 3D medical image on the user interface via the processing system to correspond to the different regions of interest.

[0076] This disclosure also provides support for a non-transitory computer-readable medium including processor-executable code that, when executed by a processing system including one or more processors, causes the processing system to: acquire a first three-dimensional (3D) medical image from a first medical imaging volume and acquire a second 3D medical image from a second medical imaging volume, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are globally registered with each other;

[0077] The system receives a selection of a region of interest (ROI) in the second 3D medical image; performs gradient domain fusion by mixing the corresponding pixel intensities between the selected ROI in the second 3D medical image and a region in the first 3D medical image corresponding to the ROI, to generate a mixed ROI; and displays the first 3D medical image in a first viewport and the mixed ROI in a second viewport located at the region in the first 3D medical image corresponding to the ROI on a user interface.

[0078] This written description uses examples to disclose the subject matter of the invention, including best practices, and also enables those skilled in the art to practice the subject matter, including making and using any device or system and performing any included methods. The patent scope of this subject matter is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. A system comprising: A memory that encodes processor-executable routines; and A processing system, comprising one or more processors and configured to access the memory and execute processor-executable routines, wherein the processor-executable routines, when executed by the processing system, cause the processing system to: A first three-dimensional (3D) medical image is obtained from a first medical imaging volume, and a second 3D medical image is obtained from a second medical imaging volume, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are globally registered with each other; Receive the selection of a region of interest in the second 3D medical image; Gradient domain fusion is performed by blending the corresponding pixel intensities between the region of interest selected in the second 3D medical image and the region in the first 3D medical image corresponding to the region of interest, in order to generate a blended region of interest. as well as The first 3D medical image is displayed in a first viewport on the user interface, and the blended region of interest is displayed in a second viewport located at the region in the first 3D medical image corresponding to the region of interest.

2. The system of claim 1, wherein the processor-executable routine, when executed by the processing system, causes the processing system to: integrally register the first medical imaging volume acquisition with the second medical imaging volume acquisition.

3. The system according to claim 1, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are acquired using different medical imaging modalities.

4. The system according to claim 1, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are acquired using the same medical imaging modality.

5. The system of claim 1, wherein the processor-executable routine, when executed by the processing system, causes the processing system to: display the second 3D medical image in a third viewport adjacent to the first viewport.

6. The system of claim 1, wherein the processor-executable routine, when executed by the processing system, causes the processing system to: receive a first user input to display the blended region of interest in a second viewport located at the region in the first 3D medical image corresponding to the region of interest.

7. The system of claim 6, wherein the processor-executable routine, when executed by the processing system, causes the processing system to: receive a second user input to hide the second viewport, and alternatively to display the region in the first 3D medical image corresponding to the region of interest.

8. The system of claim 1, wherein the processor-executable routine, when executed by the processing system, causes the processing system to: Receive user input to change to different 3D medical images acquired from the second medical imaging volume, wherein the region of interest is the same in the different 3D medical images; Gradient domain fusion is performed by mixing the corresponding pixel intensities between the regions of interest in the different 3D medical images and the regions corresponding to the regions of interest in the first 3D medical image to generate different mixed regions of interest. as well as The first 3D medical image is displayed in a first viewport on the user interface, and the different blended regions of interest are displayed in a second viewport located at the region in the first 3D medical image corresponding to the region of interest.

9. The system of claim 1, wherein the processor-executable routine, when executed by the processing system, causes the processing system to: Receive user input that changes the position of the second viewport on the first 3D medical image; Different regions of interest are obtained in the second 3D medical image, the different regions of interest corresponding to the position of the second viewport on the first 3D medical image; Gradient domain fusion is performed by mixing the corresponding pixel intensities between the different regions of interest in the second 3D medical image and the different regions in the first 3D medical image that correspond to the different regions of interest, in order to generate different mixed regions of interest; as well as The first 3D medical image is displayed in a first viewport on the user interface, and the different blended regions of interest are displayed in a second viewport located at the different regions in the first 3D medical image corresponding to the different regions of interest.

10. The system of claim 1, wherein the processor-executable routine, when executed by the processing system, causes the processing system to: Receive another selection of different regions of interest in the second 3D medical image; Gradient domain fusion is performed by blending the corresponding pixel intensities between the different regions of interest selected in the second 3D medical image and the different regions in the first 3D medical image corresponding to the different regions of interest, to generate different blended regions of interest; and On the user interface, the display position of the different blended regions in the second viewport of the first 3D medical image is changed to correspond to the different regions of interest.

11. The system of claim 1, wherein the second viewport includes a three-dimensional cursor having three-dimensional cursor settings for rendering the region of interest.

12. A computer-implemented method, the computer-implemented method comprising: A first three-dimensional (3D) medical image is acquired from a first medical imaging volume via a processing system including one or more processors, and a second 3D medical image is acquired from a second medical imaging volume, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are globally registered with each other; The processing system receives a selection of a region of interest in the second 3D medical image. The processing system performs gradient domain fusion by mixing the corresponding pixel intensities between the region of interest selected in the second 3D medical image and the region in the first 3D medical image corresponding to the region of interest, in order to generate a mixed region of interest. as well as The processing system displays the first 3D medical image in a first viewport on a user interface and the blended region of interest in a second viewport located in the region of the first 3D medical image corresponding to the region of interest.

13. A non-transitory computer-readable medium comprising processor-executable code, which, when executed by a processing system comprising one or more processors, causes the processing system to: A first three-dimensional (3D) medical image is obtained from a first medical imaging volume, and a second 3D medical image is obtained from a second medical imaging volume, wherein the first medical imaging volume acquisition and the second medical imaging volume acquisition are globally registered with each other; Receive the selection of a region of interest in the second 3D medical image; Gradient domain fusion is performed by blending the corresponding pixel intensities between the region of interest selected in the second 3D medical image and the region in the first 3D medical image corresponding to the region of interest, in order to generate a blended region of interest. as well as The first 3D medical image is displayed in a first viewport on the user interface, and the blended region of interest is displayed in a second viewport located at the region in the first 3D medical image corresponding to the region of interest.

Images