System and method for improving artifact correction in reconstructed 2d / 3d images

Abstract

An image processing system and method (1301) is provided for correcting artifacts within a three-dimensional (3D) volume (171) reconstructed from a plurality of two-dimensional (2D) projection images (101-109) of an object (132). The system and method (1301) is implemented on an imaging system (100) having a processing unit (150) operable to control the operation of radiation (140) and a detector (145) to generate a plurality of 2D projection images (101-109). The system (100) further comprises a memory (160) connected to the processing unit (150) and storing processor executable code that, when executed by the processing unit (150), operates to reconstruct the 3D volume (171) from the plurality of 2D projection images (101-109), the 3D volume (171) is defined in a pseudo-parallel geometry based on a zero angle (143) from the plurality of 2D projection images (101-109), where reconstructing the 3D volume (171) comprises reconstructing a 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) from the plurality of 2D projection images (101-109), and correcting the 3D virtual object (612) to form the 3D volume (171).

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to a continuation-in-part of U.S. application serial number 17 / 667,764, filed on February 9, 2022, entitled Fast And Low Memory Usage Convolutional Neural Network For Tomosynthesis Data Processing And Feature Detection, the entire contents of which are expressly incorporated herein by reference for all purposes. Technical Field

[0003] The subject matter disclosed herein relates generally to image reconstruction, and more particularly, to systems and methods for deep learning based image reconstruction. Background Art

[0004] Radiography is commonly used to look for abnormalities in an object of interest. A radiographic image represents a projection of an object, such as an organ of a patient. In a more specific, non-limiting example, the organ is a breast and the image is a mammographic image. Mammography has been used for decades to screen and diagnose breast cancer. Radiographic images are typically obtained by placing an object between a source that emits X-rays and an X-ray detector so that the X-rays that reach the detector have passed through the object. The radiographic image is then constructed from the data provided by the detector and represents the object projected on the detector in the direction of the X-rays.

[0005] In the case of mammography, an experienced radiologist can distinguish radioactive markers that indicate potential problems such as microcalcifications, masses or other opacities. However, in two-dimensional (2D) projection images, the overlap of tissues can hide lesions, and in any case their actual location in the subject of interest is unknown because the physician has no information about the depth of the radioactive marker in the projection direction.

[0006] Tomosynthesis is used to address these problems. In tomosynthesis, a three-dimensional (3D) representation of an organ is obtained as a series of consecutive slices. The slices are reconstructed from projections of the object of interest at various angles. To do this, the object of interest is generally placed between a source that emits X-rays and an X-ray detector, such as Figure 1 and Figure 2As shown schematically. The source and / or the detector are movable so that the projection direction of the object on the detector can be changed (for example, within an angular range of 30 degrees, etc.). Therefore, several projections of the object of interest are obtained at different angles, and a three-dimensional representation of the object can be reconstructed based on the several projections, usually by, for example, a reconstruction method. In order to determine and / or identify various views of the object, a coordinate system 2000 is defined by the imaging system 2001, so that various views of the imaged object can be selected along one or more of the XY plane 2002, the XZ plane 2004 or the YZ plane 2006 defined by the coordinate system 2000, for example, wherein the XY plane is a plane parallel to the XY plane (the same for XZ, YZ).

[0007] Although both standard mammography and tomosynthesis are currently used by radiologists, each technique has advantages. Standard mammography may form better images than tomosynthesis when imaging microcalcifications. Tomosynthesis is superior in imaging soft tissue lesions (e.g., jagged masses) because the reconstruction in tomosynthesis largely clears the tissue above and below the lesion and enables its localization within the organ.

[0008] Although radiologists can acquire both standard mammography images and tomosynthesis images to exploit the advantages of each technique, these imaging procedures are typically performed sequentially.

[0009] In recent years, digital breast tomosynthesis (DBT) has been shown to be an effective cancer detection technology, and CE-DBT technology is under development. DBT and / or CE-DBT use x-rays to create three-dimensional (3D) images of the breast. By taking multiple x-rays of each breast from multiple angles, a computer can generate a 3D image for detecting abnormalities. A key part of the DBT / CE-DBT process is image reconstruction, because image reconstruction directly affects the content of the data, and the radiologist will examine the data to determine any diagnosis. In order to reconstruct the image, the algorithm is designed, trained (if applicable to the algorithm used with artificial intelligence (AI) as its component), and used to reduce noise and minimize any artifacts present in the reconstructed volume, such as streaks, limited angle artifacts, overshoot and undershoot, etc. These algorithms can take various forms. In the past few years, these algorithms have most often tended to include one or more convolutional neural networks (CNNs) as their components, with specific examples being one or more of the following: FBP ConvNET disclosed in "Deep convolutional neural network for inverse problems in imaging" by Jin, Kyong Hwan et al. As published in Wang, Ge, Jong Chul Ye, and Bruno DeMan, “Deep learning for tomographic image reconstruction,” IEEE Transactions on Image Processing 26.9 (2017): 4509-4522., unrolled reconstruction.In "Deep learning reconstruction of digital breast tomosynthesis images for accurate breast density and patient-specific radiation dose estimation" by Jonas Teuwen, Nikita Moriakov, Christian Fedon, Marco Caballo, Ingrid Reiser, Pedrag Bakic, Eloy García, Oliver Diaz, Koen Michielsen, Ioannis Sechopoulos, Medical Image Analysis, Vol. 71, 2021, 102061, ISSN 1361-8415, Nature Machine Intelligence 2.12 (2020): 737-748., or learned primal dual reconstructions, the entire contents of which are expressly incorporated herein by reference for all purposes. The algorithm may include a CNN in the projection domain and / or a CNN in the volumetric domain. In the volume domain, CNNs can process each plane individually (i.e., 2D CNNs), each plane with adjacent planes or some context from adjacent planes (i.e., 2.5D CNNs) or use 3D CNNs (using local 3D neighborhoods and 3D features). In tomosynthesis, particularly in DBT / CE-DBT, CNNs can be used to overcome / remove artifacts from the constrained acquisition geometry present in DBT, i.e., the limited angular range over which projections are obtained, i.e., typically angular coverage from 15 to 50 degrees and / or sparse angular sampling, i.e., typically 9 to 25 projections are obtained over a limited angular range. Artifacts around a given object in a volume tend to be locally aligned with the back-projected lines that intersect at the object and are therefore very dependent on the acquisition geometry. Using e.g. Figure 3 The orientation of artifacts is position-dependent in 3D space, as shown in the typical beam geometry schematically represented in Figure 1. When the imaging system is operated to perform DBT by emitting a cone beam from a source toward a detector close to the patient's chest wall, the artifacts are mostly contained in the XZ plane, but their orientation depends on the position / depth within that plane. As we move away from the chest wall along the y-axis, the artifacts tend to align with the tilted plane defined by the beam path within the cone beam between the source and the detector. For specific reference, Figure 3, for example, an artifact at position p1 is positioned close to the chest wall that is roughly parallel to the XZ plane. However, an artifact at position p2 is set at a significant angular orientation relative to the XZ plane. At another artifact at position p3, these artifacts are rotated compared to their orientation at p1, but they are in the same XZ plane.

[0010] To correct for artifacts at positions p1, p2, and p3, a reconstruction algorithm containing a network (e.g., a CNN) to reduce artifacts must be trained with artifacts of all possible appearances and orientations to be able to adapt to the angular position of artifacts with respect to orientation variations, and must also be tested for variability in artifact appearance. Furthermore, while a simple 2D CNN operating in the XZ plane can effectively correct artifacts close to the chest wall where they are all contained in these XZ planes, such as at point p1, a 2D trained CNN will not be as effective at correcting artifacts far from the XZ plane, such as at point p2. This means that some CNN designs cannot be effectively used to correct artifacts in DBT projections, particularly some 2D and 2.5D CNNs.

[0011] In contrast, for computed tomography using a fan beam emitted from a radiation source, reconstruction artifacts that impair a given structure due to limited angles and sparse sampling tend to be almost perfectly located in the reconstruction plane and similar in each plane due to the narrow plane imaged by the fan beam. This enables a simpler training and testing process for algorithms and deep learning networks (e.g., CNNs), as well as handling artifacts with 2D CNNs.

[0012] Designing, training, and testing these CNNs for DBT is inconvenient due to the need to accommodate changes in the orientation of artifacts relative to the coordinate axes. The need to geometrically resolve the location and appearance of artifacts in tilted planes during the reconstruction process complicates the formulation, training, and testing of the CNNs required to ensure that artifact orientations are appropriately sampled for all geometries (e.g., 2D, 2.5D, and 3D) in both training and testing, which requires significant additional computational power and attendant complexity of the CNNs in order to accommodate these orientation changes. Omitting these adjustments during design, training, and testing also limits the ability to use CNNs trained on CT data or CT-like geometries to adapt them for use in DBT.

[0013] To help create more accurate volumes and improve artifact correction within reconstructions of 2D and 3D images, certain systems and methods have been developed, such as the systems and methods disclosed in U.S. Pat. No. 11,227,418, entitled Systems And Methods For Deep Learning-Based Image Reconstruction (the '418 patent), the entire contents of which are expressly incorporated herein by reference for all purposes. In the system and method, the system obtains a plurality of two-dimensional (2D) tomosynthesis projection images of an organ by: rotating an x-ray emitter to a plurality of orientations relative to the organ and emitting a first level of x-ray excitation from the emitter for each of the plurality of 2D tomosynthesis projection images; reconstructing a three-dimensional (3D) volume of the organ from the plurality of 2D tomosynthesis projection images; obtaining an x-ray image of the organ using a second level of x-ray excitation; generating a synthetic 2D image generation algorithm from the reconstructed 3D volume based on a similarity measure between the synthetic 2D image and the x-ray image; and deploying a model that instantiates the synthetic 2D image generation algorithm.

[0014] In addition, PCT patent application publication number WO2021 / 155123A1, entitled Systems And Methods For Artifact Reduction In Tomosynthesis With Deep Learning Image Processing (the '123 application), also provides an alternative solution to this problem that relies on tomosynthesis acquisition of a dataset and processes the dataset using a very specific type of deep learning-based reconstruction using standard single energy acquisition to reduce artifacts. The entire contents of the PCT patent application disclosure are expressly incorporated herein by reference for all purposes. Although the systems and methods of the '418 patent and the '123 application enhance the reconstruction of 3D volumes compared to existing reconstruction systems and processes, the disclosures of the '418 patent and the '123 application still require the use of complex deep learning algorithms that are trained to attenuate and / or remove artifacts in a manner that uses the natural geometry of the object reconstructed using a known conical X-ray beam shape to perform backprojection and forward projection operations. In this case, the artifact orientation depends on the X, Y, and Z positions of the voxels.

[0015] A side effect of this standard way of operating an imaging system is that artifacts due to the acquisition sequence and beam geometry vary significantly with spatial position in the volume, i.e., are non-translationally invariant. As previously described, the imaging system may be a radiographic or mammographic imaging system employed in other image reconstruction processes (which are denoted in this application as "conical geometry"), such as those employed in the '418 patent and the '123 application. Thus, many artifacts are not related to the spatial position in the volume. Figure 3 The reconstruction coordinate axes and planes shown are aligned, i.e., they tend to be locally aligned with tilted planes or even curved surfaces. This significantly complicates the sampling of training and test coverage for deep learning algorithms (e.g., 2D, 2.5D, or 3D CNN networks) in order to accommodate the requirement for non-translation invariance to accommodate artifacts. It also impairs the direct use of 2D or 2.5D networks, which can be easily used to reconstruct objects along the XZ plane, since artifacts are not contained in this plane, as well as any potential adaptation of CT-like methods.

[0016] Therefore, it is desirable to develop an improved system and method for image reconstruction that simplifies the beam geometry utilized in both the training and operation of a deep learning algorithm (e.g., a CNN) to improve the processing speed and performance of the deep learning algorithm, and combines this simplified geometry with an appropriate CNN design and training strategy. In particular, it is desirable to develop a system and method, such as a CNN, that provides correction of artifacts in a manner that does not require consideration of the non-translation invariance of the artifacts during the training and testing phases. It is also desirable to develop a system and method, such as a CNN, that provides correction of artifacts in the XZ plane of a 3D volume in a manner that employs geometric correction to allow the CNN to be constructed as a 2D CNN trained with a 2D image database to alleviate the database construction task. Summary of the invention

[0017] According to one aspect of an exemplary embodiment of the present disclosure, a method for correcting artifacts within a three-dimensional (3D) volume reconstructed from multiple two-dimensional (2D) projection images of an object includes the following steps: providing an imaging system having a radiation source, a detector positionable to receive radiation emitted from the radiation source and passing through an object positioned between the source and the detector, a display for presenting information to a user, a processing unit connected to the display and operable to control the operation of the radiation source and the detector to generate multiple 2D projection images of the object, and a memory operably connected to the processing unit and storing processor executable code for a reconstruction algorithm, the processor executable code being operable when executed by the processing unit to reconstruct a three-dimensional (3D) volume of the organ from the multiple 2D projection images, obtaining the multiple 2D projection images, selecting a zero angle from a range of angles for obtaining the multiple 2D projection images, reconstructing the 3D volume from the multiple 2D projection images, and defining the 3D volume in a pseudo-parallel geometry based on the zero angle from the multiple 2D projection images.

[0018] According to another aspect of an exemplary embodiment of the present disclosure, an imaging system includes a radiation source, a detector capable of being positioned to receive radiation emitted from the radiation source and passing through an object positioned between the source and the detector, a display for presenting information to a user, a processing unit connected to the display and capable of operating to control the operation of the radiation source and the detector to generate multiple 2D projection images of the object, and a memory operably connected to the processing unit and storing processor executable code for a reconstruction algorithm, the processor executable code being capable of operating when executed by the processing unit to reconstruct a three-dimensional (3D) volume of an organ based on the multiple 2D projection images, wherein the memory includes processor executable code for reconstructing a 3D volume based on the multiple 2D projection images, defining the 3D volume in a pseudo-parallel geometry based on a zero angle from the multiple 2D projection images, wherein the step of reconstructing the 3D volume includes reconstructing a 3D virtual object defined in the pseudo-parallel geometry based on the zero angle based on the multiple 2D projection images; and correcting the 3D virtual object to form the 3D volume.

[0019] These and other exemplary aspects, features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings illustrate the best mode presently contemplated of practicing the invention.

[0021] In the attached picture:

[0022] Figure 1 is a schematic diagram of an imaging device used for radiographic image reconstruction and the associated coordinate system represented thereon.

[0023] Figure 2 yes Figure 1 Schematic diagram of an imaging system and the associated coordinate planes defined by the coordinate system.

[0024] Figure 3 is a schematic diagram of an example geometry for correction of artifacts of a reconstructed object in a plane along the XZ axis.

[0025] Figure 4 is a schematic diagram of a radiographic imaging system employing an improved pseudo-parallel geometry reconstruction and correction processing system according to an exemplary embodiment of the present disclosure.

[0026] Figure 5 yes Figure 4 Schematic diagram of the movement of the imaging system.

[0027] FIG. 6A to FIG. 6B are schematic diagrams of the form of a virtual object before and after applying a pseudo-parallel geometry reconstruction and correction processing system according to an embodiment of the present disclosure.

[0028] Figure 7 is a schematic diagram of a first embodiment of a training process of a pseudo-parallel geometry reconstruction and correction processing system according to an embodiment of the present disclosure.

[0029] Figure 8 is a schematic diagram of a second embodiment of a training process of a pseudo-parallel geometry reconstruction and correction processing system according to an embodiment of the present disclosure.

[0030] Fig. 9 is a schematic diagram of a first embodiment of a method of operating a pseudo-parallel geometry reconstruction and correction processing system according to an embodiment of the present disclosure.

[0031] Fig.10 is a schematic diagram of a second embodiment of an operating method of a pseudo-parallel geometry reconstruction and correction processing system according to an embodiment of the present disclosure.

[0032] Fig.11 is used as an exemplary embodiment according to the present disclosure Fig. 9 or Fig.10 Schematic diagram of the architecture of a first particular exemplary embodiment of a convolutional neural network that is part of a pseudo-parallel geometry reconstruction and correction processing system.

[0033] Fig.12 is a schematic diagram of a third embodiment of a method of operating a pseudo-parallel geometry reconstruction and correction processing system according to an embodiment of the present disclosure.

[0034] Fig.13 is used as an exemplary embodiment according to the present disclosure Fig.12 Schematic diagram of the architecture of a second particular exemplary embodiment of a primal-dual reconstruction network that is part of a pseudo-parallel geometry reconstruction and correction processing system.

[0035] Fig.14 is a method for implementing an exemplary embodiment of the present disclosure Fig. 9 or Fig.10 A processor graph for a method.

[0036] Fig.15 is a schematic diagram of an exemplary tomography system according to an exemplary embodiment of the present disclosure.

[0037] Fig.16 is a schematic diagram of an exemplary tomography system according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] One or more specific embodiments will be described below. In order to provide a concise description of these embodiments, all features of an actual implementation may not be described in the specification. It should be understood that, as in any engineering or design project, in the development of any such actual implementation, numerous implementation-specific decisions must be made to achieve the developer's specific goals, such as complying with system-related and business-related constraints that may differ between implementations. In addition, it should be understood that such development efforts may be complex and time-consuming, but are still routine tasks of design, fabrication, and manufacturing for ordinary technicians who benefit from this disclosure.

[0039] In the following detailed description, reference is made to the accompanying drawings which form a part thereof, and in which specific examples which can be practiced are shown by way of illustration. These examples are described in sufficient detail to enable those skilled in the art to practice the subject matter, and it should be understood that other examples may be utilized. Therefore, the following detailed description is provided to describe exemplary implementations and is not to be construed as limiting the scope of the subject matter described in the present disclosure. Certain features from the different aspects described below may be combined to form new aspects of the subject matter discussed below.

[0040] When introducing elements of various embodiments of the present disclosure, the words "a," "an," "the," and "said" are intended to mean that there are one or more of the elements. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements besides the listed elements.

[0041] As used herein, the terms "system," "unit," "module," and the like may include hardware and / or software systems that operate to perform one or more functions. For example, a module, unit, or system may include a computer processor, a controller, and / or other logic-based devices that perform operations based on instructions stored on a tangible and non-transitory computer-readable storage medium, such as a computer memory. Alternatively, a module, unit, or system may include a hard-wired device that performs operations based on the hard-wired logic of the device. The various modules, units, engines, and / or systems shown in the accompanying drawings may represent hardware that operates based on software or hard-wired instructions, software that instructs hardware to perform operations, or a combination thereof.

[0042] As used herein, the term "projection" or "projection image" refers to an image obtained by emitting x-rays from a specific angle or viewing angle. A projection can be considered a specific example of mapping, in which a collection of projection images are captured from different angles of a 3D object. The reconstruction algorithm can then map or combine / fuse them to reconstruct the volume and / or create a synthetic 2D image. Each projection image is typically captured relative to a central projection (e.g., a baseline projection, a straight line projection, a zero-angle projection, etc.). The resulting image from the projection is either a 3D reconstructed image that is substantially the same as the original 3D object, or a 2D synthetic image that combines each projection together and benefits from the information in each view, and can rely on the form of the 3D reconstruction.

[0043] As used herein, the term "acquisition geometry" is a specific series of positions of an x-ray source and / or detector relative to a 3D object (eg, a breast) to obtain a series of 2D projections.

[0044] As used herein, the term "central projection" is a projection within a series of 2D projections obtained at or closest to zero orientation of the x-ray source relative to the detector, ie, where the x-ray source is approximately orthogonal to the detector.

[0045] As used herein, the term "reference angle" is the angle at which the x-ray source is positioned relative to the detector for the central projection and from which the angular position of each additional projection obtained within a series of 2D projections is calculated across the entire acquisition geometry.

[0046] Although certain examples are described below in the context of a medical or healthcare workplace, other examples may be implemented outside of a medical environment.

[0047] In many different applications, deep learning techniques have utilized learning methods that allow machines to be given raw data and determine the representations needed for data classification or data repair. Deep learning uses a back-propagation algorithm that is used to change the internal parameters of the deep learning machine (e.g., node weights) to determine the structure in the data set. Deep learning machines can utilize a variety of multi-layer architectures and algorithms. For example, while machine learning involves identifying features to be used to train the network, deep learning processes raw data to identify features of interest without external identification.

[0048] Deep learning in a neural network environment consists of many interconnected nodes called neurons. Input neurons activated by external sources activate other neurons based on connections to those other neurons controlled by machine parameters. Neural networks function in a certain way based on their own parameters. Learning improves the machine parameters, and more broadly, the connections between neurons in the network, so that the neural network functions in the desired way.

[0049] Deep learning operates on the understanding that many data sets consist of high-level features that in turn consist of lower-level features. For example, when examining an image, instead of looking for objects, it is more efficient to look for edges, which form motifs, which form parts, which form the objects you are looking for. These hierarchies of features can be seen in many different forms of data, such as speech and text.

[0050] Deep learning has been applied to inverse problems in imaging to improve image quality (such as denoising, deblurring) or to generate desired images (such as in super-resolution or 3D reconstruction). The same deep learning principles are applied, but instead of delivering labels (as for classification tasks), the network delivers processed images. Such a network is for example the UNet architecture.

[0051] One example use of deep learning technology in the medical field is a radiographic system. Radiography and in particular mammography are used to screen for breast cancer and other abnormalities. Traditionally, mammograms have been formed on x-ray films. However, flat-panel digital imagers that collect radiographic images or mammograms in digital form have recently been introduced, which facilitates the analysis and storage of collected images. In addition, a large amount of attention and technical development has been devoted to obtaining three-dimensional images of the breast. Three-dimensional (3D) mammography is also called digital breast tomosynthesis (DBT). Two-dimensional (2D) mammography is full-field digital mammography, and 2D synthetic mammography generates 2D pictures derived from 3D data by combining DBT volumes and / or individual enhanced slices (e.g., 1 mm, 2 mm, etc.) of the original projection. The breast tomosynthesis system reconstructs a 3D image volume based on a series of two-dimensional (2D) projection images, each of which is obtained at a different angular displacement of the x-ray source. The reconstructed 3D image volume is typically presented as a plurality of image data slices that are geometrically reconstructed on planes parallel to the imaging detector in a reference position.

[0052] A deep learning machine can learn the neuron weights for a task in a fully supervised manner if a dataset of inputs and outputs is provided. For example, in one embodiment, the input may be a volume reconstructed with artifacts, and the output may be a volume without artifacts, such as when a deep learning net (such as FBPConvnet) is employed. In other embodiments, the input may be the projections themselves, and the output is a volume without artifacts, such as when a learned primal-dual reconstruction is employed. Deep learning networks may also be trained in a semi-supervised or unsupervised manner.

[0053] Once the desired neural network behavior has been achieved (e.g., the machine is trained to operate according to specified performance, etc.), the machine can be deployed for use (e.g., the machine is tested using "real" data, etc.). During operation, the neural network outputs can be confirmed or rejected (e.g., by expert users, expert systems, reference databases, etc.) to continue to improve the neural network behavior. Thus, the parameters that determine the neural network behavior can be updated based on ongoing interactions.

[0054] Example systems and associated methods

[0055] Figure 4An example radiographic imaging system, such as a mammography imaging system 100, for obtaining one or more images of an object of interest, such as the systems and methods disclosed in U.S. Patent No. 11,227,418, entitled Systems And Methods For Deep Learning-Based Image Reconstruction (the '418 patent), the entire contents of which are expressly incorporated herein by reference for all purposes, is illustrated. The example system 100 includes a radiation source, such as an x-ray beam source 140 facing a detector 145. The x-ray beam source or emitter 140 and the detector 145 are connected by an arm 144. An object of interest 132 can be placed between the detector 145 and the source 140. In Figure 1 In the example of FIG. 1 , an x-ray source 140 moves in an arc over a single detector 145. Multiple positions of the detector 145 and x-ray sources 140 ′ and 140 ″ along the arc (see dashed lines) are shown in dashed / solid lines and in perspective partial views. Figure 5 In the arrangement shown in the example of , the detector 145 is fixed in the position shown, and only the x-ray source 140 moves. Angle α is a projection angle encompassed by the zero orientation, i.e., the orientation in which the x-ray source 140 is approximately orthogonal to the detector 145, and any other orientations such as 141 and 142. Using this configuration, multiple views of breast (e.g., object of interest 132) tissue can be acquired via at least one x-ray source 140.

[0056] Still see Figure 4 , a partial perspective view of the imaging system 100 including the detector 145 and the x-ray source 140 is shown on the left. Different positions of the x-ray source 140, 140' and 140" are generally depicted to show the movement of the x-ray source 140. There are nine different projection views 101, 102, 103, 104, 106, 107, 108, 109, including the zero or center projection 105, indicated as straight lines, which are taken within a range of angles that are all directed toward the center of the detector 145. Alternatively, in certain exemplary embodiments, the radiographic imaging system 100 may be formed as a cone beam computed tomography (CBCT) system.

[0057] The angular range over which the set of 2D projection images is obtained may be predetermined for a given imaging process, or may be set by a user via the user interface 180, as well as the angular spacing of the individual 2D projection images over the angular range between the first projection image and the last projection image. The 2D projection image that is closest to the zero orientation or orientation of the source 140 approximately orthogonal to the detector 145 is named by approximation as the center projection or zero projection. However, for purposes of the present disclosure, the zero orientation or zero angle may be defined as the angle at which the zero projection is obtained, or as another angle within the angular range at which the 2D projection images are obtained, selected by the user via the user interface 180 or set by the system 100 and orthogonal to the detector 145, such as the angle of the plane XZ, as Figure 3 as shown in .

[0058] The object of interest 132 shown in the display unit 170 is a breast compressed by a compression paddle 133 and a cover (not shown) disposed on the detector 145, which helps ensure uniform compression and immobilization of the breast during radiation exposure to obtain optimal image quality. The breast 132 includes point-like objects 131, such as calcifications, which are positioned in a zero orientation 143 perpendicular to the plane of the detector 145. For example, a user can examine calcifications or other clinically relevant structures for diagnosis.

[0059] The detector 145 and the x-ray source 140 form an acquisition unit, which is connected to a processing unit 150 via a data acquisition line 155. The processing unit 150 comprises a memory unit 160, which may be connected via an archiving line 165, for example.

[0060] A user, such as a medical professional, may input control signals via the user interface 180. Such signals are transmitted from the user interface 180 to the processing unit 150 via a signal line 185. By using the example system 100, an enhanced 2D projection image displayed as a 2D mammogram may be obtained. Based on the high quality image, a radiologist and / or other user may identify clinical signs associated with breast screening. In addition, previously stored 2D mammograms may be displayed for comparison with new 2D projection images acquired by tomosynthesis. The tomosynthesis images may be viewed and archived, and 3D markers may be provided by a CAD system, a user, or the like. A height map of a point-like object or other object obtained from the image data may be combined with height information provided by a CAD system based on a 3D marker indicated by a user through a 3D review, or the like. In addition, the user may decide to archive the 2D full volume image and / or other images. Alternatively or in addition, the saving and storage of images may be performed automatically.

[0061] In some examples, the memory unit 160 can be integrated and / or separate or remote from the processing unit 150. The memory unit 160 allows for storage of data such as 2D enhanced projection images and / or tomosynthesis 3D images. Generally, the memory unit 160 can include a computer readable medium such as a hard disk or CD-ROM, a floppy disk, a ROM / RAM memory, a DVD, a digital source such as a network or the Internet, etc. The processing unit 150 is configured to execute program instructions stored in the memory unit 160, which program instructions cause the computer to perform the methods and / or implement the systems disclosed and described herein. One technical effect of performing one or more methods and / or implementing one or more systems is that less x-ray sources can be used because the enhanced 2D projection images can replace known 2D mammograms, which are typically obtained using additional x-ray exposures to obtain high quality images.

[0062] As the emitter 140 rotates around the organ, the emitter 140 may further include beam shaping (not depicted) to direct the X-rays through the organ to the detector 145. For example, the emitter 140 can be rotated around the organ 132 to a plurality of orientations relative to the organ 132. In one example, the emitter 140 can be rotated a total arc of 30 degrees relative to the organ 132, or can be rotated 30 degrees in each direction (clockwise and counterclockwise) relative to the organ 132. It should be appreciated that these rotation arcs are merely examples and are not intended to limit the range of angle measurements that can be used.

[0063] It should be appreciated that the emitter 140 can be positioned at a location or orthogonal to one or both of the organ 132 and the detector 145. In this orthogonal or central location, in one exemplary operating mode of the system 100, full field digital mammography (FFDM) can be acquired, particularly when a single emitter 140 and detector 145 are used to acquire both FFDM images and digital breast tomosynthesis (DBT) projection images in an exemplary configuration. FFDM images (also known as digital mammography images) allow the full field of view of an object (e.g., a breast, etc.) to be imaged, rather than a small field of view (FOV) within the object. The digital detector 145 allows the full field of view of the target object or organ 132 to be imaged, rather than having to move and combine multiple images representing portions of the organ 132.

[0064] The DBT projection images are acquired at various angles of the emitter 140 with respect to the organ 132. Various imaging workflows may be implemented using the example system 100. In one example, if FFDM images are acquired, the FFDM images are acquired at a position orthogonal to the organ, and DBT projection images are acquired at various angles relative to the organ 132, including DBT projection images acquired at a position of the emitter 140 that is orthogonal to the organ 132. For example, during reconstruction, the DBT projection images are used to reconstruct a 3D volume of the organ. In an exemplary embodiment, the DBT volume is reconstructed from the acquired DBT projection images, and / or the system 100 may acquire both DBT projection images and FFDM and reconstruct the DBT volume only from the DBT projection images.

[0065] Thus, by rotating the x-ray emitter 140 to a plurality of orientations relative to the organ 132 and emitting x-ray excitation from the emitter 140 for each of a plurality of projection images, a plurality of 2D tomosynthesis projection images of the organ 132 are obtained, thereby enabling imaging of various different examples of the manner in which the system 100, the organ, and / or other object of interest 132 operates, such as Figure 4 and Figure 5 A 3D volume 171 of the organ 132 is then reconstructed from the plurality of tomosynthesis projection images 101 - 109 , which may be used for direct presentation on a display 170 and / or for generating selected 2D views of portions of the 3D volume 171 .

[0066] In addition to the mammography imaging device or system 100, the processing unit 150 / AI 152 / CNN 154 ( Figure 4 ) provides a pseudo-parallel geometry reconstruction system and method 1301 ( Fig. 9 and Fig.10 ) can also be implemented on the digital X-ray radiography tomosynthesis system 1100, 1200, such as Fig.15 and Fig.16 As shown in the exemplary embodiment of . Fig.16 11. A table-top acquisition configuration is illustrated having an X-ray source 1102 attached to a structure 1160 and an X-ray detector 1104 positioned within a table 1116 under a tabletop 1118 (similar in function to a Figure 4 and Figure 5 Detector 145), and Fig.15A wall mount configuration is illustrated having an X-ray source 1202 attached to a structure 1260 and an X-ray detector 1204 attached to a wall mount 1216. The digital X-ray radiography tomosynthesis radiography system 1100, 1200 includes X-ray sources 1102, 1202 that subject an examined patient 1106, 1206 to radiation in the form of X-ray beams 1108, 1208. The X-ray beams 1108, 1208 are emitted by the X-ray sources 1102, 1202 and impinge on the examined patient 1106, 1206. A portion of the radiation from the X-ray beams 1108, 1208 passes through or around the patient and impinges on the detectors 1104, 1204.

[0067] In an exemplary embodiment, the X-ray source 100, 1102, 1202 may be an X-ray tube, and the object or patient 132, 1106, 1206 may be a human patient, an animal patient, a test phantom, and / or other inanimate object being examined. The patient 1106, 1206 is placed between the X-ray source 1102, 1202 and the detector 1104, 1204. During tomosynthesis acquisition, the X-ray source 1102, 1202 is directed along the X-ray source 1102, 1202. Fig.16 and Fig.15 The planes 1110, 1210 shown travel and rotate synchronously so that the X-ray beams 1108, 1208 are always directed toward the detectors 1104, 1204 during acquisition. As mentioned above, the X-ray source 1102, 1202 typically moves along a single plane 1110, 1210 that is parallel to the planes 1112, 1212 of the detectors 1104, 1204, although it can move out of a single plane that is substantially parallel to the detectors 1104, 1204. When acquiring radiographs, the detectors 1104, 1204 are maintained at fixed positions. The detectors 1104, 1204 acquire multiple discrete projection radiographs of the patient 1106, 1206 at discrete locations along the path 1110, 1112 of the X-ray source 1102, 1202. After acquiring the projection image data from the projection radiographs, the application software can reconstruct the slice images.

[0068] The digital X-ray radiography tomosynthesis imaging process includes a series of low-dose exposures during a single scan of an X-ray source 1102, 1202, which is moved within a limited angular range 1114, 1214 (scan angle) by arc rotation and / or linear translation of the X-ray source 1102, 1202 and focused toward a fixed detector 1104, 1204. The X-ray source 1102, 1202 delivers multiple exposures from multiple projection angles during a single scan. The scan angle 1114, 1214 is the angle from the first projection exposure to the last projection exposure. The scan angle 1114, 1214 is typically in the range of 20 degrees to 60 degrees.

[0069] In an exemplary embodiment, the detectors 1104, 1204 may include a plurality of detector elements, typically corresponding to pixels, which sense the intensity of the X-rays passing through and around the patient and generate electrical signals representing the intensity of the incident X-ray beam at each detector element. These electrical signals are collected and processed to reconstruct a 3D volumetric image of the patient's anatomical structure. Depending on the X-ray attenuation and absorption of the intermediate structures, the intensity of the X-rays striking each detector element will vary.

[0070] Fig.15 and Fig.16 Also schematically illustrated is a computer workstation 1130, 1230 of a digital tomosynthesis imaging system 1120, 1220 coupled to the digital X-ray radiography tomosynthesis system 1100, 1200, providing a user interface 1140, 1240 for selecting at least one reconstruction, dose and / or acquisition parameter for a digital X-ray radiography tomosynthesis acquisition as described herein.

[0071] The digital tomosynthesis imaging system 1120, 1220 can be used to acquire and process projection image data and reconstruct a volumetric image or a three-dimensional (3D) image representing the imaged patient. The digital tomosynthesis imaging system 1120, 1220 is designed to acquire projection image data and process the image data for observation and analysis.

[0072] The computer workstation 1130, 1230 includes at least one imaging system / computer 1132, 1232 having a controller 1134, 1234, a processor 1136, 1236, a memory 1138, 1238, and a user interface 1140, 1240. The processor 1136, 1236 may be coupled to the controller 1134, 1234, the memory 1138, 1238, and the user interface 1140, 1240. A user interacts with the computer workstation 1130, 1230 for controlling the operation of the digital X-ray radiography tomosynthesis system 1100, 1200. In an exemplary embodiment, the memory 1138, 1238 may be in the form of a memory device, a memory board, a data storage device, or any other storage device known in the art.

[0073] The digital tomosynthesis imaging system 1120, 1220 is controlled by a controller 1134, 1234, which can provide both power signals and control signals for the digital tomosynthesis examination sequence, including the positioning of the x-ray source relative to the patient and the detector. The controller 1134, 1234 can command the acquisition of the signals generated in the detector. The controller 1134, 1234 can also perform various signal processing and filtering functions, such as initial adjustment for dynamic range, digital image data interleaving, etc. In general, the controller 1134, 1234 commands the operation of the digital tomosynthesis imaging system 1120, 1220 to execute the examination protocol and process the acquired data. In an exemplary embodiment, the controller 1134, 1234 receives instructions from the computer 1132, 1232. In an exemplary embodiment, the controller 1134, 1234 can be part of the digital tomosynthesis imaging system 1120, 1220, rather than part of the computer workstation 1130, 1230.

[0074] In an exemplary embodiment, the computer 1132, 1232 includes or is coupled to a user interface 1140, 1240 for interaction by a user to select and / or change clinically relevant parameters such as dose parameters, slice placement (reconstruction settings) parameters, and acquisition parameters. In an exemplary embodiment, the operation of the digital X-ray radiography tomosynthesis system 1100, 1200 is achieved using a software program or algorithm downloaded on or integrated into the computer 1132, 1232.

[0075] In an exemplary embodiment, the user interface 1140, 1240 is a visual interface that can be configured to include a plurality of predefined tools that will allow the user to view, select, and edit reconstruction parameters (settings); view and select dose parameters; and view, select, and edit tomosynthesis acquisition parameters. The plurality of predefined tools may include a tomosynthesis preference editing tool, a "scout" acquisition editing tool, a tomosynthesis acquisition editing tool, and a plurality of slice image processing editing tools. The user interface 1140, 1240 also allows the user to view the reconstructed image.

[0076] In an exemplary embodiment, the user interface 1140, 1240 may include at least one input device for inputting and / or selecting information of a plurality of predefined tools displayed on a display of the user interface 1140, 1240. In an exemplary embodiment, the at least one input device may be in the form of a touch screen display, a mouse, a keyboard, at least one button, or any other input device known in the art.

[0077] The processor 1136, 1236 receives the projection data from the detector 1104, 1204 and performs one or more image analyses through one or more image processing operations, including image analysis by a computer-aided detection (CAD) system, etc. The processing unit / processor 1136, 1236 exemplarily operates to create a 3D volume using the projection data / projections, and analyze slices of the 3D volume to determine the location of lesions and other masses present within the 3D volume, and store the 3D volume in a mass storage device 1138, 1238, where, as non-limiting examples, the mass storage device 1138, 1238 may include a hard drive, a floppy disk drive, a compact disk read / write (CD-R / W) drive, a digital versatile disk (DVD) drive, a flash drive, and / or a solid-state storage device. As used herein, the term computer is not limited to those integrated circuits known in the art as computers, but refers broadly to processors, microcontrollers, microcomputers, programmable logic controllers, application specific integrated circuits, and any other programmable circuits, and these terms are used interchangeably herein. It should be understood that any one or more of the processors and / or controllers described herein may be performed by or in conjunction with the processing unit / processor 1136, 1236, for example by executing computer-readable code stored on a computer-readable medium accessible and executable by the processing unit / processor 1136, 1236. For example, the computer / processing unit / processor 1136, 1236 may include a processor configured to execute machine-readable instructions stored in a mass storage device 1138, 1238, which may be a non-transitory memory. The processing unit / processor / computer 1136, 1236 may be single-core or multi-core, and the program executed thereon may be configured for parallel or distributed processing. In some embodiments, the processing unit 1136, 1236 may optionally include separate components distributed in two or more devices, which may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the processing units 1136, 1236 may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration. According to other embodiments, the processing unit / computer 1136, 1236 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. According to other embodiments, the processing unit / computer 36 may include multiple electronic components capable of performing processing functions. For example, the processing unit / computer 1136, 1236 may include two or more electronic components selected from a list of electronic components, including: a central processing unit, a digital signal processor, a field programmable gate array, and a graphics board.In still further embodiments, the processing unit / computer 1136, 1236 may be configured as a graphics processing unit (GPU) including a parallel computing architecture and parallel processing capabilities.

[0078] Reference again Figure 4 In the process of reconstruction of 3D planes / slices and / or volumes 171, processing unit 150 includes an artificial intelligence or deep learning network 152, such as CNN 154, which has used a pseudo-parallel geometric reconstruction system and process 1301 ( Fig. 9 and Fig.10 ) are trained to the projection images / projection sets 101-109 in the formation of the reconstructed 3D planes / slices and / or volumes 171, particularly with respect to the reconstruction of the 3D XZ planes / slices 171. In an exemplary embodiment, the processing unit 150 may access instructions stored in the memory 160 for operation of the AI ​​152 / CNN 154 to perform a pseudo-parallel geometry reconstruction process or method 1301 ( Fig. 9 and Fig.10 ).

[0079] When training and / or preparing the AI ​​152 / CNN 154 to be instantiated on the system 100 for providing pseudo-parallel geometry in the reconstruction of 3D planes / slices and / or volumes 171, the system and process 1301, in one exemplary embodiment, the most straightforward way to train the deep learning network 152 (e.g., CNN 154) in the reconstruction pipeline (i.e., for correcting and reconstructing the 3D volume 171 from the plurality of 2D images 101-109) is by utilizing a fully supervised training model. However, embodiments of the present disclosure are not limited to fully supervised training models, as training may also be performed using partially or semi-supervised training models or unsupervised training models, and we describe the fully supervised training model only as an exemplary embodiment of a suitable training model for a deep learning network. In a fully supervised training setting of a deep learning network for 3D volume reconstruction of DBT projection images, the training may be based on the use of simulated acquisitions of digital objects (e.g., digital phantoms) that are provided as input to the network. The digital object can be a simulated breast anatomy, such as the same or similar anatomy as those utilized in the Virtual Imaging Clinical Trial for Regulatory Evaluation (VICTRE trial), which used 2,986 computer simulated images to compare digital mammography and digital breast tomosynthesis, or other breast / breast anatomy imaged using other modalities such as MRI or breast CT, or a combination of some or all of these images. The simulated acquisition used as input during the training of the deep learning network (e.g., CNN) can be by means of a digital twin technology that models the X-ray imaging systems compared in the VICTRE trial. Using such an object, the training database construction can simulate the acquisition of multiple DBT projection images at a given source position. Pairs for supervised learning can be formed from a collection of simulated projections as input, and the original digital object is converted or corrected to a virtual object in a pseudo-parallel geometry as a ground truth for comparison with the output from the pseudo-parallel geometry correction and reconstruction AI or deep learning network 152 / CNN 154. By repeating this process on a database containing a plurality of different digital objects, such as synthetic digital object phantoms derived from one or more of a CT or MRI scan of an organ of interest, a breast CT or breast MRI scan, and a chest CT or chest MRI scan, a database of 2D image pairs for input and associated real-world conditions is created that can be used to train a reconstruction algorithm, such as an AI 152 / CNN 154, in a supervised manner for operating a pseudo-parallel geometry correction and reconstruction system and process 1301 as part of the reconstruction of a 3D location / slice and / or volume 171 employed by the system 100.It is important to note here that, as explained in this specification, the underlying real-world object must be converted or corrected into a virtual object in a pseudo-parallel geometry so that its geometry matches that of the pseudo-parallel geometry correction and / or reconstruction AI or deep learning network 152 / CNN 154.

[0080] Another way to define a training database for training the AI ​​152 / CNN 154 for use in the pseudo-parallel geometry correction and reconstruction system and process 1301 is to use a volume obtained by reconstructing another set of simulated DBT projections that simulates a desired acquisition sequence (more views, more doses, wider angular range, etc.) as the ground truth associated with the simulated projections used to reconstruct the volume. Again, in such a training system or process, the volume defining the ground truth must be reconstructed in the pseudo-parallel geometry or corrected to match that geometry.

[0081] Utilizing any of the aforementioned training pairs or any other suitable training pair or method, the processing unit 150 / AI 152 / CNN 154 that forms and / or employs the pseudo-parallel geometry correction and / or reconstruction system and method 1301 disclosed herein enables greatly simplified training and evaluation of deep learning networks (e.g., CNNs) for correction and reconstruction of 3D planes / slices and / or volumes 171. More specifically, the pseudo-parallel geometry correction and reconstruction system and process 1301 performed by the processing unit 150 / AI 152 / CNN 154 enables downstream use of 2D or 2.5D trained networks (CNNs) for simplified reconstruction of 3D volumes 171 from 2D projection images 101-109 and / or artifact removal within the XZ and XY planes of the reconstructed 3D slices and / or volumes 171. In particular, the pseudo-parallel geometry correction system and method 1301 disclosed herein enables the processing unit 150 / AI 152 / CNN 154 to geometrically correct the reconstructed virtual object and / or volume reconstructed from the projections 101-109 and any artifacts therein, as defined by the system 100, prior to being used as input to a separate volume reconstruction AI / deep learning network (e.g., CNN) for reconstruction of the corrected 3D slices and / or volumes 171. The pseudo-parallel geometry correction system and method 1301 thereby eliminates the need to correct the angular orientation of artifacts relative to the XZ or XY plane by the 3D reconstruction CNN, thereby eliminating the need for simulation of these corrections in the training and testing coverage of the reconstruction deep learning algorithm (e.g., 2D, 2.5D or 3D CNN network). Thus, the pseudo-parallel geometry system and method 1301 disclosed herein enables the direct use of a 2D or 2.5D reconstruction CNN to reconstruct the corrected 3D slices and / or volumes 171 in the XZ and / or XY planes. In particular, utilizing the simplification of information provided by the pseudo-parallel geometry correction system and process 1301 as input to the 2D or 2.5D reconstruction CNN 1320 (i.e., projections 101-109 and / or virtual objects, slices / planes 1302 or volumes 1310), the 2D / 2.5D reconstruction CNN 1320 can be trained and used to construct corrected 3D slices and / or volumes 171 along the XZ axis using a 2D image training database. This can also alleviate the complexity of the training database construction task, since 2D image databases / training data sets are easier to collect. Alternatively, or in conjunction or combination with the above, the processing unit 150 / AI 152 / CNN 154 can be trained from a 2D image database associated with or obtained from a 1D projection.

[0082] Reference now FIG. 6A to FIG. 6B, illustrates a representation of the operation of pseudo-parallel geometric correction and / or reconstruction to be employed by AI 152 / CNN 154, which operation is not discussed in detail here, but is disclosed in Nett, Brian E., Shuai Leng, and Guang-Hong Chen, "Planar tomosynthesis reconstruction in a parallel-beam framework via virtual object reconstruction," Medical Imaging 2007: Physics of Medical Imaging. Vol. 6510. SPIE, 2007 (Nett), which is incorporated herein by reference in its entirety for all purposes. In a manner similar to Figure 3 of Fig. 6A , there is shown a digital or simulated parallelepiped object 600 including a plurality of artifacts 602, 604, 606. When a simulated projection image of the digital object 600 is obtained at points s1, s2, and s3 of a radiation source 140, the artifacts 602, 604, 606 are arranged or oriented at various angles relative to an XZ plane 608 defined by a coordinate system 610 for a simulated imaging process from which a projection image of the digital object 600 is obtained, such as a standard geometry and reconstruction of the digital object 600.

[0083] Reference now Figure 6B , in performing the pseudo-parallel correction of the orientation of the artifacts 602, 604, 606 in the digital object 600, the object 600 is reconstructed into a virtual object 612 whose overall shape changes from a parallelepiped to a trapezoid due to the deformation of the shape of the virtual object 612 caused by the magnification based on the height of the reconstructed XZ plane. More specifically, the result of the pseudo-parallel reconstruction performed on the simulated object 600 is that the reconstructed artifacts 602, 604, 606 are oriented relative to a selected reference angle (such as a selected angle of the source 122 relative to the artifact 602) to be aligned in nearly the same direction everywhere in the volume of the virtual object 612 and are contained in the XZ plane, similar to the source angle / projection image of the artifact 602. As the orientation of the artifacts 604 and 606 changes, they are completely contained within the selected plane, such as in Figure 6B In the XZ plane in , the geometric structure of the virtual object 612 is therefore different from the geometric structure of the digital object 600 .

[0084] Reference now Figure 7 and Figure 8 , illustrates some exemplary embodiments of methods for training AI 152 / CNN 154 to perform pseudo-parallel geometry correction prior to instantiation on imaging system 100. Figure 7In the illustrated method 700, initially in step 702, a digital object 701, which may store information about an actual object, a phantom object, or a simulated object (including a synthetic digital object phantom derived from a CT or MRI scan of an organ of interest, a breast CT or breast MRI scan, and a chest CT or chest MRI scan), is used as an object for a simulated DBT acquisition to generate a set of simulated projection images 704. The same digital object 700 is also subjected to pseudo-parallel geometry correction in step 706 to produce a corrected virtual digital object 708, wherein the pseudo-parallel geometry construction process performed is similar to the process discussed with respect to FIG.

[0085] In step 710, the set of simulated projection images 704 is provided as input to a CNN (e.g., CNN 154), which performs pseudo-parallel geometry reconstruction on the set of images 704 to produce a reconstructed virtual object 712. In step 714, the reconstructed virtual object 712 is compared with the corrected virtual object 708 (i.e., the ideal pseudo-parallel geometry corrected object) to calculate a loss function and update the model weights for the CNN 154 accordingly for use in subsequent executions of the method 700 to further train the operation of the CNN 154 until the loss function reaches the desired parameters. In this embodiment, the corrected virtual object 708 serves as the ground truth for comparison with the reconstructed virtual object 712.

[0086] In another exemplary embodiment for training AI 152 / CNN 154, Figure 8 In the illustrated method 800, a digital object 801 (e.g., a physical object or a simulated object) is used as a subject for each of a simulated actual DBT acquisition in step 802 and a simulated desired DBT acquisition in step 804 to produce a pair of simulated projection sets 806, 808, wherein the desired DBT acquisition includes an optimal number of simulated projections and the omission of one or more non-optimal parameters (such as dispersion, etc.).

[0087] The projection set 806 from the simulated actual acquisition is input into the CNN 154 to undergo reconstruction of the projections using pseudo-parallel geometric reconstruction in step 810 to produce a reconstructed actual virtual object 812 as an output from the CNN 154. Additionally, the projection set 808 resulting from the simulated desired or optimal acquisition is input into a separate reconstruction algorithm, network, or CNN 813 to undergo reconstruction of the projections 808 using pseudo-parallel geometric reconstruction in step 814 to produce a reconstructed desired virtual object 816 as an output from the CNN 813. In an alternative embodiment, a suitable algorithm other than artificial intelligence or a CNN is used to reconstruct the desired virtual object using pseudo-parallel geometric reconstruction.

[0088] Subsequently, in step 818, the reconstructed actual virtual object 812 is compared to the reconstructed desired virtual object 816 to calculate a loss function, thereby updating the model weights of the CNN 154 for use in subsequent executions of the method 700, thereby further training the operation of the CNN 154 until the loss function reaches the desired parameters. In this embodiment, the reconstructed desired virtual object 816 is used as the ground truth for comparison with the actual virtual object 812.

[0089] exist Fig. 9 In certain examples of the reconstruction system and method 1300 employed by the system 100 shown in FIG. 1 , including a trained AI 152 / CNN 154 for reconstructing and correcting artifacts in a 3D volume 171 from 2D / DBT projection images 101-109 employing a pseudo-parallel geometry correction and reconstruction process and / or method 1301, initially in step 1303, projection images 101-109 are obtained by the system 100 in the manner previously described. These projection images 101-109 are provided as input to a processing unit 150 / AI 152 / CNN 154, which performs the pseudo-parallel geometry reconstruction and correction process and / or method 1301 thereon.

[0090] In a first embodiment, in a first step 1350 of process 1301, processing unit 150 reconstructs a 3D virtual object, such as one or more virtual slices (XY) or planes (XZ, XY or YZ), slabs 1302 and / or virtual volumes 1310, from a set of projections 101-109 using pseudo-parallel geometry reconstruction, as described with respect to FIG6. In this process, artifacts are reoriented to a reference artifact orientation as shown in FIG6. In a specific implementation of this pseudo-parallel geometry correction in step 1350, each of the planes 1302 and / or volumes 1310 comprising the virtual object is transformed to be geometrically aligned with a reference angle for a zero or center projection 105, as described in FIG6. Figure 4 , such that the virtual object (e.g., each plane 1302 and / or volume 1310 and the artifacts contained therein) is oriented approximately perpendicular to the detector 140. In an exemplary embodiment, the planes 1302 and / or objects / volumes 1310 reconstructed in the pseudo-parallel geometric reconstruction method or system 1301 are 3D virtual objects reconstructed from 2D images of an actual object such as DBT projection images 101-109 of a breast. Such pseudo-parallel reconstruction methods have been proposed for planar tomosynthesis as described in Nett, but have not previously been applied to the process of reconstructing a 3D volume 171 oriented in the manner of the present disclosure, such as in the XZ and / or XY axes or planes 1302, using a CNN 154 such as a 2D or 2.5D CNN.

[0091] In step 1305, each of the pseudo-parallel reconstruction planes 1302 and / or volumes 1310 is fed into a CNN 154 that is specifically trained for correcting artifacts. With this altered geometric orientation of the virtual object (e.g., each plane 1302 and / or object volume 1310 and the artifacts therein), the subsequent reconstruction deep learning network or CNN 154 does not have to correct the angular displacement of any artifact located within the plane 1302 / volume 1310 during the correction of artifacts in the reconstruction of the corrected 3D slice or volume 171, because each artifact of the orientation is parallel to the reference angle, thereby greatly simplifying the processing and removal of artifacts within the plane 1302 when the volume 171 is reconstructed by the trained reconstruction AI 152 / CNN 154. In an alternative embodiment, instead of being performed in the processing unit 150, the first step 1350 of the initial reconstruction in the pseudo-parallel geometry can be implemented as part of the AI ​​152 or within a layer in the CNN 154.

[0092] In another exemplary embodiment, reference is now made to Fig.10 , which has been made public Fig. 9 As an alternative to method 1300, the artifacts are mostly planar in the XZ plane due to the geometric realignment of the plane 1302 to align with the reference angle / zero projection by the pseudo-parallel geometry correction system and method 1301 as implemented by the processing unit 150. As a result, the subsequently trained CNN 154 can simply operate as a 2D or 2.5D network in the XZ plane. Therefore, by using the realigned XZ axis planes 1302 and / or volume 1310 to reconstruct the output 3D volume 171, the training and operation of the CNN 154 employed by the system 100 for correcting artifacts in the 3D volume 171 in step 1305 based on the pseudo-parallel geometry reconstruction projections 101-109, virtual slices / planes 1302 and / or virtual objects / volumes 1310 are pipelined and / or simplified.

[0093] Reference now Figures 9 to 11 Regarding an exemplary implementation of the process of the present pseudo-parallel geometry reconstruction and correction method 1301, the processing unit 150 initially uses the 2D DBT projection images / tomographic fusion datasets 101-109 of the object / organ to reconstruct the organ (e.g., breast) as a zero projection 105 ( Figure 4) or a reference angle (e.g., a selected angle of 0° perpendicular to the detector 145) or the angle of the central projection of the tomosynthesis data set or another suitable angle (relative to the patient's chest wall). Within the reconstruction algorithm employed by the processing unit 150 in step 1350, or alternatively employed as part of the deep learning network 152 / CNN 154 to perform the pseudo-parallel geometric reconstruction and correction process 1301, two operators are necessary: ​​a back projection that maps the projection images 101-109 to the volume 1310, and a forward projection that maps the volume 1310 to the projection images 101-109, as shown in FIG. Fig.11 and Fig.12 The forward projection operator is used in a network implementing an unfolding or primal-dual method. It can also be used in an iterative reconstruction step performed before (such as in step 1350) or after the correction provided by the trained AI 152 / CNN 154.

[0094] Fig.11 Illustrated in Fig. 9 or Fig.10 1300, which is illustrated as being formed together with a UNet architecture 1312, although other suitable network architectures may also be employed. In this embodiment, the UNet network 1312 takes as input a first virtual object volume 1310 reconstructed in a pseudo-parallel geometry and applies artifact correction thereto. The Unet architecture / network 1312 generally involves convolutions and nonlinearities 1401 at a constant spatial resolution, followed by a max pooling operator 1402 in an analysis path 1410 which is repeated in a series of layers. On the synthesis portion 1420, the data undergoes convolutions and nonlinearities 1403, followed by an upconvolution operator 1404. The analysis and synthesis blocks are linked by skip connections 1405 to enable the network 1312 to output a corrected DBT plane / 3D volume 171.

[0095] Reference now Fig.12 and Fig.13, illustrates another specific example of the method 1300 and the trained CNN 154 employed therein, wherein the CNN 154 is formed using a learned primal-dual reconstruction algorithm 1320, wherein the forward / projection operators and the backprojection operators are replaced by pseudo-parallel geometry forward projection 1502 and backprojection 1504 operators. In this embodiment, the network 154, 1320 is typically fed with the projections 101-109 (denoted as g in the figure), the initialized principal and dual vectors (h0 and f0 in the figure) initialized in step 1360, and in some embodiments of system information, including but not limited to breast thickness or a 3D breast mask, which is in Fig.13 denoted as m in the figure. The dual block 1506 in the upper row 1508 operates in the projection domain. The dual block is coupled to the original block 1510 in the bottom row 1512. They are interconnected by a forward projection operator 1502 and a back projection operator 1504. In the present invention, these operators 1502, 1504 are implemented in a pseudo-parallel geometry. Utilizing this pseudo-parallel geometry directly within the network 154, 1320, any artifacts are mostly aligned in the XZ plane and are almost unchanged in orientation by the reconstruction of the plane 1302 and / or the volume 1310. This simplifies the training and testing of the CNN 154, 1320 and enables it to operate as a 2D or 2.5D network in the correction of artifacts set in the XZ plane of the reconstruction plane 1302 and / or the volume 1310.

[0096] In the operation of the learned primal-dual reconstruction algorithm 1320 to reconstruct the plane 1302 and / or object / 3D volume 1310 in pseudo-parallel geometry from the projections 101-109, after initialization, to perform back-projection, a first pseudo-parallel geometry operator 1504 reconstructs the plane 1302 / virtual object 1310 from the projection images / projection data sets 101-109 that are back-projected in parallel geometry to a reference angle defined by the null projection 105 relative to the detector 145 such that when projected with the parallel projection operator, the reconstructed plane 1302 / virtual object 1310 matches the center or null projection 105. One exemplary way to achieve this is to combine standard cone-beam back-projection with magnification that depends on the reconstruction height of the plane, as previously described with respect to FIG. 6 and referenced in Nett.

[0097] Furthermore, during the orthographic projection performed by the CNN 154, 1320, a second pseudo-parallel geometry operator 1502 reprojects (or orthographically projects) the virtual object 1310 along different projection angles of each initial projection 101-109 by combining downscaling depending on the height of the orthographic projection plane of the virtual object 1310 and standard cone-beam orthographic projection.

[0098] The processing unit 150 and the trained AI 152 / CNN 154 and combinations thereof can employ the same pseudo-parallel geometric correction and reconstruction system and method 1301 for any other reference angle within the range of positions of the source 140 relative to the detector 145. In this way, the geometric transformation of the voxels of each plane or slice 1302 and / or virtual object volume 1310 parallel to the reference angle (i.e., parallel to the detector 145) significantly simplifies the structural and / or computational requirements for the reconstruction and / or correction deep learning network or CNN 154, 1312, 1320 used to reconstruct and / or correct the visible 3D volume 171 using the virtual object volume 1310 as input. By using the system and method 1301, artifacts present in the plane 1302 / virtual object volume 1310 are transformed from artifacts that are highly translationally varying and oriented along a plane that is angled relative to the reconstructed coordinate axis to artifacts 602, 604, 606 that are substantially translationally invariant and oriented along a plane that is aligned with the reconstructed coordinate axis, as depicted in FIG. 6 . Thus, with this geometric alignment of plane 1302 / volume 1310 relative to a reference angle (e.g., perpendicular to detector 145), the speed and / or performance of training, testing, and performance of a deep learning network 152 (e.g., CNN 154, 1312, 1320) employed on imaging system 100 is significantly improved relative to prior art deep learning networks.

[0099] In the illustrated exemplary embodiment, after being trained according to one of the aforementioned training processes, the algorithm / AI 152 / CNN 154 employing the pseudo-parallel geometry correction system and method 1301 can be instantiated on the imaging system 100 in various suitable ways. For example, the algorithm / AI 152 / CNN 154 can be used as machine-readable instructions that include instructions for a processor (such as described below in conjunction with Fig.14 The program is a program executed by the processor 1612 shown in the example processor platform 1600 discussed above and forms an exemplary embodiment for the processing unit 150 of the system 100. The program may be embodied in software stored on a tangible computer-readable storage medium such as a CD-ROM, floppy disk, hard drive, digital versatile disk (DVD), Blu-ray disk, or memory associated with the processor 1612, but the entire program and / or portions thereof may alternatively be executed by a device other than the processor 1612 and / or embodied in firmware or dedicated hardware. In addition, although reference is made to Figures 9 to 13 An example procedure is described, but many other methods of implementing the example method 1301 may alternatively be used, altered, eliminated, or combined.

[0100] Fig.14 is achievable Figures 9 to 13The processor platform 1600 can be, for example, a server, a personal computer, a mobile device (e.g., a cell phone, a smart phone, an iPad, etc.). TM tablet computer), personal digital assistant (PDA), Internet appliance, DVD player, CD player, digital video recorder, Blu-ray player, game console, personal video recorder, set-top box, or any other type of computing device.

[0101] The processor platform 1600 of the illustrated example includes a processor 1612. The processor 1612 of the illustrated example is hardware. For example, the processor 1612 may be implemented by one or more integrated circuits, logic circuits, microprocessors, or controllers from any desired family or manufacturer.

[0102] The processor 1612 of the illustrated example includes a local memory 1613 (e.g., a cache). The processor 1612 of the illustrated example communicates with a main memory including a volatile memory 1614 and a non-volatile memory 1616 via a bus 1618. The volatile memory 1614 may be implemented by a synchronous dynamic random access memory (SDRAM), a dynamic random access memory (DRAM), a RAMBUS dynamic random access memory (RDRAM), and / or any other type of random access memory device. The non-volatile memory 1616 may be implemented by a flash memory and / or any other desired type of memory device. Access to the main memory 1614, 1616 is controlled by a memory controller.

[0103] The processor platform 1600 of the illustrated example also includes an interface circuit 1620. The interface circuit 1620 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and / or a PCI express interface.

[0104] In the example shown, one or more input devices 1622 are connected to the interface circuit 1620. The input devices 1622 allow a user to enter data and commands into the processor 1612. The one or more input devices may be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, buttons, a mouse, a touch screen, a trackpad, a trackball, isopoint, and / or a voice recognition system.

[0105] One or more output devices 1624 are also connected to the interface circuit 1620 of the illustrated example. The output device 1624 may be implemented, for example, by a display device (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display, a cathode ray tube display (CRT), a touch screen, a tactile output device, a printer, and / or a speaker). Thus, the interface circuit 1620 of the illustrated example typically includes a graphics driver card, a graphics driver chip, or a graphics driver processor.

[0106] The interface circuitry 1620 of the illustrated example also includes communication devices, such as transmitters, receivers, transceivers, modems, and / or network interface cards, to facilitate the exchange of data with an external machine (e.g., any kind of computing device) via a network 1626 (e.g., an Ethernet connection, a digital subscriber line (DSL), a telephone line, a coaxial cable, a cellular telephone system, etc.).

[0107] The processor platform 1600 of the illustrated example also includes one or more mass storage devices 1628 for storing software and / or data. Examples of such mass storage devices 1628 include floppy disk drives, hard disk drives, optical disk drives, Blu-ray disk drives, RAID systems, and digital versatile disk (DVD) drives, and the mass storage devices may be local, such as forming an integral part of the platform 1600, or remote, such as being operably connected to the processor platform 1600 via a network 1626.

[0108] Fig.14 The coded instructions 1632 may be stored in the mass storage device 1628, in the volatile memory 1614, in the non-volatile memory 1616, and / or on a removable tangible computer-readable storage medium (such as a CD or DVD).

[0109] As mentioned above, Figures 9 to 13The exemplary process 1301 and the CNNs 154, 1312, 1320 employed therein may be implemented using coded instructions (e.g., computer and / or machine readable instructions 1632) stored on a tangible computer readable storage medium, such as a hard drive, flash memory, read-only memory (ROM), compact disk (CD), digital versatile disk (DVD), cache, random access memory (RAM), and / or any other storage device or storage disk where information is stored for any duration (e.g., extended period of time, permanent, transient, for temporary buffering and / or for caching of information). As used herein, the term tangible computer readable storage medium is expressly defined to include any type of computer readable storage device and / or storage disk, and excludes propagating signals and transmission media. As used herein, "tangible computer readable storage medium" and "tangible machine readable storage medium" are used interchangeably. As used herein, when the phrase "at least" is used as a transitional term in the preamble of a claim, it is also open ended, just as the term "comprising" is open ended.

[0110] Although certain example methods, apparatus, and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus, and articles of manufacture that reasonably fall within the scope of the claims of this patent.

[0111] Technical effects of the disclosed subject matter include providing systems and methods for utilizing AI (e.g., a deep learning network) to provide enhanced artifact reduction in a reconstructed volume, wherein the AI ​​is trained to employ a pseudo-parallel reconstruction system and a process based on selected reference projections, which greatly simplifies the computational requirements, efficiency, training, and testing steps of the AI ​​152 / CNN 154, 1312, 1320 in the reconstruction process.

[0112] This written description uses examples to disclose the subject matter, including the best mode, and also to enable those skilled in the art to practice the subject matter, including making and using any devices or systems and performing any included methods. The patentable scope of the disclosed subject matter is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

Claims

1. A method for correcting artifacts in a three-dimensional (3D) volume (171) reconstructed from a plurality of two-dimensional (2D) projection images (101-109) of an object, the method comprising the following steps: a. Providing an imaging system (100), the imaging system comprising: i. a radiation source (140); ii. a detector (145) positionable to receive radiation emitted from the radiation source (140) and passing through an object (132) positioned between the source (140) and the detector (145); iii. a display (170) for presenting information to a user; iv. a processing unit (150) connected to the display (170) and operable to control the operation of the radiation source (140) and the detector (145) to generate a plurality of 2D projection images (101-109) of the object (132); and v. a memory (160) operably connected to the processing unit (150) and storing processor executable code for a reconstruction algorithm (152), the processor executable code being operable, when executed by the processing unit (150), to reconstruct a three-dimensional (3D) volume (171) of the organ (132) based on the plurality of 2D projection images (101-109); b. Obtaining the plurality of 2D projection images (101-109); c. Selecting a zero angle (143) from the range of angles for obtaining the plurality of 2D projection images (101-109); d. Reconstructing the 3D volume (171) from the plurality of 2D projection images (101-109), defining the 3D volume (171) in a pseudo-parallel geometry based on the zero angle (143) from the plurality of 2D projection images (101-109).

2. The method according to claim 1, wherein the step of reconstructing the 3D volume (171) comprises: a. reconstructing a 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109); and b. Correcting the 3D virtual object (612) to form the 3D volume (171).

3. The method of claim 2, wherein the reconstruction algorithm (152) comprises a network (154) operable to reconstruct and correct the 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109).

4. The method of claim 3, wherein the network (154) operable to reconstruct and correct the 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109) is selected from a 2D or 2.5D network (154).

5. The method of claim 4, wherein the network (154) operable to reconstruct and correct the 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109) comprises reconstructing and correcting the network (154) of the 3D virtual object (612) along an XZ plane (2004) defined by the imaging system (100).

6. The method of claim 4, wherein the network (154) operable to reconstruct and correct the virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109) comprises reconstructing and correcting the network (154) of the virtual object (612) along an XY plane (2002) defined by the imaging system (100).

7. The method of claim 3, wherein the network (154) is a convolutional neural network (154).

8. The method of claim 3, wherein the network (154) embeds a backprojection operator (1504) and a forward projection operator (1502), and wherein the backprojection (1504) operator and the forward projection (1502) operator are implemented in the pseudo-parallel geometry.

9. The method of claim 3, wherein the 3D virtual object (612) is selected from the group consisting of a 3D slice, a 3D slab, a 3D plane, a 3D volume, and combinations thereof.

10. The method of claim 1, wherein the processor executable code for a reconstruction algorithm (152) is a 2D or 2.5D convolutional neural network (154).

11. The method of claim 1, wherein the reconstruction algorithm (152) comprises a network (154) operable to reconstruct and correct a 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) from the plurality of 2D projection images (101-109), and wherein the step of reconstructing the 3D volume (171) comprises: a. reconstructing the 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109); and b. Providing the 3D virtual object (612) as input to the network (154).

12. An imaging system (100), comprising: a. a radiation source (140); b. a detector (145) positionable to receive radiation emitted from the radiation source (140) and passing through an object (132) positioned between the source (140) and the detector (145); c. a display (170) for presenting information to a user; d. a processing unit (150) connected to the display (170) and operable to control the operation of the radiation source (140) and the detector (145) to generate a plurality of 2D projection images (101-109) of the object (132); and e. a memory (160) operably connected to the processing unit (150) and storing processor executable code for a reconstruction algorithm (152), the processor executable code being operable, when executed by the processing unit (150), to reconstruct a three-dimensional (3D) volume (171) of the organ (132) based on the plurality of 2D projection images (101-109); The memory (160) includes processor executable code for: reconstructing the 3D volume (171) from the plurality of 2D projection images (101-109), defining the 3D volume (171) in a pseudo-parallel geometry based on a zero angle (143) from the plurality of 2D projection images (101-109), Wherein the step of reconstructing the 3D volume (171) comprises:

1. Reconstructing a 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109); and 2. Correcting the 3D virtual object (612) to form the 3D volume (171).

13. The imaging system of claim 12, wherein the processor executable code for the reconstruction algorithm (152) comprises processor executable code for a network (154) operable to reconstruct and correct the 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109).

14. The imaging system of claim 12, wherein the processor executable code for the reconstruction algorithm (152) comprises processor executable code for a 2D or 2.5D network (154) operable to reconstruct and correct the 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109).

15. The imaging system of claim 12, wherein the processor executable code for reconstructing the 3D virtual object (612) comprises processor executable code for a network (154), the network being operable to reconstruct and correct the 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109), for reconstructing the 3D virtual object (612) defined in the pseudo-parallel geometry based on the zero angle (143) according to the plurality of 2D projection images (101-109) using the reconstruction algorithm (152); and providing the 3D virtual object (612) as input to the network (154).

Images