Automatic calculation of BPE levels

Abstract

Methods and systems are provided for automatically assessing a BPE level of a patient of an imaging system (100) based on one or more medical images of one or more breasts of the patient using a deep learning (DL) model (322). The medical image may include a contrast-enhanced mammography (CEM) image (400), a magnetic resonance (MR) image, or a different type of image. The one or more images may include one or more images of the same breast, where the BPE evaluation (326) output by the DL model (322) may include a score (1004), such as a percentage of BPE detected in the image. The one or more images may include images of a left breast (1102) and a right breast (1104) of the patient, wherein the BPE evaluation (326) output by the DL model (322) may include whether an asymmetry is detected between the BPE evaluations of the left breast (1102) and the right breast (1104).

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to systems and methods for contrast-enhanced breast imaging. Background Art

[0002] Contrast-enhanced imaging is a screening / diagnostic method that can be used to visualize lesions in tissue such as breast tissue. Contrast-enhancing agents are administered intravenously and may localize in lesion tissue due to rapid neovascularization around the lesion, resulting in imperfect blood vessels that may leak contrast agent into surrounding tissue. Contrast enhancement is particularly useful in imaging lesions in breasts with dense breast tissue. Contrast-enhancing agents may additionally be delivered to normal, healthy breast tissue and appear in contrast-enhanced images as areas of enhanced contrast in a phenomenon known as breast background parenchymal enhancement. Summary of the invention

[0003] In one embodiment, a method includes using a deep learning (DL) model trained based on different types of breast medical images, performing an assessment of background parenchymal enhancement (BPE) of a patient's breast based on one or more images of the breast acquired via an imaging system; and displaying the one or more images and the BPE assessment at a display device.

[0004] It should be understood that the above brief description is provided to introduce in a simplified form selected concepts that are further described in the detailed description. It is not meant to identify key features or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages noted above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The present invention will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0006] Figure 1 is a schematic diagram of an imaging system according to one or more embodiments of the present disclosure;

[0007] Figure 2 is an image processing system according to one or more embodiments of the present disclosure;

[0008] Figure 3 is a block diagram illustrating a training system for a deep learning (DL) model according to one or more embodiments of the present disclosure;

[0009] Figure 4 is a medical image showing background parenchymal enhancement (BPE) as in the prior art;

[0010] Figure 5 is a flow chart illustrating a method for training a DL model to perform BPE evaluation on an image according to one or more embodiments of the present disclosure;

[0011] Figure 6 is a flow chart illustrating a method for performing BPE evaluation on an image using a trained DL model according to one or more embodiments of the present disclosure;

[0012] Figure 7 is a first flow chart illustrating a first method for detecting asymmetry between a BPE level of a first breast of a patient and a BPE level of a second breast of the patient according to one or more embodiments of the present disclosure;

[0013] Figure 8 is a second flow chart illustrating a second method for detecting asymmetry between a BPE level of a first breast of a patient and a BPE level of a second breast of the patient according to one or more embodiments of the present disclosure;

[0014] Fig. 9 is a first exemplary image of a breast including a first type of BPE assessment according to one or more embodiments of the present disclosure;

[0015] Fig.10 is a second exemplary image of a breast including a second type of BPE assessment according to one or more embodiments of the present disclosure; and

[0016] Fig.11 An exemplary display of two breasts of a patient in which no BPE asymmetry was detected is shown, according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] The following detailed description relates to systems and methods for displaying contrast-enhanced breast images including two-dimensional (2D) images and three-dimensional (3D) image volumes. Contrast-enhanced breast images may include images in which contrast agents are administered intravenously and are preferentially localized in lesions or other suspicious areas due to leaky neovascularization surrounding the lesions. As a non-limiting list, breast images may include contrast-enhanced digital mammography (CEM) images, contrast-enhanced digital breast tomosynthesis (CE-DBT) image volumes, CE-DBT slices (e.g., planes of a CEDBT volume) and / or CE-DBT blocks (e.g., thick slices created from a combination of CE-DBT slices), CEM biopsy images, and / or CEM synthetic 2D images (e.g., synthetic 2D images created from a CE-DBT image volume that represents the most important information contained in a 3D image volume in a 2D view).

[0018] In order to analyze the risk of breast cancer, doctors can perform background parenchymal enhancement (BPE) analysis. BPE can occur during CEM or contrast-enhanced MRI procedures or contrast-enhanced CT procedures, where contrast agents are also positioned in normal, healthy breast tissue. The level of BPE in the breast can depend on the tissue venous blood pool and its permeability to contrast agents. Clinicians can use both the BPE level in the breast and the asymmetry of the BPE level between breasts to help diagnose and estimate cancer risk. Therefore, the BPE level can be indicated in the report of CEM according to clinical guidelines.

[0019] During a CEM examination, two images are acquired, a low energy (LE) image and a high energy (HE) image, which are recombined to produce an iodine uptake image (REC), which can show bilateral enhancement of normal breast parenchyma after administration of contrast agent. The breast can be divided into four categories based on the REC image, which may be related to the risk of cancer and diagnostic performance. However, the classification of breast BPE levels is typically performed by radiologists using a manual workflow, which may increase the amount of time spent by the radiologist reading the image and increase the use of imaging system resources. In addition, assessing the BPE level may be difficult and may depend on the experience of the radiologist. Therefore, the differences between the assessments performed by different radiologists may be large. For example, a first radiologist may assess the BPE level of a medical image as moderate, while a second radiologist may assess the BPE level of a medical image as mild.

[0020] In order to solve the problem of large differences, reduce the amount of time spent by radiologists reading images, and reduce the use of imaging system resources, this article proposes a system and method for automatically calculating and displaying BPE assessments of one or more breasts based on one or more images using a deep learning (DL) model such as a convolutional neural network (CNN). The automatically calculated BPE assessment may be more accurate than the manual BPE assessment performed by the radiologist, thereby making the diagnosis of cancer risk more accurate and / or helping the radiologist detect lesions in the breast. Compared with the ML model for breast density assessment, the DL model of the present invention can take multiple images of the breast as input. For example, the DL model can take both the low-energy image (LE) and the reconstructed image (REC) of the CEM examination as input, and then output the BPE assessment based on both the LE image and the REC image. By adjusting the X-ray beam of the imaging system between acquisitions, LE images and REC images can be generated during the same breast examination. An additional advantage of the method of the present invention is that the efficiency of the entire process of reading mammography images can be improved, thereby speeding up diagnosis and improving patient care. Furthermore, the output of the DL model may be used to analyze whether there is an asymmetry in the BPE between the patient's right and left breasts and / or whether there is an inconsistency in the BPE between different images of the same breast.

[0021] A block diagram of an image processing system that may store one or more methods for identifying and displaying LE, HE, and / or REC images (all of which are collectively referred to herein as CEM images) is shown in FIG. Figure 2 The image processing system may be communicatively coupled to an imaging system, which may be a digital mammography system or a DBT imaging system. A schematic diagram of the imaging system is shown in Figure 1 The imaging system may be used to generate a contrast agent uptake image that may be displayed on a display device of the imaging system. The displayed image may be automatically processed by the image processing system, and the image may be updated according to the result of the processing. Specifically, the image processing system may use the DL model to automatically calculate the BPE level of the breast in the image, and display the calculated BPE level in the image on the display device. The DL model may be based on Figure 5 The method is used in Figure 3 After training, you can follow the training system shown in the training system. Figure 6 In one or more steps of the method, the DL model is applied to contrast agent uptake breast images, such as those obtained during a CEM examination. An example of a contrast agent uptake image including BPE is shown in Figure 4 In addition, you can use Figure 7 and Figure 8 The method compares the BPE assessments performed by the DL model for the patient's left breast and the patient's right breast. An exemplary display of a contrast agent uptake image including the output of the DL model is shown in Fig. 9 , Fig.10 and Fig.11 middle.

[0022] Now turn to Figure 1 , an imaging system 100 including an x-ray system 10 for acquiring images of a breast according to one or more embodiments of the present disclosure is shown. In some examples, the x-ray system 10 may be a digital mammography system or a tomosynthesis system, such as a digital breast tomosynthesis (DBT) system. In addition, the x-ray system 10 may be used to perform one or more procedures, including digital tomosynthesis imaging, DBT-guided breast biopsy, and CEM examinations.

[0023] The x-ray system 10 includes a support structure 42 to which a radiation source 16, a radiation detector 18, and a collimator 20 are attached. The radiation source 16 is housed within a gantry 15 that is movably coupled to the support structure 42. Specifically, the gantry 15 may be mounted to the support structure 42 so that the gantry 15 including the radiation source 16 may rotate relative to the radiation detector 18 about an axis 58. The rotation angle range of the gantry 15 housing the radiation source 16 indicates rotation up to a desired angle in either direction about the axis 58. For example, the rotation angle range of the radiation source 16 may be -θ to +θ, where θ may make the angle range a limited angle range of less than 360 degrees. An exemplary x-ray system may have an angle range of ±11 degrees, which may allow the gantry to rotate from -11 degrees to +11 degrees (i.e., rotation of the radiation source) about the axis of rotation of the gantry. The angle range may vary according to manufacturing specifications. For example, the angle range of a DBT system may be approximately ±11 degrees to ±60 degrees, depending on manufacturing specifications.

[0024] The radiation source 16 is oriented toward the volume or object to be imaged and is configured to emit radiation rays at desired times to acquire one or more images. The radiation detector 18 is configured to receive the radiation rays via the surface 24. The radiation detector 18 may be any of a variety of different detectors, such as an x-ray detector, a digital radiography detector, or a flat panel detector. The collimator 20 is disposed adjacent to the radiation source 16 and is configured to adjust the irradiation zone of the subject.

[0025] In some embodiments, the x-ray system 10 may also include a patient shield 36 mounted to the radiation source 16 via a mask rail 38 so that a body part (e.g., head) of the patient is not directly exposed to the radiation. The x-ray system 10 may also include a compression paddle 40 that may be movable upward and downward relative to the support structure along a vertical axis 60. Thus, the compression paddle 40 may be adjusted to be positioned closer to the radiation detector 18 by moving the compression paddle 40 downward toward the radiation detector 18, and the distance between the radiation detector 18 and the compression paddle 40 may be increased by moving the compression paddle upward away from the detector along the vertical axis 60. The movement of the compression paddle 40 may be adjusted by a user via a compression paddle actuator (not shown) included in the x-ray system 10. The compression paddle 40 may hold a body part (such as a breast) in position against the surface 24 of the radiation detector 18. The compression paddle 40 may compress the body part and hold the body part in position while optionally providing a hole to allow insertion of a biopsy needle, such as a core needle or a vacuum-assisted core needle. Thus, compression paddles 40 may be used to compress a body part to minimize the thickness through which x-rays pass and to help reduce movement of the body part due to patient movement. X-ray system 10 may also include a subject support (not shown) on which a body part may be positioned.

[0026] The imaging system 100 may also include a workstation 43 including a controller 44 including at least one processor and memory. The controller 44 may be communicatively coupled to one or more components of the x-ray system 10, including one or more of the radiation source 16, the radiation detector 18, the compression paddle 40, and the biopsy device. In one embodiment, communication between the controller and the x-ray system 10 may be via a wireless communication system. In other embodiments, the controller 44 may be in electrical communication with one or more components of the x-ray system via a cable 47. Furthermore, in an exemplary embodiment, as shown in FIG. Figure 1 As shown, the controller 44 is integrated into the workstation 43. In other embodiments, the controller 44 may be integrated into one or more of the various components of the system 10 disclosed above. In addition, the controller 44 may include processing circuitry that executes stored program logic, and may be any of a variety of computers, processors, controllers, or combinations thereof that are usable and compatible with various types of equipment and devices used in the x-ray system 10.

[0027] Workstation 43 may include a radiation shield 48 that protects an operator of system 10 from radiation rays emitted by radiation source 16. Workstation 43 may also include a display 50, keyboard 52, mouse 54, and / or other suitable user input devices that facilitate control of system 10 via user interface 56.

[0028] The controller 44 may regulate the operation and functionality of the x-ray system 10. For example, the controller 44 may provide timing control regarding when the x-ray source 16 emits x-rays, and may further regulate how the radiation detector 18 reads and transmits information or signals after the x-rays strike the radiation detector 18, as well as how the x-ray source 16 and the radiation detector 18 move relative to each other and relative to the body part being imaged. The controller 44 may also control the manner in which information, including images and data acquired during operation, is processed, displayed, stored, and manipulated. The operations performed by the controller 44 as described herein are described herein. Figure 2 and Figure 3 The various processing steps described may be provided by a set of instructions stored in a non-volatile memory of the controller 44 .

[0029] Further, as described above, the radiation detector 18 receives radiation rays emitted by the radiation source 16. Specifically, during imaging with the x-ray system, a projection image of the imaged body part may be obtained at the radiation detector 18. In some embodiments, data received by the radiation detector 18, such as projection image data, may be communicated electronically and / or wirelessly from the radiation detector 18 to the controller 44. The controller 44 may then reconstruct or reassemble one or more scanned images based on the projection image data, for example by implementing a reconstruction algorithm or a reassembly algorithm. The reconstructed image or reassembled image may be displayed to the user on the user interface 56 via the display 50 (e.g., a display screen).

[0030] The radiation source 16 together with the radiation detector 18 form part of the x-ray system 10, which provides x-ray images for the purpose of screening for abnormalities, diagnosis, dynamic imaging and image-guided biopsy. For example, the x-ray system 10 can be operated in a mammography mode to screen for abnormalities. During mammography, the patient's breast is positioned and compressed between the radiation detector 18 and the compression paddle 40. Therefore, the volume of the x-ray system 10 between the compression paddle 40 and the radiation detector 18 is the imaging volume. The radiation source 16 then emits radiation rays onto the compressed breast, and a projection image of the breast is formed on the radiation detector 18. The projection image can then be reconstructed or reassembled by the controller 44 and displayed on the display 50. During mammography, the gantry 15 can be adjusted at different angles to obtain images of different orientations, such as head-tail (CC) images and medial-lateral oblique (MLO) images. In one example, the gantry 15 can rotate around the axis 58 while the compression paddle 40 and the radiation detector 18 remain stationary. In other examples, gantry 15 , compression paddles 40 , and radiation detector 18 may rotate about axis 58 as a single unit.

[0031] In some examples, a breast imaging system such as imaging system 100 can be configured to perform contrast-enhanced imaging, wherein a contrast agent such as iodine can be injected into a patient, which travels to a region of interest (ROI) (e.g., a lesion) within the breast. The contrast agent is absorbed in blood vessels surrounding a cancerous lesion in the ROI, thereby enhancing the ability to locate the lesion. In some examples, the contrast agent can additionally be absorbed by healthy breast tissue, causing the BPE to be visible in the acquired images.

[0032] The use of contrast agents can be combined with images of the ROI taken using dual energy imaging techniques and technologies. In dual energy imaging, a low energy (LE) image and a high energy (HE) image of the breast containing the ROI are taken. For each view, a pair of images is acquired: a low energy (LE) image and a high energy (HE) image. The LE and HE images are typically acquired at average energies above and below the k-edge of the contrast agent. At x-ray energies just above the k-edge of the contrast agent, the absorption of x-rays by the contrast agent increases, resulting in a difference in signal intensity between the LE image and the HE image. The LE image and the HE image are therefore recombined to produce a REC image corresponding to the iodine uptake image.

[0033] In dual-energy 3D or stereotactic procedures, LE and HE image acquisitions are performed with the x-ray source having at least two different positions relative to the detector. The images are then recombined to show iodine uptake information about the internal structures of the imaged tissue. Contrast agents can pool in healthy breast tissue, leading to BPE.

[0034] An example of a REC image 400 of a breast is shown in Figure 4 . The breast in REC image 400 does not include a lesion or other type of suspicious area. However, REC image 400 includes region 402, which is shown as having a higher contrast (e.g., brighter) than surrounding breast tissue. Region 402 may be attributable to BPE. BPE may be healthy tissue, but the level of BPE may be associated with the risk of developing a cancerous lesion. For this reason, in addition to or in lieu of identifying suspicious areas, clinicians may also be interested in identifying BPE in REC images or other contrast-enhanced images.

[0035] Figure 2 A block diagram 200 of an image processing system 202 is shown according to one embodiment. In some embodiments, the image processing system 202 is incorporated into the imaging system 100. In some embodiments, at least a portion of the image processing system 202 is disposed at a device (e.g., an edge device, a server, etc.) that is communicatively coupled to the imaging system 100 via a wired connection and / or a wireless connection. In some embodiments, at least a portion of the image processing system 202 is disposed at a separate device (e.g., a workstation) that can receive images from the imaging system 100 or from a storage device that stores images / data generated by the imaging system 100. The image processing system 202 can be operably / communicatively coupled to a user input device 232 and a display device 234. At least in some examples, the user input device 232 can include the user interface 56 of the imaging system 100, and the display device 234 can include the display device 50 of the imaging system 100.

[0036] The image processing system 202 includes a processor 204 configured to execute machine-readable instructions stored in a non-transitory memory 206. The processor 204 may be a single-core processor or a multi-core processor, and the program executed thereon may be configured for parallel processing or distributed processing. In some embodiments, the processor 204 may optionally include separate components distributed on two or more devices, which may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the processor 204 may be virtualized and performed by a remotely accessible networked computing device configured in a cloud computing configuration.

[0037] The non-transitory memory 206 may store a model module 208, a training module 210, an inference module 212, and an image database 214. The model module 208 may include at least one machine learning (ML) model and / or a deep learning (DL) model, and instructions for implementing at least one DL model to automatically assess the BPE level of the breast based on an image of the breast, as described in more detail below. The model module 208 may include various types of models, including trained and / or untrained neural networks such as CNNs, statistical models, or other models, and may also include various data or metadata related to one or more models stored therein.

[0038] The non-transitory memory 206 may also store a training module 210, which may include instructions for training at least one DL model stored in the model module 208. Specifically, the training module 210 may include instructions that, when executed by the processor 204, cause the image processing system 202 to perform one or more steps of the method 500 for training a neural network model, which will be referred to below. Figure 5 In some embodiments, the training module 210 may include instructions for implementing one or more gradient descent algorithms, applying one or more loss functions, and / or training routines for adjusting parameters of one or more neural networks of the model module 208. The training module 210 may include a training data set for one or more models of the model module 208.

[0039] The non-transitory memory 206 also stores an inference module 212. The inference module 212 may include instructions for deploying the trained DL model, for example, to automatically assess the BPE level of the breast based on an image of the breast. Specifically, the inference module 212 may include instructions that, when executed by the processor 204, cause the image processing system 202 to perform Figure 6 Instructions for one or more steps of method 600, as described in further detail below.

[0040] The non-transitory memory 206 also stores an image database 214. The image database 214 may include, for example, images acquired via the imaging system 100. The image database 214 may include various types of medical images used in one or more training sets for training one or more neural networks of the model module 208.

[0041] In some embodiments, non-transitory memory 206 may include components disposed on two or more devices that may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of non-transitory memory 206 may include a remotely accessible networked storage device configured in a cloud computing configuration.

[0042] The user input device 232 may include one or more of a touch screen, keyboard, mouse, trackpad, motion sensing camera, or other device configured to enable a user to interact with and manipulate data within the image processing system 202. In one example, the user input device 232 may enable a user to select images for training a DL model, or for further processing using a trained DL model.

[0043] The display device 234 may include one or more display devices utilizing nearly any type of technology. In some embodiments, the display device 234 may include a computer monitor and may display ultrasound images. The display device 234 may be combined with the processor 204, the non-transitory memory 206, and / or the user input device 232 in a common housing, or may be a peripheral display device and may include a monitor, a touch screen, a projector, or other display devices known in the art that may enable a user to view ultrasound images generated by the imaging system 100 and / or interact with various data stored in the non-transitory memory 206.

[0044] It should be understood that Figure 2 The image processing system 202 shown is for illustration and not for limitation. Another suitable image processing system may include more, fewer, or different components.

[0045] refer to Figure 3 , shows an example of a BPE evaluation model training system 300. The BPE evaluation model training system 300 may be configured by, for example, Figure 2 The image processing system 202 of the image processing system is implemented to train a DL model to automatically assess the BPE level of the breast based on one or more images of the breast. In one embodiment, the BPE assessment model training system 300 includes a BPE assessment model 302 to be trained, which can be a model module 392 of the image processing system (e.g., Figure 2 Part of the model module 208).

[0046] The BPE evaluation model 302 may be trained based on a training data set 306, which may be stored in a training module 394 (e.g., Figure 2 The training module 210 of FIG. 306 may include a plurality of training pairs. Each training pair may include at least one input image including a breast, and at least one corresponding image of a breast with BPE segmented and / or a corresponding BPE level of the breast as a true image. The input image may be obtained from an image dataset 310, which may include CEM images, CE-DBT image volumes / slices / blocks, CEM biopsy images, and / or images obtained using a method such as Figure 1 The image data set 310 may be a 2D composite image of a breast generated by the imaging system 308 of the imaging system 308 and / or the imaging system 100 for various breast examinations performed on various subjects. In some examples, the image data set 310 may be stored in an image database of the image processing system (e.g., the image database 214). Alternatively, in some embodiments, the image data set 310 may be an external image data set, such as a public data set of mammography images.

[0047] The input images may include LE images, REC images, and / or both REC images and LE images and / or other types of medical images as described above. In one embodiment, REC images (e.g., contrast agent uptake images) and LE images (e.g., morphological images) may be included as input in a training pair, where both the REC images and the LE images will be input into the BPE evaluation model 302 simultaneously. For example, by adjusting the X-ray of the imaging system between acquiring high-energy images and low-energy images (e.g., Figure 1In another embodiment, one or more CE-DBT images or CEM biopsy images may be included in the training pair as input. During a CEM or CE-DBT examination, the breast may be imaged at various rotation angles of the mammography system. For example, during a CEM examination, two views may be acquired: a cranio-caudal view (CC) and a medial-lateral oblique (MLO) view. A BPE assessment may be generated for each patient, wherein the BPE assessment model 302 may use various images and different views (CC / MLO) of an examination of both breasts of the patient as input. A BPE assessment may also be generated for a breast, wherein the BPE assessment model 302 may use various images of one breast acquired in different views (CC / MLO) as input. During a CEM or CE-DBT examination, and in order to calculate a global BPE level or score for each breast and patient, injection timing (the time at which the contrast agent is injected) and acquisition timing information (the time at which the image of the breast is acquired) may be used. As an example, the BPE score of the breast may be the maximum value of a first BPE score of the breast based on the CC image and a second BPE score of the breast based on the MLO image. As another example, the BPE score of the breast may be calculated based on the first BPE score of the breast based on the CC image, the second BPE score of the breast based on the MLO image, the first time when the CC image is generated, the second time when the MLO image is generated, and / or the time when the contrast agent is injected.

[0048] The training module 394 may include a dataset generator 312 that may generate training pairs for the training dataset 306. Generating the training pairs may include labeling an input image of the image dataset 310 with a true BPE assessment. In various embodiments, the true BPE assessment may include a BPE score, wherein the score is an estimated percentage of tissue of the breast that displays BPE relative to the surface (e.g., in the case of 2D imaging) or volume (e.g., in the case of 3D imaging, such as CE-DBT) of the breast. When estimated relative to the volume of the breast, the estimated percentage of tissue of the breast that displays BPE may be expressed relative to the total glandular tissue volume or the entire breast volume. The surface or volume of the breast may be estimated by a breast surface / volume extractor. For example, if 65% of the breast tissue of the breast displays BPE, the BPE score may be 65%. In other embodiments, the true BPE assessment may include a BPE classification, rather than a score. For example, according to the Breast Imaging Reporting and Data System (BIRADS) dictionary, the BPE classification may include four categories: minimal, mild, moderate, and significant, which are based on the observed intensity and pattern. In some embodiments, a+ or a- can be added to generate eight categories based on the BIRADS dictionary (eg, minimal + / -, mild + / -, moderate + / -, marked + / -).

[0049] In some examples, a true BPE score may be assigned to an input image during a manual procedure, where a human expert (e.g., a radiologist) estimates the BPE level of the input image. In some embodiments, a real image with BPE segmented may be included in the training pair, and the BPE assessment model 302 may learn to segment the BPE in the input image. The real image with BPE segmented may be generated by a segmentation model, or manually.

[0050] In some embodiments, the training pair may include additional input data. For example, in a first embodiment, the training pair may include a single input image, a version of the single input image in which the BPE is segmented as real data, and a corresponding real label (BPE score). In a second embodiment, the training pair may include two input images of the same breast (e.g., a REC image and a LE image), a real version of the two input images in which the BPE is segmented, and the real BPE scores of both input images. The LE image can be used to determine whether the enhancement is a BPE or an artifact. In a third embodiment, the training pair may include one or more input images and other relevant clinical data of the patient to whom the breast belongs and the encoding of the real data. For example, some studies have shown that BPE may represent physiological hormone enhancement, reflecting hormone-related changes in breast composition and vascular distribution. As an example, fluctuations in BPE have been confirmed throughout the menstrual cycle (with the highest level of enhancement during the luteal phase in the second half of the menstrual cycle, when breast cell proliferation is at its highest level). BPE has also been shown to reflect changes in estrogen-mediated vascular permeability, with increased BPE seen in women receiving estrogen replacement therapy and reduced BPE seen in patients taking anti-estrogen drugs and postmenopausal patients. Thus, the training pairs may include encodings of the patient in the menstrual cycle, menopausal state, or encodings of treatment-related information, such as a comparison between a pre-neoadjuvant therapy treatment image and a post-neoadjuvant therapy treatment image. The encodings may be input into the training BPE evaluation model 302 along with the input images.

[0051] Furthermore, in some embodiments, a training pair may include a first image (or a set of reconstructed and low-energy images, a CE-DBT image volume, etc.) of a patient's first (e.g., left) breast and a second image (or a set of reconstructed and low-energy images, a CE-DBT image volume, etc.) of a patient's second (e.g., right) breast, and the BPE assessment model 302 may be trained to output an assessment of whether there is asymmetry between a first BPE of the first breast and a second BPE of the second breast. If the difference between the first BPE and the second BPE is greater than a threshold difference, such as, for example, 5%, asymmetry may exist. For example, if the first BPE is 65% and the second BPE is 55%, the difference between the first BPE and the second BPE is 10%, which is greater than the threshold 5%, such that there is BPE asymmetry between the first breast and the second breast. In other embodiments, BPE asymmetry may be determined by a separate asymmetry detection module based on BPE assessments of the left and right breasts, as described below in Figure 7 Described in more detail in .

[0052] During a CEM or CE-DBT examination, contrast media circulates within the breast, so BPE levels vary over time and with patient physiology. Multiple views of the breast may be acquired at different timings. Therefore, the iodine levels within the breast may vary for different acquisitions. Therefore, to some extent, it is expected that BPE levels may vary between breasts (left / right images) and between different views acquired during the examination (e.g., CC / MLO). Optionally, a late additional view such as a medial-lateral oblique view ML may be acquired, where a delay is imposed (e.g., more than 5 minutes) before the acquisition of the post-injection image and the initial view. In this case, it may also be expected that the BPE level may be different from other views acquired between 2 minutes and 7 minutes after injection due to the clearance of the contrast media. In order to take this natural phenomenon into account, the asymmetry detection module may take into account the injection time and the image acquisition time to produce an asymmetry result.

[0053] Once a training pair for a training dataset 306 has been generated, the training pair may be assigned to a training dataset, a validation dataset, or a test dataset. Each training pair may include a plurality of input images, a corresponding plurality of BPE segmented real versions of the input images, and / or a corresponding plurality of real BPE evaluations / scores. The training dataset may be used for optimization of the BPE evaluation model 302. The validation dataset may be used to prevent overfitting, whereby the BPE evaluation model 302 learns to map features specific to samples in the training set that are not present in the validation set. The test dataset may be used to estimate the performance of the model in deployment. The number of training pairs in the test dataset and the validation dataset may be less than the number of training pairs in the training dataset.

[0054] The BPE assessment model training system 300 can be implemented to train the BPE assessment model 302 to learn to predict the BPE level of the breast included in the input image. The BPE assessment model 302 can be configured to receive training pairs from the training module 394 and output the predicted BPE level. Then, based on the difference between the output predicted BPE level and the target BPE level included in the relevant training pair, one or more parameters of the BPE assessment model 302 can be iteratively adjusted to minimize the loss function until the error rate is reduced to below the first threshold error rate. Figure 5 The training of the BPE evaluation model 302 is described in more detail.

[0055] In various embodiments, the input images may be input into the BPE assessment model 302 as 2D or 3D arrays of image values ​​(e.g., pixels with x / y positions or voxels with x / y / z positions). In other words, each data value of each pixel / voxel of each input image may be input into an input node of the BPE assessment model 302. The image values ​​may correspond to two images, one for the LE image and one for the REC image. That is, a first 2D array of image values ​​representing the LE image may be a first set of inputs into the BPE assessment model 302, and a second 2D array of image values ​​representing the REC image may be a second set of inputs into the BPE assessment model 302. Alternatively, the image values ​​may correspond to one or more CE-DBT image volumes, wherein a 3D array of image values ​​representing the CE-DBT image volumes may be a set of inputs into the BPE assessment model 302.

[0056] The BPE evaluation model training system 300 may include a validator 320 for validating the performance of the BPE evaluation model 302. The validator 320 may take as input the trained or partially trained BPE evaluation model 302 and a validation data set of a training pair. If the error rate of the trained or partially trained BPE evaluation model 302 on the validation data set of the training pair is reduced to below a second threshold error rate, the performance of the trained or partially trained BPE evaluation model 302 may be verified, whereby the training phase of the trained or partially trained BPE evaluation model 302 may end.

[0057] The BPE assessment model training system 300 may include an inference module 396 that includes a trained BPE assessment model 322 that has been validated by the validator 320 as described above. The inference module 396 may also include instructions for deploying the trained BPE assessment model 322 to perform a BPE assessment (e.g., predict a BPE level) on a breast in a new (e.g., not included in the image dataset 310) 2D or 3D medical image (e.g., a contrast-enhanced mammography image). For example, a new medical image may be generated by the imaging system 308 during a chest examination of a patient. Specifically, the trained BPE assessment model 322 may receive a new 2D medical image as input and may output a BPE assessment 326. In various embodiments, the BPE assessment 326 may be displayed on a display device of the imaging system 308. For example, the BPE assessment 326 may be displayed as a superimposition on the new medical image, such as Figures 9 to 11 shown.

[0058] See now Figure 5 , showing the methods used to train a BPE evaluation model (e.g., Figure 3 The present invention provides an exemplary method 500 for evaluating a BPE level of a breast based on an image of the breast using a BPE assessment model 302 of the present invention. The method 500 may be performed by a training module and / or a model module of a BPE assessment model training system (such as the training module 394 and the model module 392 of the BPE assessment model training system 300). The BPE assessment model training system may be included in an image processing system such as the image processing system 202, and one or more instructions of the method 500 may be executed by a processor of the image processing system (e.g., the processor 204). In various embodiments, the image processing system may be coupled to an imaging system such as the image processing system 202. Figure 1 Imaging system 100.

[0059] Method 500 begins at 502, where method 500 includes acquiring a plurality of medical images of a breast. In some embodiments, the medical image may include a 3D image volume, such as a CE-DBT image volume obtained via a DBT system. In other embodiments, the medical image may include a 2D image, such as a CEM image obtained via a digital mammography system, or a slice / block or CEM synthesized 2D image generated from a CE-DBT image volume. The medical image may also include a CEM biopsy image. In various embodiments, the image may include a breast morphology (LE) image and / or a contrast agent uptake image from an imaging system or from an image dataset (e.g., image dataset 310) generated using an imaging system or obtained via a different source. The morphology image may correspond to an image in which no contrast agent is captured in the image, and the contrast agent uptake image may include an image in which contrast agent is captured in the image. The morphology image may be a LE image, and the contrast agent uptake image may include a REC image generated by a recombination of the LE image and a high-energy x-ray image. The contrast agent uptake image may include a REC image acquired during a contrast-enhanced mammography examination. Acquiring a CEM examination may include administering a contrast agent intravenously prior to acquiring the image, followed by image capture. A processing device, such as a controller of the imaging system, may include instructions for generating a reconstructed image from the low energy image and the high energy image. The reconstructed instructions may include performing a logarithmic weighted subtraction between the low energy image and the high energy image. The contrast uptake image may include a region of interest where contrast media has accumulated and where it may be observed that the contrast of the region is higher than the background contrast of the reconstructed image. The morphology image and the contrast uptake image may be 2D images or 3D images.

[0060] At 504, method 500 includes generating a true BPE assessment for each breast image in the collected breast images. In various embodiments, assigning a true BPE assessment may be performed by a human expert. The human expert may provide a label, such as, for example, minimal / mild / moderate / significant or a score as described above. In addition, the human expert may provide a version of one or more input images in which the area of ​​the BPE is segmented. In some embodiments, the annotations provided by the human expert may be combined to create a true (e.g., overlapping intersections, averages or functions of different scores, most frequently occurring labels, etc.), or the DL may employ multiple real data. At 506, method 500 includes creating a training data set (e.g., training data set 306) including multiple training pairs, wherein each training pair includes at least one or more input images and a corresponding true BPE assessment (e.g., a score). Each training pair may also include a real image in which the BPE corresponding to the input image is segmented. The configuration of the training pair may vary in different embodiments. In some embodiments, the training pair may include a single input image, a single real image in which the BPE is segmented, and / or a single BPE assessment. In other embodiments, the training pairs may include both a morphological breast image and a contrast uptake image of the same breast, BPE segmented true versions of the morphological breast image and the contrast enhanced image, and a single true BPE assessment for both the morphological breast image and the CEM image. In other embodiments, a greater number of images may be used as input images within a single training pair. However, each training pair may include a single true BPE assessment.

[0061] In other embodiments, the training pair may include non-image inputs input into the BPE assessment model, such as clinical information of the patient to whom the breast belongs. Clinical information may improve the accuracy and / or performance of the BPE assessment model during training. For example, clinical information may include metadata and / or timing data (e.g., injection timing, acquisition time) of the image, the patient's menstrual cycle information, the patient's age, the patient's demographic data, the presence or absence of one or more conditions of the patient, or different clinical information. In some embodiments, the additional input input into the BPE assessment model may be a definition of a specified portion of one or more input images to be excluded from the BPE assessment. For example, a suspicious ROI may have been previously detected or identified in one or more input images. Suspicious ROIs may include lesions or artifacts that show contrast agent uptake in the absence of BPE, which may bias the BPE assessment. In order to ensure that the suspicious ROI does not bias the BPE assessment, a specified portion of one or more input images including the suspicious ROI may be included in the training pair, and the specified portion may not be included in the adjustment of network parameters during training. For example, a CAD (computer-aided diagnosis) tool may be used to identify the area of ​​the suspicious ROI, which is capable of detecting and highlighting the area or ROI in the image volume. In one embodiment, the CAD tool is a DL model trained to identify cancerous tissue. CAD may define a set of pixels / voxels that include a suspicious ROI. Based on CAD, the defined set of pixels / voxels may not be included as input into the input layer of the BPE assessment model. In this way, the BPE assessment model may generate a BPE assessment during training based on the portion of the image that does not include the suspicious ROI. In order to ensure that the model is trained to accept CAD during the later inference stage, various training pairs of the training data may include such additional CAD inputs.

[0062] At 508, method 500 includes training a BPE assessment model using the training data set generated at 506 to predict a BPE label or score and / or a BPE segmentation for one or more breast images included in the training pair. In various embodiments, the BPE assessment model can be a deep learning (DL) neural network. In one embodiment, the BPE assessment model is a deep convolutional neural network (CNN). The CNN may include one or more convolutional layers, which in turn include one or more convolutional filters. The convolutional filters may include multiple weights, wherein the values ​​of these weights are learned during the training procedure. The convolutional filters may correspond to one or more visual features / patterns, thereby enabling the BPE assessment model to identify and extract features from the image.

[0063] Training the BPE assessment model may include iteratively inputting one or more input images of the training pair (and associated clinical data in some embodiments) into the input layer of the BPE assessment model. The BPE assessment model propagates the input image from the input layer through one or more hidden layers until it reaches the output layer of the BPE assessment model to generate an output, wherein the output is one or more of a BPE marker, a BPE score, and / or one or more BPE segmented images corresponding to one or more breast images. The BPE marker may indicate the BIRADS BPE level: minimal / mild / moderate / significant. The BPE score may indicate the percentage of breast tissue of the breast in the breast image in which BPE is detected relative to the total surface area or total volume of the breast. The BPE segmented image may indicate an area of ​​the breast or breast image identified as BPE.

[0064] The BPE evaluation model may be configured to iteratively adjust one or more of the multiple weights of the BPE evaluation model so as to minimize the difference between the output of the BPE evaluation model and the true BPE evaluation included in the training pair for each training pair. The difference (or loss) may be back-propagated through the BPE evaluation model to update the weights (and biases) of the hidden (convolutional) layer. In some embodiments, back propagation of loss may occur according to a gradient descent algorithm, wherein the gradient (first-order derivative or approximation of the first-order derivative) of the loss function is determined for each weight and bias of the deep neural network. Then, each weight (and bias) of the BPE evaluation model is updated by adding the negative of the gradient product determined (or approximated) for the weight (or bias) to a predetermined step size. The weights and biases may be repeatedly updated until the weights and biases of the BPE evaluation model converge, or for each iteration of the weight adjustment, the rate of change of the weights and / or biases of the deep neural network is below a threshold. It should also be noted that back propagation is used as an example, and other optimization schemes are effective for fitting the parameters of the BPE evaluation model.

[0065] In order to avoid overfitting, the training of the BPE evaluation model can be interrupted periodically to verify the performance of the BPE evaluation model on the verification image pair, as shown in the reference above. Figure 3 When the performance of the BPE evaluation model on the validation set converges (for example, when the error rate on the validation set converges to a threshold of a minimum value or converges to within a threshold of a minimum value), the training of the BPE evaluation model can be terminated. In this way, the BPE evaluation model can be trained to predict the BPE score of a new breast image during a subsequent inference phase.

[0066] After the BPE assessment model has been trained and validated, the trained BPE assessment model can be stored in a memory of the image processing system for use in a breast examination performed using the imaging system during a subsequent inference phase. For example, the trained BPE assessment model can be stored in an inference module (e.g., inference module 212) of the image processing system.

[0067] See now Figure 6 , shows the use of such as the above reference Figure 5 The BPE evaluation model described (e.g., Figure 3 The trained BPE evaluation model 302 of the BPE evaluation model generates a Figure 1 An exemplary method 600 for BPE assessment of a breast of a patient of an imaging system of the imaging system 100 of the present invention is provided. Generating a BPE assessment may include calculating a BPE level of the breast as a score, wherein the score reflects a percentage of breast tissue of a surface area of ​​a 2D image of the breast in which BPE is detected, or reflects a percentage of breast tissue of a volume of a 3D image of the breast in which BPE is detected. The trained BPE assessment model may also output an image of the breast with BPE segmentation.

[0068] The BPE assessment may be performed in an automated manner, meaning that it is performed automatically based on one or more images of the breast and without human intervention. The method 600 may be performed using an image processing system (e.g., a controller 44) stored in a controller coupled to the imaging system. Figure 2 The method 600 is executed by computer readable instructions in a non-transitory memory of a computing device of the image processing system 202. In some embodiments, the method 600 may be executed by another computing device (e.g., an edge device, a picture archiving and communication system (PACS)) without departing from the scope of the present disclosure.

[0069] Method 600 begins at 602, where method 600 includes receiving one or more medical images of a breast from an imaging system. In some examples, the one or more medical images may be stored in a memory of the imaging system or image processing system (e.g., Figure 2 In other examples, one or more medical images may be received from an imaging device of an imaging system during a breast examination (e.g., method 600 may be performed in real time during a breast examination). In some embodiments, the one or more medical images may include a first contrast agent uptake image such as a reconstructed CEM image or a CE-DBT image volume and a second low-energy image such as a morphology image.

[0070] In some embodiments, the one or more medical images of the received breast may include a previously detected or identified suspicious ROI. The suspicious ROI may include a lesion or artifact that shows contrast agent uptake in the absence of BPE. Therefore, the suspicious ROI may bias the BPE assessment. When a suspicious ROI is present in one or more medical images of the breast, the area of ​​the suspicious ROI may be excluded from the BPE assessment. For example, a CAD tool may be used to identify the area of ​​the suspicious ROI. For example, CAD may define a set of pixels / voxels of the received one or more medical images including the suspicious ROI. Based on CAD, the defined set of pixels / voxels may not be included as input into the BPE assessment model. In this way, the BPE assessment model may output a BPE assessment based on the non-excluded pixels / voxels of the received one or more medical images and without considering the defined set of pixels / voxels including the suspicious ROI.

[0071] At 604, method 600 includes obtaining a predicted BPE assessment for the medical image using the trained BPE assessment model. Obtaining the predicted BPE assessment for the medical image using the trained BPE assessment model may include inputting the medical image into the trained BPE assessment model in a manner similar to that described above with reference to method 500, and receiving the predicted BPE assessment as an output of the trained BPE assessment model.

[0072] At 606, method 600 includes displaying a predicted BPE assessment (e.g., a BPE score or level) output by a BPE assessment model on a display device (e.g., display device 234) of an image processing system. In a preferred embodiment, the predicted BPE assessment can be displayed on the display device in real time, meaning that the inspector is conducting the inspection. In some embodiments, the predicted BPE assessment can be displayed together with (e.g., side by side) the image of the breast input into the BPE assessment model. For example, the predicted BPE assessment can be displayed in a mark superimposed on the image of the breast. Additionally or alternatively, the predicted BPE assessment can be stored in a storage device of the imaging system or a different storage device.

[0073] As an example of the output of the BPE evaluation model, Fig. 9 A first exemplary CEM image 900 of a patient's breast is shown, where the BPE can be seen as a brighter portion 902 of the CEM image 900. A BPE assessment 904 of the breast output by a BPE assessment model is shown superimposed on the CEM image 900. Fig. 9In the embodiment shown in , the BPE assessment 904 classifies the BPE as "moderate" based on the degree of BPE indicated by the lighter portion 902. Specifically, the BPE assessment model can be trained to output a code indicating a classification into a category or a set of different categories such as "minimal", "mild", "moderate" or "significant" based on the ratio of the lighter portion 902 relative to the total surface area (or total volume, for 3D image volume) of the breast. For example, the BPE assessment model can be trained to output a binary string indicating the predicted category, such as 1000 indicating minimal, 0100 indicating mild, 0010 indicating moderate, or 0001 indicating significant, or different types of codes. Alternatively, the BPE assessment model can be trained to output a BPE score (e.g., a percentage), and the score can be converted to one or more classifications by applying a set of rules and / or thresholds to the score.

[0074] Additionally or alternatively, the score may be indicated on the image of the breast. Fig.10 A second exemplary CEM image 1000 of a patient's breast is shown, where the CEM image 1000 may be a BPE segmented image output by a BPE assessment model. Fig.10 , for clarity, the BPE segmented portion 1002 of the CEM image 1000 is shaded. A BPE assessment 1004 of the breast output by the BPE assessment model is shown superimposed on the CEM image 1000, where the BPE assessment 1004 is a BPE score indicating the percentage of the lighter portion 902 relative to the total surface area (or total volume, for 3D image volume) of the breast.

[0075] Back to Figure 6 At 608, method 600 optionally includes determining whether there is an asymmetry between a first BPE of the patient's left breast and a second BPE of the patient's right breast. If the first BPE differs from the second BPE by more than a threshold difference, a user of the imaging system can be alerted. Because BPE asymmetry is associated with breast cancer progression, the detected asymmetry can be highlighted to the user, for example, in the form of a score or a warning, in addition to the calculated BPE level. The presence of BPE asymmetry may affect the level of cancer suspicion and severity, ultimately affecting the patient care pathway. Figure 7 and Figure 8 Describes the determination of whether BPE asymmetry exists.

[0076] In some embodiments, method 600 may optionally include determining whether there is an asymmetry or inconsistency between different images of the same breast or between different types of images. For example, a first BPE score may be generated based on a first image of a breast using a BPE assessment model, and a second BPE score may be generated based on a second image of the same breast using the BPE assessment model. If the difference between the first BPE score and the second BPE score exceeds a threshold difference, the BPE score inconsistency may be displayed on a display device to alert a user of the imaging system.

[0077] At 610, method 600 includes displaying an indication of BPE asymmetry (e.g., whether BPE asymmetry exists) on a display device and / or storing the indication of BPE asymmetry in a storage device, and method 600 ends. In various embodiments, the indication of BPE asymmetry may be displayed next to or superimposed on both the first image of the left breast and the second image of the right breast so that a user of the imaging system (e.g., a radiologist) can compare the BPE in the first image and the second image.

[0078] Fig.11 An exemplary BPE asymmetry display 1100 that may be displayed on a display screen is shown, wherein no BPE asymmetry is detected. The BPE asymmetry display 1100 includes a first medical image 1102 of a left breast of a patient and a second medical image 1104 of a right breast of the patient of an imaging system. Fig.11 , a first BPE 1106 of the left breast shown in the first medical image 1102 and a second BPE 1108 of the right breast shown in the second medical image 1104 are approximately equal, and a first BPE assessment 1110 (e.g., 65%) of the left breast displayed on the first medical image 1102 is equal to a second BPE assessment 1112 (e.g., 65%) of the right breast displayed on the second medical image 1104. Thus, an indication 1114 that no BPE asymmetry is detected is displayed in the exemplary BPE asymmetry display 1100. It should be understood that the configuration of the visual components of the exemplary BPE asymmetry display 1100 is for schematic illustration, and in other embodiments, the first medical image 1102, the second medical image 1104, the first BPE assessment 1110, the second BPE assessment 1112, and the indication 1114 may be included in different locations, sizes, etc. in the exemplary BPE asymmetry display 1100.

[0079] See now Figure 7 , shows the use of such as the above reference Figure 5The BPE assessment model described herein is a method 700 of determining whether there is a BPE asymmetry between a first breast (e.g., a left breast) of a patient in an imaging system and a second breast (e.g., a right breast) of the patient. The imaging system may be Figure 1 The method 700 is a non-limiting example of an imaging system 100 of the present invention. The BPE asymmetry determination can be performed in an automatic manner, meaning that it is performed automatically based on images of the first breast and the second breast and without human intervention. The method 700 can be performed using an image processing system (e.g., a controller 44) stored in a controller coupled to the imaging system. Figure 2 The method 700 may be executed by computer readable instructions in a non-transitory memory of a computing device of the image processing system 202. In various embodiments, the method 700 may be used as described above with reference to Figure 6 A portion of method 600 is described.

[0080] Method 700 begins at 702, where method 700 includes receiving a first image of a left breast of a patient, and at 704, method 700 includes receiving a second image of a right breast of the patient. The first image and the second image may be received from an imaging device of an imaging system, for example, during a breast examination of a patient, or may be received from a storage device of an imaging system or an image processing system. For example, a breast examination may be performed on a patient at a first time, and breast BPE asymmetry may be determined at a later time according to method 700. During a breast examination, a first breast may be imaged, and subsequently a second breast may be imaged as part of the same examination.

[0081] At 706, method 700 includes using a Figure 3 The trained BPE evaluation model 322 obtains a first BPE evaluation (e.g., score) for the first image of the left breast. The first BPE evaluation can be obtained by following the above Figure 6 Similarly, at 708 , method 700 includes obtaining a second BPE estimate (eg, score) for a second image of the right breast using the trained BPE estimate model, also according to method 600 .

[0082] At 710, method 700 includes determining whether a difference between the first BPE score and the second BPE score is greater than a threshold difference. For example, if the first BPE score is 65% and the second BPE score is 55%, the difference is 10%. The threshold difference can be, for example, 5%, where the difference (10%) is greater than the threshold difference (5%), and the answer is yes. Alternatively, if the first BPE score is 65% and the second BPE score is 62%, the difference is 3% (e.g., less than 5%), where the answer is no.

[0083] In some embodiments, the threshold difference can be variable. In one embodiment, the threshold difference can vary with injection time and / or acquisition time. For example, if the acquisition time between the first breast and the second breast is two minutes, the threshold can be set to 5%. If the acquisition time between the first breast and the second breast is 15 minutes, the threshold can be set to 25%. Because the BPE evolves over time and the BPE assessment can depend on the amount of contrast agent uptake when the BPE assessment is performed, the difference between the first BPE assessment and the second BPE assessment can be explained by the relative difference in contrast agent uptake rather than BPE asymmetry. To this end, the asymmetry threshold can vary according to different injection / acquisition timings.

[0084] If, at 710, it is determined that the difference between the first BPE score and the second BPE score is not greater than a threshold difference (e.g., no), method 700 proceeds to 712. At 712, method 700 may include indicating on a display device that BPE asymmetry was not detected, and method 700 ends. In some embodiments, if BPE asymmetry is not detected, no indication may be displayed on the display device. Alternatively, if, at 710, it is determined that the difference between the first BPE score and the second BPE score is greater than a threshold difference (e.g., yes), method 700 proceeds to 714. At 714, method 700 includes indicating BPE asymmetry on a display device of the imaging system, and method 700 ends.

[0085] Figure 8 shows the same use as above reference Figure 5 Description ( Figure 2 The BPE evaluation model (e.g., BPE evaluation model 322) trained by the image processing system of the present invention and based on the BPE evaluation model trained by the image processing system of the present invention such as Figure 1An alternative method 800 is provided for determining whether BPE asymmetry exists between a first breast (e.g., a left breast) of a patient and a second breast (e.g., a right breast) of the patient based on an image generated by an imaging system of an imaging system 100. However, for purposes of method 800, a trained BPE assessment model may be trained to simultaneously receive as input a first image of a first breast and a second image of a second breast, and to output a prediction of whether a first BPE assessment of the first breast is within a threshold difference of a second BPE assessment of the second breast. For example, an image processing system may include a trained BPE assessment model trained to predict a BPE score for a breast in an input image and a trained BPE asymmetry model trained to predict asymmetry between two breasts of the same patient in two different input images. The trained BPE assessment model may be used to generate a single BPE score or classification for breast images, and the trained BPE asymmetry model may be used to determine breast asymmetry as an independent task. The outputs of the trained BPE assessment model and the BPE asymmetry model may be combined and / or displayed together in a common display, such as Fig.11 shown.

[0086] The alternative method 800 may also be performed in an automated manner, meaning that it is performed automatically based on images of the first breast and the second breast and without human intervention. The method 800 may be performed using an image processing system (e.g., a controller 44) stored in a controller coupled to the imaging system. Figure 2 The method 800 may be executed by computer readable instructions in a non-transitory memory of a computing device of the image processing system 202. In various embodiments, the method 800 may be used as described above with reference to Figure 6 A portion of method 600 is described as being performed as an alternative to method 700 .

[0087] Method 800 begins at 802, where method 800 includes receiving a first image of a left breast of a patient, and at 804, method 800 includes receiving a second image of a right breast of the patient. The first image and the second image may be received from an imaging device of an imaging system, for example, during a breast examination of the patient, or may be received from a storage device of an imaging system or an image processing system. For example, a breast examination may be performed on a patient at a first time, and breast BPE asymmetry may be determined at a later time according to method 800. During a breast examination, a first breast may be imaged, and subsequently a second breast may be imaged as part of the same examination.

[0088] At 806, method 800 includes inputting the first image and the second image into a trained BPE asymmetry model, and receiving as an output a prediction of whether breast BPE asymmetry exists. In various embodiments, the BPE asymmetry model may include either or both of injection timing and acquisition timing as input. The BPE asymmetry model may be trained based on a training pair including injection and / or acquisition timing of images of a left breast and a right breast, and the BPE asymmetry model may predict BPE asymmetry based on a first acquisition and / or injection timing of the image of the left breast and a second acquisition and / or injection timing of the image of the right breast. By using acquisition and injection timing information, the accuracy of the BPE asymmetry model in predicting BPE asymmetry may be improved.

[0089] At 808, if breast BPE asymmetry is not predicted, the method 800 proceeds to 810. At 810, the method 800 includes indicating on a display device that BPE asymmetry is not detected, and the method 800 ends. Alternatively, if breast BPE asymmetry is predicted at 808, the method 800 proceeds to 812. At 812, the method 800 includes indicating on a display device of the imaging system that BPE asymmetry is not detected, and the method 800 ends.

[0090] Therefore, a DL model-based method for assessing breast BPE is proposed, wherein the DL model is trained to predict the BPE level of one or both breasts of a patient of an imaging system such as a digital mammography system or a DBT system based on one or more images acquired via the imaging system. The BPE level can be assessed by inputting one or more images into the DL model and receiving the BPE assessment as the output of the model. The one or more images may include one or more images of the same breast, wherein the BPE assessment output by the model may be a score, such as a percentage of BPE detected in any one of the images or portions of the images including the breast. The model may also be trained to output an image in which the BPE of one or both breasts is segmented. By performing breast BPE assessment using a DL model instead of relying on a manual BPE assessment performed by a radiologist, the BPE assessment may be more accurate and / or more consistent between radiologists and / or patients. A more accurate BPE assessment may help radiologists detect lesions and / or distinguish lesions from normal breast tissue. By automatically performing breast BPE assessment using a DL model, the radiologist's workflow may be reduced or made more efficient, thereby reducing the use of imaging system resources and increasing the available time for radiologists to attend to other patients. In this way, the overall process of reading mammograms can be made more efficient, resulting in faster patient results and more consistent and accurate diagnosis of cancer risk, thereby improving patient outcomes.

[0091] Alternatively, the one or more images may include both images of the patient's left breast and the patient's right breast, and the BPE assessment output by the model may be a prediction of whether the BPE levels of the two breasts are asymmetric (e.g., greater than a threshold difference). The model may be trained based on pairs of images of the two breasts. The accuracy of the DL model in detecting asymmetry may be greater than the accuracy of manual determination performed by a radiologist. Accurately detecting asymmetry in the BPE levels of the two breasts can improve the accuracy of the patient's diagnosis while reducing the time spent by the radiologist comparing images of the two breasts.

[0092] The technical effect of automating breast BPE assessment using DL models is that the automated BPE assessment can make it easier for radiologists to detect lesions in the breast and can reduce the radiologist's workflow or make it more efficient, thereby reducing the use of imaging system resources and increasing the radiologist's available time to attend to other patients.

[0093] The present disclosure also provides support for a method, the method comprising: using a deep learning (DL) model trained based on different types of breast medical images, performing an assessment of background parenchymal enhancement (BPE) of a patient's breast based on one or more images of the breast acquired via an imaging system, and displaying the one or more images and the BPE assessment at a display device, wherein the one or more images include at least one of the following: a contrast-enhanced mammography (CEM) image, a contrast-enhanced digital breast tomosynthesis (CE-DBT) image volume, a CEM biopsy image, and a synthetic two-dimensional (2D) image. In a first example of the method, the output of the DL model includes one or more images showing segmentation of the BPE within the breast, and one or more images showing segmentation of the BPE, and the BPE assessment is displayed on the display device. In a second example of the method that optionally includes the first example, performing a BPE assessment on the breast based on one or more images of the breast using the DL model also includes inputting at least two different types of images of the breast into the DL model, and receiving the BPE assessment as an output of the DL model. In a third example of the method, which optionally includes one or both of the first and second examples, the at least two different types of images include a reconstructed (REC) image generated from a low energy (LE) image and a high energy (HE) image. In a fourth example of the method, which optionally includes one or more or each of the first to third examples, the at least two different types of images of the breast include both LE images and REC images. In a fifth example of the method, which optionally includes one or more or each of the first to fourth examples, performing a BPE assessment on the breast based on one or more images of the breast using a DL model also includes inputting additional clinical information into the DL model, and receiving a BPE assessment as an output of the DL model, the additional clinical information including at least one of the following items: metadata of one or more images, timing of contrast agent injection into the patient, acquisition time of one or more images, menstrual cycle information of the patient, age of the patient, demographic data of the patient, presence or absence of one or more conditions of the patient, and a definition of a specified portion of one or more images, which will be excluded from the BPE assessment. In a sixth example of the method that optionally includes one or more or each of the first to fifth examples, the one or more images include one of a 2D CEM image and a three-dimensional (3D) CE-DBT image volume, wherein the BPE assessment of the 2D CEM image is a score indicating a percentage of tissue of the breast that exhibits BPE relative to a surface area of ​​the breast, wherein the BPE assessment of the 3D CE-DBT image volume is a score indicating a percentage of tissue of the breast that exhibits BPE relative to a total volume of the breast. In a seventh example of the method that optionally includes one or more or each of the first to sixth examples, the BPE assessment is a classification of the breast into one of a plurality of categories.In an eighth example of the method that optionally includes one or more or each of the first to seventh examples, performing a BPE assessment on a breast based on one or more images of the breast using the DL model further includes determining whether asymmetry is detected between a first BPE level in a first image of a first breast of the patient and a second BPE level in a second image of a second breast of the patient. In a ninth example of the method that optionally includes one or more or each of the first to eighth examples, determining whether asymmetry is detected between the first BPE level and the second BPE level further includes: inputting a first image of the first breast into the DL model and receiving a first BPE assessment of the first breast as a first output of the DL model, inputting a second image of the second breast into the DL model and receiving a second BPE assessment of the second breast as a second output of the DL model, and in response to a difference between the first BPE assessment and the second BPE assessment being greater than a threshold difference, displaying an indication on a display device that an asymmetry is detected. In a tenth example of the method, which optionally includes one or more or each of the first to ninth examples, the DL model is trained to predict asymmetry between a first BPE level and a second BPE level, and determining whether asymmetry exists between the first BPE level and the second BPE level further includes: inputting a first image of the first breast and a second image of the second breast into the DL model, and receiving the predicted BPE asymmetry between the first BPE level and the second BPE level as an output of the DL model, and displaying an indication that the predicted BPE asymmetry is detected on a display device. In an eleventh example of the method, which optionally includes one or more or each of the first to tenth examples, the method further includes: determining whether asymmetry exists between the first BPE level and the second BPE level based on a first time of acquiring the first image and a second time of acquiring the second image. In a twelfth example of the method, which optionally includes one or more or each of the first to eleventh examples, the method further includes: acquiring a cranio-caudal view (CC) and a mediolateral oblique (MLO) view of the breast during a CEM examination, and generating a BPE assessment of the breast using the DL model based on both the CC view and the MLO view.

[0094] The present disclosure also provides support for an image processing system, which includes: a display device, a processor and a non-volatile memory, the non-volatile memory storing instructions that can be executed by the processor to: receive a first set of one or more medical images of a first breast of a patient acquired via an imaging system, input the first set of one or more medical images into a deep learning (DL) model, the DL model being trained to perform an assessment of a first BPE level of the first breast based on the first set of one or more medical images, receive a first BPE assessment of the first breast as an output of the DL model, the first BPE assessment including at least one of a score and a classification indicating a percentage of glandular tissue of the first breast displaying the BPE relative to a surface or volume of the first breast, and display the first BPE assessment on a display device. In a first example of the system, additional instructions are stored in the non-volatile memory that, when executed, cause the processor to input additional information into the DL model along with the first set of one or more medical images, and generate a first BPE assessment of the first breast based on the first set of one or more medical images and the additional information, the additional information including at least one of the following: metadata of the one or more medical images, timing of injection of contrast agent into the patient, acquisition time of the one or more medical images, menstrual cycle information of the patient, age of the patient, demographic data of the patient, presence or absence of one or more conditions of the patient, and a definition of a specified portion of the one or more medical images that is to be excluded from the first BPE assessment. In a second example of the system that optionally includes the first example, the additional instructions are stored in a non-transitory memory, which when executed cause the processor to: receive a second set of one or more medical images of a second breast of the patient, input the second set of one or more medical images into the DL model to obtain a second BPE assessment of a second BPE level of the second breast based on the second set of one or more medical images, determine whether there is BPE asymmetry between the first BPE level of the first breast and the second BPE level of the second breast, and in response to determining that BPE asymmetry exists, display an indication that BPE asymmetry is detected on a display device. In a third example of the system that optionally includes one or both of the first and second examples, the additional instructions are stored in a non-transitory memory, which when executed cause the processor to determine whether there is BPE asymmetry based on a first timing of a first acquisition of the first set of one or more medical images and a second timing of a second acquisition of the second set of one or more medical images.In a fourth example of the system, optionally including one or more or each of the first to third examples, the one or more medical images include at least one of the following: a reconstructed contrast-enhanced mammography (CEM) image, both a low-energy morphology image and a reconstructed CEM image, a contrast-enhanced digital breast tomosynthesis (CE-DBT) image volume, a CEM biopsy image, a synthesized two-dimensional (2D) image, and both a craniocaudal (CC) view and a mediolateral oblique (MLO) view of the first breast acquired during the CEM examination. In a fifth example of the system, optionally including one or more or each of the first to fourth examples, the output of the DL model includes one or more images showing segmentation of the BPE within the first breast, and one or more images showing segmentation of the BPE, and the BPE assessment is displayed on a display device.

[0095] The present disclosure also provides support for a method for an imaging system, the method comprising: receiving a first set of one or more medical images of a first breast of a patient acquired via the imaging system at a first time, receiving a second set of one or more medical images of a second breast of the patient acquired via the imaging system at a second time, inputting at least the first set of one or more medical images, the second set of one or more medical images, the first time, and the second time into a deep learning (DL) model to obtain an assessment of BPE asymmetry between the first breast and the second breast, displaying at least one image of the first set of medical images of the first breast on a display device of the imaging system, displaying at least one image of the second set of medical images of the second breast on the display device, and displaying an indication of BPE asymmetry on the display device.

[0096] As used herein, the elements or steps listed in the singular and beginning with the word "one" or "a kind of" should be understood as not excluding a plurality of said elements or steps, unless such exclusion is explicitly stated. In addition, the reference to "one embodiment" of the present invention is not intended to be interpreted as excluding the existence of additional embodiments that also include the cited features. In addition, unless explicitly stated to the contrary, "comprising", "including" or "having" an embodiment of an element or multiple elements with a specific characteristic may include additional such elements that do not have the characteristic. The terms "including" and "in..." are used as the concise language equivalents of the corresponding terms "including" and "wherein". In addition, the terms "first", "second" and "third" etc. are only used as marks, and are not intended to impose numerical requirements or specific positional order on their objects.

[0097] This written description uses examples to disclose the invention, including the best mode, and also to enable a person skilled in the relevant art to practice the invention, including making and using any devices or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to a person skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

Claims

1. A method, comprising: Using a deep learning (DL) model trained based on different types of breast medical images, performing an assessment of background parenchymal enhancement (BPE) of a patient's breast based on one or more images of the breast acquired via an imaging system (604); as well as displaying the one or more images and the BPE assessment at a display device (606); wherein the one or more images include at least one of: Contrast-enhanced mammography (CEM) images; Contrast-enhanced digital breast tomosynthesis (CE-DBT) image volumes; one or more slices of the CE-DBT image volume; a synthetic two-dimensional (2D) image generated from the CE-DBT image volume; as well as CEM biopsy image.

2. The method of claim 1, wherein the output of the DL model comprises one or more images showing a segmentation of the BPE within the breast, and the one or more images showing the segmentation of the BPE, and the BPE assessment is displayed on the display device.

3. The method of claim 1 , wherein performing the BPE assessment on the breast based on the one or more images of the breast using the DL model further comprises inputting at least two different types of images of the breast into the DL model and receiving the BPE assessment as an output of the DL model. 4 . The method of claim 3 , wherein the at least two different types of images include a reconstructed (REC) image generated from a low energy (LE) image and a high energy (HE) image.

5. The method of claim 4, wherein the at least two different types of images of the breast include both LE images and REC images.

6. The method of claim 1 , wherein performing the BPE assessment on the breast based on the one or more images of the breast using the DL model further comprises inputting additional clinical information into the DL model and receiving the BPE assessment as an output of the DL model, the additional clinical information comprising at least one of: metadata of the one or more images; timing of injection of contrast agent into said patient; the acquisition time of the one or more images; Menstrual cycle information of the patient; the patient's age; demographics of the patients described; the presence or absence of one or more conditions in the patient; as well as Definition of specified portions of the one or more images, which are to be excluded from the BPE evaluation.

7. The method according to claim 1, wherein: The one or more images include one of the 2D images, wherein the BPE assessment is a score indicating a percentage of tissue of the breast that exhibits BPE relative to a surface area of ​​the breast; and a three-dimensional (3D) image volume, wherein the BPE assessment is a score indicating the percentage of tissue of the breast that exhibits BPE relative to the total volume of the breast.

8. The method of claim 1, wherein the BPE assessment is a classification of the breast into one of a plurality of categories.

9. The method of claim 1, wherein performing the BPE assessment on the breast based on the one or more images of the breast using the DL model further comprises determining whether asymmetry is detected between a first BPE level in a first image of the patient's first breast and a second BPE level in a second image of the patient's second breast (608).

10. The method of claim 9, wherein determining whether the asymmetry is detected between the first BPE level and the second BPE level further comprises: inputting the first image of the first breast into the DL model, and receiving a first BPE estimate of the first breast as a first output of the DL model (706); inputting the second image of the second breast into the DL model, and receiving a second BPE estimate of the second breast as a second output of the DL model (708); and In response to a difference between the first BPE estimate and the second BPE estimate being greater than a threshold difference, displaying an indication on the display device that the asymmetry is detected (714).

11. The method of claim 9, wherein the DL model is trained to predict the asymmetry between the first BPE level and the second BPE level, and determining whether the asymmetry exists between the first BPE level and the second BPE level further comprises: inputting the first image of the first breast and the second image of the second breast into the DL model, and receiving as an output of the DL model a predicted BPE asymmetry between the first BPE level and the second BPE level (806); and An indication that the predicted BPE asymmetry is detected is displayed on the display device (812).

12. The method of claim 9, further comprising determining whether the asymmetry exists between the first BPE level and the second BPE level based on a first time when the first image is acquired and a second time when the second image is acquired.

13. The method of claim 1 , further comprising acquiring a craniocaudal view (CC) and a mediolateral oblique (MLO) view of the breast during a CEM examination, and generating a BPE assessment of the breast using the DL model based on both the CC view and the MLO view.

14. An image processing system (202), the image processing system comprising: Display device (234); A processor (204) and a non-transitory memory (206) storing instructions executable by the processor (204) to: receiving a first set of one or more medical images of a first breast of a patient acquired via an imaging system (308); inputting the first set of one or more medical images into a deep learning (DL) model (322), the DL model being trained to perform an assessment of a first BPE level of the first breast based on the first set of one or more medical images; receiving as an output of the DL model (322) a first BPE assessment (326) of the first breast, the first BPE assessment (326) comprising at least one of a score (1004) and a classification (904) indicating a percentage of glandular tissue of the first breast exhibiting BPE relative to a surface or volume of the first breast; and The first BPE assessment (326) is displayed on the display device (234).

15. The image processing system (202) of claim 14, wherein further instructions are stored in the non-transitory memory (206), which when executed cause the processor (204) to input additional information into the DL model (322) along with the first set of one or more medical images, and to generate the first BPE assessment (326) of the first breast based on the first set of one or more medical images and the additional information, the additional information comprising at least one of the following: metadata of the one or more medical images; timing of injection of contrast agent into said patient; the acquisition time of the one or more medical images; Menstrual cycle information of the patient; the patient's age; demographics of the patients described; the presence or absence of one or more conditions in the patient; as well as Definition of specified portions of the one or more medical images, which specified portions are to be excluded from the first BPE evaluation.

Images