Training machine learning models for use in medical image processing applications based on a combination of incomplete sample sets and simulated sample images derived from them.

Abstract

A solution is proposed for training machine learning models for use in medical imaging applications. The method involves providing an incomplete sample set for each imaging procedure, each sample set containing a sample target image corresponding to a sample target dose of contrast agent and a sample baseline image. From each incomplete sample set, a sample source image is simulated that mimics a sample source dose of contrast agent lower than the sample target dose. A complete sample set is generated for each imaging procedure by combining the simulated sample source image from other incomplete sample sets with the incomplete sample set itself. The generated complete sample set is used to train a machine learning model and optimize its capabilities. Furthermore, a method for using the machine learning model in medical imaging applications is proposed. A computer program implementing the method is also proposed.

Description

[Technical Field]

[0001] This disclosure relates to the field of machine learning. More specifically, this disclosure relates to machine learning for use in medical image processing applications. [Background technology]

[0002] The background to this disclosure is presented through a discussion of technology relevant to its context. However, even if this discussion refers to documents, actions, artifacts, etc., it does not imply or represent that the technology discussed is part of the prior art or general knowledge in the field relevant to the current disclosure.

[0003] In the medical field, imaging technology is commonly used, allowing physicians to visually represent parts of a patient's body as images, enabling observation and examination (usually in a nearly non-invasive manner, even when the body part cannot be directly seen). For this purpose, during imaging diagnostic procedures, it is common to administer contrast agents to the patient to enhance the contrast of the target (biological) of interest, such as a lesion, making it more prominent in the image.

[0004] In this context, the use of "low-dose contrast imaging," which involves reducing the amount of contrast agent administered, has also been proposed. Low dose refers to an amount less than the "standard dose" that is commonly used in clinical practice. To achieve this, in the imaging diagnostic procedure, a "zero-dose" image of the body part is first acquired before contrast agent administration. Then, after administering a low dose of contrast agent, one or more "low-dose images" of the same body part are acquired. Next, a "standard-dose image," equivalent to what would occur if the patient were administered a standard dose of contrast agent, is simulated using a deep learning network (DLN) from the zero-dose image and the corresponding low-dose image. This deep learning network restores the lack of contrast enhancement in the low-dose image to the desired level that should be achievable with a standard dose of contrast agent. The deep learning network is trained using a sample set consisting of the following: a zero-dose image acquired before contrast agent administration, a low-dose image acquired after low-dose contrast agent administration, and a standard-dose image acquired after standard contrast agent administration. These are all obtained from the same type of body part, from the same patient (or from two or more zero-dose images acquired under different imaging conditions, or from two or more low-dose images acquired at different low doses).

[0005] However, the number of (zero-dose / low-dose / normal-dose) image sample sets available for training deep learning networks is relatively small. This is mainly because low-dose images are not typically obtained in standard clinical practice (because imaging procedures need to be modified to administer multiple doses of low-dose and normal-dose contrast agents). The small sample set size degrades the training quality of deep learning networks. This negatively impacts the network's robustness, particularly its ability to predict normal-dose images.

[0006] Simulations using low-dose images have also been proposed.

[0007] For example, Patent Document 1 (International Publication No. 2021 / 061710) discloses including an augmented dataset obtained by simulation as training input data for a deep learning network used to improve the quality of images acquired with low doses of contrast agents. For this purpose, image data from a clinical database can be used to generate low-quality image data that mimics image data acquired with low doses of contrast agents. This result is obtained by adding artifacts to the original image data.

[0008] On the other hand, Non-Patent Literature 1 (Johannes Haubold et al., "Contrast agent dose reduction in computed tomography with deep learning using a conditional generative adversarial network, European Radiology (2021) 31:6087-6095") discloses a method for simulating reduced images of iodine-based contrast agents (ICM) and verifying the possibility of virtually enhancing ICM. For this purpose, dual-energy CT images based on ICM are acquired, and "ICM-separated images" are generated by encoding the distribution of ICM. These are used to create virtual non-contrast (VCN) images. Dual-energy CT images corresponding to 50% or 80% reduction of ICM are simulated by proportional subtraction. Input image and target image pairs are obtained by combining the reduced ICM image and the ICM-separated image with the VCN image, respectively. Then, a generative adversarial network (GAN) is trained using this input / target image pair to simulate and verify ICM enhancement. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] International Publication No. 2021 / 061710 [Non-patent literature]

[0010] [Non-Patent Document 1] The paper by Johannes Haubold et al., "Contrast agent dose reduction in computed tomography with deep learning using a conditional generative adversarial network, European Radiology (2021) 31:6087-6095" [Overview of the project] [Problems that the invention aims to solve]

[0011] However, simulating normal-dose images using deep learning networks from zero-dose images and corresponding low-dose images remains challenging. In particular, the resulting results may suffer from problems such as artifacts, non-uniform textures, and variations in signal intensity. [Means for solving the problem]

[0012] A simplified summary of this disclosure is provided below. While this is intended to provide a basic understanding, its sole purpose is to briefly introduce some of the concepts of this disclosure and serve as a prelude to the more detailed explanation that follows. Therefore, it should not be interpreted as identifying or defining the scope of any significant elements of this disclosure.

[0013] Generally speaking, this disclosure is based on the idea of ​​reducing (decorating) the correlation of images used to train machine learning models.

[0014] In particular, as one aspect, a method for training a machine learning model used in a medical image processing application is provided. This method includes preparing an incomplete sample set for each imaging diagnosis procedure. This incomplete sample set includes a sample target image (one or more) corresponding to a predetermined target dose of a contrast agent and a sample baseline image (one or more). From each incomplete sample set (or a part thereof), a sample source image (one or more) that mimics a contrast agent at a predetermined source dose lower than the target dose is simulated. Then, by combining the sample source images simulated from other incomplete sample sets with the incomplete sample sets, one or more complete sample sets are generated for each imaging diagnosis procedure. The machine learning model is trained using this complete sample set.

[0015] As yet another aspect, a method for using a machine learning model in a medical image processing application is provided.

[0016] As yet another aspect, a computer program for implementing each method is provided.

[0017] As yet another aspect, a corresponding computer program product is provided.

[0018] As yet another aspect, a computing system for implementing each method is provided.

[0019] As yet another aspect, a corresponding medical method is provided.

[0020] [[ID=,24]]More specifically, one or more aspects of the present disclosure are described in the independent claims, and the advantageous features thereof are described in the dependent claims. The language of all claims is incorporated herein by literal reference (the advantageous features provided in relation to a particular aspect are equally applicable to all other aspects as necessary).

Brief Description of the Drawings

[0021] The solutions of this disclosure, and their further features and advantages, will be best understood by referring together with the detailed description below and the accompanying drawings. The description herein is provided as an example without limitation, and for the sake of brevity, corresponding elements are denoted by the same or similar reference numerals, and their descriptions are not repeated. In addition, the names of each component are generally used to indicate both their type and attributes (value, content, expression, etc.). [Figure 1] Figure 1 shows a schematic block diagram of infrastructure that may be used to implement a solution according to one embodiment of the present disclosure. [Figure 2] Figure 2 illustrates the general principle of a solution according to one embodiment of the present disclosure. [Figure 3] Figure 3 shows the main software components that may be used to implement the solution according to one embodiment of the present disclosure. [Figure 4A] Figures 4A to 4D show activity diagrams illustrating the flow of activities related to the implementation of a solution according to one embodiment of this disclosure. [Figure 4B] Figures 4A to 4D show activity diagrams illustrating the flow of activities related to the implementation of a solution according to one embodiment of this disclosure. [Figure 4C] Figures 4A to 4D show activity diagrams illustrating the flow of activities related to the implementation of a solution according to one embodiment of this disclosure. [Figure 4D] Figures 4A to 4D show activity diagrams illustrating the flow of activities related to the implementation of a solution according to one embodiment of this disclosure. [Figure 5A] Figures 5A to 5C show representative experimental results for a solution according to one embodiment of the present disclosure. [Figure 5B] Figures 5A to 5C show representative experimental results for a solution according to one embodiment of the present disclosure. [Figure 5C] Figures 5A to 5C show representative experimental results for a solution according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0022] In particular, referring to Figure 1, a schematic block diagram of infrastructure 100 that may be used to implement a solution according to one embodiment of the present disclosure is shown.

[0023] Infrastructure 100 includes the following components:

[0024] One or more (medical) imaging systems 105 each comprise a corresponding scanner 110 and a control computing system (hereinafter simply referred to as control computer 115). Each scanner 110 administers a contrast agent to enhance the contrast of a corresponding (biological) target, such as a lesion, and acquires an image representing the body part of the subject containing that target during the performance of the corresponding (medical) imaging diagnostic procedure. The subject may be an animal in a preclinical trial or a human in a clinical application. For example, if scanner 110 is an MRI scanner (not shown in the figure), it comprises a gantry for housing the subject, containing a superconducting magnet that generates a very strong static magnetic field, multiple sets of tilt coils that adjust the static magnetic field in different axial directions, and an RF coil for applying magnetic pulses to a specific body part and receiving a corresponding response signal. Separately, if scanner 110 is a CT scanner (also not shown in the figure), it comprises a gantry for housing the subject, containing an X-ray generator, an X-ray detector, and motors that rotate these around the body part of the subject. A corresponding control computer 115, such as a PC, is used to control the operation of the scanner 110. For example, if the scanner 110 is an MRI, the control computer 115 is located outside the scanner room used to shield the scanner 110 and connected by a cable running through a through panel. On the other hand, if it is a CT scanner, the control computer 115 is located near the scanner.

[0025] The image processing system 105 is installed in one or more facilities. For preclinical imaging diagnostic procedures, this would be a research facility such as a university, and for clinical imaging diagnostic procedures, it would be a medical facility such as a hospital. Each (research / medical) facility includes one or more image processing systems 105. Furthermore, a facility may include a central computing system (i.e., a central server 120), which communicates with the control computer 115 of the facility's image processing systems 105 via a network 125 such as the facility's LAN. The central server 120 collects information on imaging diagnostic procedures performed by at least some of the image processing systems 105 from the control computer 115 via the network 125. In particular, for each imaging diagnostic procedure, this includes a sequence of images representing the corresponding body part, and additional information related to the imaging diagnostic procedure, such as subject identification information, the results of the imaging diagnostic procedure, and acquisition parameters of the imaging diagnostic procedure.

[0026] The configuration computing device 130 (configuration computer 130) is used to configure the control computer 115 of the image processing system 105. The configuration computer 130 can communicate with the facility's central server 120 via a network 135 such as the Internet. The configuration computer 130 anonymously collects information from the corresponding central server 120 regarding imaging diagnostic procedures performed at least at some facilities (alternatively, although not shown, the same information can also be collected from the control computer 115 via the network 135 or via a removable storage medium such as a USB from the central server 120 and / or the control computer 115). The information thus collected by the configuration computer 130 is used to configure the control computer 115 of at least some of the image processing systems 105, either on-site after shipment to the facility or at the factory before shipment to the facility (details below).

[0027] Each computer implementing the control computer 115, the central server 120, and the configuration computer 130 comprises multiple units interconnected by a bus structure 140. In particular, one or more microprocessors (μPs) 145 provide the logic processing functions for computers 115, 120, and 130. Non-volatile memory (ROM) 150 stores the basic code for bootstrapping computers 115, 120, and 130, while volatile memory (RAM) 155 is used as working memory by the microprocessor 145. Computers 115, 120, and 130 are equipped with mass storage devices 160 (e.g., SSDs) for storing programs and data. Furthermore, computers 115, 120, and 130 are equipped with multiple controllers 165 for peripheral devices or input / output (I / O) devices. Peripherals related to this disclosure include a keyboard, mouse, monitor, network adapter (NIC) for connecting to corresponding networks 125, 135, a drive for reading and writing removable storage devices (such as USB memory), and, in each control computer 115, a trackball and corresponding drive for the associated unit of its scanner 110.

[0028] Next, referring to Figure 2, the general principles of a solution according to one embodiment of the present disclosure are shown.

[0029] This concerns training a (work-in-progress) machine learning model used in medical imaging applications to simulate increasing doses of contrast agents administered to patients. For this purpose, sample images are collected from multiple sample imaging procedures on sample body parts of a sample subject. Specifically, multiple incomplete sample sets are prepared for each imaging procedure. Each incomplete sample set includes one or more sample target images and one or more sample baseline images. The sample target images represent the corresponding body part of a subject administered a sample target dose of contrast agent. The sample baseline images, on the other hand, represent the corresponding body part under different conditions (e.g., without contrast agent). For each incomplete sample set (or part thereof) of each imaging procedure, one or more sample source images are simulated (or synthesized) from other sample images, i.e., sample baseline images and sample target images (e.g., analytically). The sample source images represent the body part that would appear if the corresponding subject were administered a sample source dose of contrast agent. The sample source dose is less than the sample target dose, and the ratio between the two is equal to a reduction factor of less than 1 (e.g., 1 / 10).

[0030] In a solution according to one embodiment of the present disclosure, one or more complete sample sets are generated for each imaging procedure. Each complete sample set consists of a sample baseline image and a sample target image included in an incomplete sample set for that imaging procedure, and a sample source image simulated from another (different) incomplete sample set for the same imaging procedure.

[0031] The complete sample set is then used to train a machine learning model. This is to optimize the ability to predict the sample target image (ground truth data) for each complete sample set from the sample baseline image and sample source image of that complete sample set. For example, a portion of the complete sample set can be used as the training set for the machine learning model, and another portion as the validation set.

[0032] Machine learning models trained in this manner can be used in various ways in medical image processing applications. In particular, in each (operational) imaging procedure, the corresponding scanner acquires (operational) images representing the (operational) body parts of the (operational) patient being examined. These operational images include one or more operational baseline images and one or more operational administration images. Operational administration images are acquired from patients who have been administered an operational dose of contrast agent. On the other hand, operational baseline images are acquired under different conditions (e.g., without contrast agent administration). The control computer associated with the scanner uses the machine learning model to generate (or synthesize) one or more operational simulation images from the operational baseline images and the corresponding operational administration images. The operational simulation images represent body parts that mimic the case where the patient has been administered an operational simulation dose of contrast agent. The operational simulation dose is greater than the operational dose, and the ratio is equal to the upscaling coefficient (e.g., equivalence) which is the reciprocal of the downscaling coefficient of the machine learning model. Therefore, the operational dose and operational administration image are also called "operational low dose" and "operational low dose image," respectively, while the operational simulation dose and operational simulation image are also called "operational high dose" and "operational high dose image," respectively. Subsequently, the representation of body parts based on the operational simulation image is output (for example, by display) to the physician in charge of the image diagnosis procedure.

[0033] The above solution significantly improves the training of machine learning models and positively impacts their robustness. This is because the proposed combination of "sample source images simulated from different sample baseline / target images" and "sample baseline / target images" eliminates (or at least significantly reduces) their correlation. As a result, the machine learning model is prevented from learning unwanted features (e.g., stochastic fluctuations) that might be transferred to the operational simulation images. This reduces artifacts, non-uniform textures, and / or signal intensity variations in the operational simulation images.

[0034] As a result, machine learning models can predict simulated images in operation with relatively high accuracy. This has a beneficial effect on the quality of medical image processing applications, for example, significantly reducing the risk of false positives / false negatives and incorrect follow-up in diagnostic applications, reducing the risk of reduced treatment effectiveness and damage to healthy tissue in therapeutic applications, and reducing the risk of incomplete lesion resection or over-resection of healthy tissue in surgical applications.

[0035] The solutions described above can be directly applied to sample images obtained in preclinical imaging procedures. Furthermore, they can be extended to sample images obtained in (future) clinical imaging procedures by simply acquiring one or more additional images, without affecting standard clinical practice (i.e., contrast agent dosage and injection protocol).

[0036] In particular, sample baseline images may be obtained from subjects who have never received contrast agent, or from subjects for whom sufficient time has elapsed since previous contrast agent administration to ensure that the contrast agent has been almost completely eliminated from the body. In this case, since there is no contrast agent (or at least no significant amount) in the corresponding body part, these sample baseline images are called sample zero-dose images. The sample target dose of contrast agent may be equal to the value used as standard in clinical practice (sample target images are obtained at any point after contrast agent administration and may be the same as or different from the standard value), in which case the sample target dose and sample target image are called full-dose and sample full-dose images, respectively. Therefore, the sample source dose of contrast agent is less than the value used as standard in clinical practice, in which case the sample source dose and sample source image are called low-dose and sample low-dose images, respectively.

[0037] For simplicity, the following will refer to the implementation of the sample baseline image, sample target dose, sample target image, sample source dose, and sample source image (although the same concept applies to other implementations).

[0038] For example, Figure 2 shows a simple scenario in which two incomplete sample sets 205a and 205b are prepared for the same imaging diagnostic procedure. Incomplete sample set 205a includes a zero-dose sample image 210a and a full-dose sample image 215a. Incomplete sample set 205b includes a zero-dose sample image 210b and a full-dose sample image 215b. A low-dose sample image 220a is simulated from incomplete sample set 205a (zero-dose sample image 210a and full-dose sample image 215a), and a low-dose sample image 220b is simulated from incomplete sample set 205b (zero-dose sample image 210b and full-dose sample image 215b). The complete sample set 225a is generated by adding the low-dose sample image 220b simulated from incomplete sample set 205b to the zero-dose sample image 210a and full-dose sample image 215a from incomplete sample set 205a. On the other hand, the complete sample set 225b is generated by adding the sample zero-dose image 210b and sample full-dose image 215b from the incomplete sample set 205b to the sample low-dose image 220a simulated from the incomplete sample set 205a.

[0039] Regarding the use of machine learning models in medical image processing applications, as mentioned above, operational baseline images may be obtained from patients who have never received contrast agents, or from patients for whom sufficient time has elapsed since previous contrast agent administration, ensuring that the contrast agent has been almost completely eliminated from the body. In this case, since there is no contrast agent (or at least no significant amount) in the corresponding body part, these operational baseline images are called operational zero-dose images.

[0040] When the operational dose and operational simulation dose of contrast agent are equal to the low dose and full dose, respectively, the operational administration image and operational simulation image are called the operational low-dose image and operational full-dose image. In this case, it is possible to reproduce the contrast enhancement normally obtained with full-dose contrast agent administration (particularly useful when full-dose administration is dangerous for the patient), and furthermore, it is possible to improve contrast by reducing artifacts caused by motion and aliasing that may occur when the actual full dose is administered.

[0041] Furthermore, the operational dose of contrast agent may be equal to the full dose, while the operational simulation dose may be even more enhanced (boosted). In this case, the operational administration image is called the operational full dose image, and the operational simulation dose and operational simulation image are called the operational boosted dose and operational boosted dose images, respectively. This allows the operational boosted dose image to enhance contrast as if the contrast agent had been administered at a (virtual) dose exceeding the amount actually administerable in current clinical practice (particularly useful when contrast enhancement is insufficient), without affecting standard clinical practice. As a further improvement, operational composite images can be generated by applying HDR (High Dynamic Range) technology to the operational baseline image, operational full dose image, and corresponding operational boosted dose image (for example, by giving more emphasis to the contribution of the boosted dose image). This allows the target to stand out more while being appropriately positioned within the morphological context of the body part.

[0042] Furthermore, it is possible to have multiple versions of the machine learning model trained with different downscaling factors. In this case, the medical image processing application can select a value from the corresponding upscaling factors. This increases flexibility, allowing physicians to see the effects of different upscaling factors in real time and select the one that provides the best contrast enhancement.

[0043] Referring to Figure 3, the main software components that may be used to implement the solution according to one embodiment of the present disclosure are shown.

[0044] All software components (programs and data) are collectively referred to as reference number 300. The software components 300 are typically stored in mass storage and, at program execution time, are loaded (at least partially) into the working memory of the configuration computer 130, along with the operating system and other application programs (omitted in the diagram for brevity) that are not directly related to the solutions of this disclosure. Programs are initially installed into mass storage, for example, from removable storage or a network. In this regard, each program may be a module, segment, or part of code containing one or more executable instructions for implementing a specified logical function.

[0045] The configuration computer 130 stores a copy of the machine learning model to be trained. For example, this machine learning model is implemented as the (operational) neural network 305 (we will refer to this operational neural network 305 below, but the same concept applies to other implementations).

[0046] Basically, machine learning is used to perform a specific task (in this case, generating operational simulation images) by automatically inferring a method from examples without using explicit procedures (by leveraging a corresponding model learned from examples). In one embodiment of this disclosure, deep learning techniques, a subfield of machine learning based on deep neural networks, are applied. A neural network is a data processing system inspired by the workings of the human brain. A neural network has basic processing elements (neurons) that perform operations based on corresponding weights, and neurons are connected to each other by unidirectional channels (synapses) to transmit data. Neurons are organized into layers that perform different operations and always have an input layer that receives input data and an output layer that provides output data. Deep neural networks, in particular, have one or more hidden layers arranged sequentially along the processing direction between the input and output layers. In one embodiment of this disclosure, this neural network is a convolutional neural network (CNN), and one or more of its hidden layers perform (cross)convolution operations. In particular, this neural network is an autoencoder (encoder-decoder) type convolutional neural network, comprising an encoder that compresses data into a denser form (so-called latent space) and a decoder that performs the desired processing using the compressed data and expands the result into the required form. More specifically, the input layer is configured to receive two input images (a sample baseline image and a sample source image during training, and a baseline image during operation and an image administered during operation in medical image processing applications). The encoder consists of three groups, each consisting of three convolutional layers, followed by a corresponding max pooling layer. Similarly, the decoder consists of three groups, each consisting of three convolutional layers, followed by a corresponding upsampling layer. Each convolutional layer performs a convolution operation using a convolution matrix (filter or kernel) defined by the corresponding weights. This operation is performed sequentially on a limited region (receptive field) of the application data, applying the filter by moving it by a predetermined number of cells (stride). In some cases, zero-value cells (padding) are added to the boundary of the application data to enable the application of the filter.Subsequently, batch normalization (fixing the mean and variance of the corresponding data) is performed, and an activation function that introduces nonlinearity is applied. For example, each convolutional layer uses a 3x3 filter, applied with padding of 1 and stride of 1, and each neuron applies the ReLU (Rectified Linear Unit) activation function. Each max pooling layer is a pooling layer (downsampling of the applied data), replacing the values ​​in each region (window) of the applied data with a single value (in this case, the maximum value), and applying this while moving the window by a predetermined number of cells (stride). For example, each max pooling layer has a 2x2 window and a stride of 1. Each upsampling layer is an amplifying layer (the inverse operation of pooling), expanding each value to the surrounding region (window). For example, when using the max amplifying method, the value is placed where the maximum value was during downsampling, and the surrounding area is filled with zeros. Each upsampling layer has a 2x2 window. Bypass connections are added between the symmetrical encoder and decoder layers (to prevent resolution degradation), and skip connections are added within each convolutional layer group and between the input and output layers (to focus on the differences between input images). The output layer generates an output image corresponding to the input images (a sample target image during training, and a simulated image during operation in medical image processing applications). For example, this is generated by adding the obtained result (with contrast enhancement between input images increased to the desired level) to the sample / operational baseline image.

[0047] The operational neural network 305 loads an operational configuration repository 310 that defines one or more (operational) configurations of the operational neural network. For example, the operational configuration repository 310 has entries corresponding to each configuration of the operational neural network 305, and these entries store the configuration of the operational neural network 305 (defined by its weights) and the upscaling coefficients provided when operating with that configuration.

[0048] The sample image repository 315 contains information about sample images used to train the operational neural network 305. For example, the sample image repository 315 has an entry corresponding to each (sample) imaging diagnostic procedure, and that entry stores multiple sample images acquired or (as described below) generated in that imaging diagnostic procedure. Each sample image is defined by a corresponding bitmap, i.e., a matrix of cells (e.g., 512 rows x 512 columns), where each cell stores a pixel value representing the corresponding location of a body part in the imaging diagnostic procedure (in the case of a 3D sample image, voxels that make up the basic volume). Each voxel value defines brightness (e.g., grayscale) based on the (signal) intensity of the response signal associated with that location. For example, in the case of an MRI scanner, the response signal represents the response to the magnetic field applied to that location, and in the case of a CT scanner, the response signal represents the attenuation of X-rays irradiated to that location. Furthermore, the entry also stores one or more acquisition parameters related to the acquisition of the sample image. In particular, the acquisition parameters include external parameters related to the scanner settings used to acquire the sample image and internal parameters related to the corresponding body part (e.g., the average value of the major tissue of the body part). The collector 320 collects acquired sample images from a central server or control computer in a research or medical facility (not shown). Sample images for at least some imaging procedures are incomplete and include only zero-dose and full-dose sample images of the corresponding body part. In some cases, sample images for one or more imaging procedures are complete and may also include low-dose sample images of the corresponding body part. The collector 320 writes these sample images to the sample image repository 315. The preprocessor 325 preprocesses the sample images for each imaging procedure as needed (e.g., alignment, denoising). Furthermore, the preprocessor can also reduce or expand the sample images by generating one or more (generated) sample images (details below). The preprocessor 325 reads from and writes to the sample image repository 315.

[0049] The creator 330 creates multiple incomplete sample sets for each imaging diagnostic procedure by combining incomplete sample images (or parts thereof). Each set consists of at least one zero-dose sample image and at least one full-dose sample image. The creator 330 reads the sample image repository 315 and writes it to the incomplete sample set repository 335, which contains information about the incomplete sample sets. For example, the incomplete sample set repository 335 has an entry corresponding to each imaging diagnostic procedure that provides incomplete sample images, and that entry stores the corresponding incomplete sample set. Each set includes bitmaps of its zero-dose sample image and full-dose sample image. Furthermore, the entry indicates the acquisition parameters for the corresponding imaging diagnostic procedure (for example, a pointer to the relevant location in the sample image repository 315).

[0050] The analysis engine 340 simulates (or synthesizes) at least one low-dose sample image from the zero-dose / full-dose images of each incomplete sample set (or part thereof). The analysis engine 340 has a user-operable interface. The analysis engine 340 reads the incomplete sample set repository 335 and the simulation formula repository 345 which stores one or more simulation formulas used to simulate the low-dose sample images.

[0051] For example, in the case of an MRI scanner, when spin echo is selected as the operating mode, the signal intensity that defines each voxel value of the sample image (which is given by the transverse component of the magnetization at the corresponding location of the body part when the spins of protons in water molecules return to equilibrium after the application of magnetic pulses by the RF coil) is expressed by the following signal law.

number

number

number

number

number

number

number

Equation

[0052] From the above, the simulation formula is as follows.

Equation

Equation

Equation

Equation

[0053] Similarly, in the case of a CT scanner, the signal intensity (which is given by the X-ray radiation remaining after attenuation when passing through the corresponding position) defining each voxel value of the sample image is represented by the following signal rule.

Equation

number

number

number

number

number

number

[0054] Based on the above, the simulation formula is as follows.

number

number

number

number

[0055] The proposed implementation (a method of deriving the simulation formula by linearizing the signal law with respect to the local concentration of the contrast agent) is computationally very simple, and the reduction in accuracy of the low-dose sample images obtained by this method (due to the linearization of the signal law) is within an acceptable range for the purpose of training the neural network in operation.

[0056] Alternatively, the signal law can be approximated as a function of the local concentration of the contrast agent using a higher-order Taylor expansion (second-order, third-order, etc.). In this case, solving the equation obtained for the local concentration of the contrast agent at full dose yields multiple solutions, requiring evaluation to exclude those that are not physically meaningful. This method improves the accuracy of the simulated low-dose sample images, with higher order approximations resulting in higher accuracy. Yet another method involves numerically solving the signal law for the local concentration of the contrast agent at full dose (again, evaluation to exclude physically meaningless solutions is necessary). This further improves the accuracy of the simulated low-dose sample images.

[0057] The analysis engine 340 writes to the sample low-dose image repository 350, which stores bitmaps of simulated sample low-dose images. Each image is associated with an incomplete sample set in the repository 335 used to simulate it, for example, by a pointer or colocation information. Optionally, a noise corrector 355 corrects noise in the sample low-dose images. The noise corrector 355 reads from and writes to the sample low-dose image repository 350. In practice, the sample zero-dose and sample full-dose images of each incomplete sample set contain noise, and this noise propagates to the corresponding sample low-dose images according to the simulation formula. However, the noise obtained in this way (simulation noise) differs slightly from the statistical distribution of the noise obtained when actually administering a low dose of contrast agent (real noise). In particular, the noise in the sample zero-dose image and the noise in the sample full-dose image are thought to have a zero-mean normal distribution, with their respective standard deviations propagating to the sample low-dose images according to the law of error (or uncertainty) propagation.

number

number

number

[0058] However, the inventors obtained the (theoretical) σ in this way. artificial We found that better results could be obtained by increasing the value of by an empirically determined correction factor. This correction factor is, for example, 1.5 to 2.5, preferably 1.7 to 2.3, and more preferably 1.9 to 2.1 (e.g., 2.0).

[0059] When artificial noise is injected in a multiplicative / convolutional form, the inventors calculate the standard deviation σ based on the value obtained above when artificial noise is injected in an additive form. artificial We found that this can be set. For example, this value can be calculated by multiplying it by an (empirical) conversion coefficient, which is 0.05 to 2.00, preferably 0.1 to 1.0, and more preferably 0.3 to 0.7 (e.g., 0.5).

[0060] For each imaging procedure (or part thereof), the combiner 360 generates one or more complete sample sets (each set consisting of at least one zero-dose sample image, a full-dose sample image, and a low-dose sample image). In particular, if the imaging procedure provides incomplete sample images, the combiner 360 generates a complete sample set by combining the incomplete sample set with a low-dose sample image simulated from other incomplete sample sets of the imaging procedure. On the other hand, if the imaging procedure provides complete sample images, the combiner 360 generates a complete sample set by directly combining the sample images of the imaging procedure. To this end, the combiner 360 reads the incomplete sample set repository 335 and the low-dose image repository 350 for imaging procedures that provide incomplete sample images, and reads the sample image repository 315 for imaging procedures that provide complete sample images. Furthermore, the combiner 360 writes to the complete sample set repository 365, which contains information about the complete sample set. For example, the complete sample set repository 365 has an entry corresponding to each imaging diagnostic procedure, and that entry stores the corresponding complete sample set, each set containing bitmaps of zero-dose sample images, full-dose sample images, and low-dose sample images.

[0061] Alternatively, or additionally, low-dose images of incomplete sample sets may be simulated (or synthesized) by an additional (training) machine learning model. For example, this training machine learning model may be implemented as a training neural network 370, which is in particular an autoencoder-type convolutional neural network as described above (we will refer to this training neural network 370 below, but similar ideas apply to other implementations). The training neural network 370 is controlled by the analysis engine 340. The training neural network 370 reads the training configuration repository 375 (i.e., its weights) which stores the (training) configuration, reads the incomplete sample set repository 335, and writes to the low-dose image repository 350.

[0062] The training engine 380 trains the operational neural network 305 and (if available) the training neural network 370. The training engine 380 reads the complete sample set repository 365 and writes it to the operational configuration repository 310 of the operational neural network 305. Furthermore, if the training neural network 370 is available, the training engine 380 also writes to the training configuration repository 375, the low-dose sample image repository 350, and the complete sample set repository 365 of the training neural network 370.

[0063] Referring to Figures 4A to 4D, an activity diagram is shown illustrating the flow of activities related to the implementation of a solution according to one embodiment of the present disclosure.

[0064] In particular, this figure shows an example process that can be used to train a working neural network by Method 400. In this regard, each block may correspond to one or more executable instructions for implementing a specified logical function on the configuration computer.

[0065] The process begins at the black starting circle 401 when it is necessary to train the operational neural network. In particular, this is done before the initial delivery of the operational neural network. Furthermore, this is performed periodically, or when there are significant changes in the operating conditions of the imaging system (e.g., delivery of a new scanner model, changes in the patient population being imaged), during maintenance of the operational neural network, or when a new version is released, with the aim of maintaining or improving the required performance of the imaging system. Accordingly, the analysis engine in block 402 prompts the operator (via the user interface) to input the desired upscaling coefficients for which the operational neural network should be trained. Corresponding downscaling coefficients are also defined (e.g., as their reciprocals, unless predefined as a single fixed value).

[0066] The collector in block 403 collects (incomplete or possibly complete) sample images obtained from multiple (sample) imaging procedures and stores them in the corresponding repository. These imaging procedures are performed on the same type of body part that the operational neural network is intended to use on. The imaging procedures may be clinical or preclinical. In fact, the inventors have found that operational neural networks trained on (at least some of) sample images obtained from animals perform well when applied to humans. As a result, the information necessary for training the operational neural network can be provided relatively easily. At this stage, the preprocessor can also preprocess the sample images (within the corresponding repository). For example, the preprocessor aligns and spatially correlates the sample images (e.g., by applying a rigid body transformation). Or, additionally, the preprocessor denoises the sample images to reduce noise. For this purpose, an autoencoder (convolutional neural network) can be used. This autoencoder is unsupervisedly trained using a large number of sample images (e.g., all collected images). In particular, autoencoders are trained to optimize their ability to encode each sample image, ignoring unimportant data (such as noise), decoding the resulting image, and reconstructing the same sample image with reduced noise.

[0067] Sample images may include the corresponding raw data used to generate them. For example, in the case of an MRI scanner, the raw data is acquired as k-space images. Each k-space image is defined by a matrix of cells where the horizontal axis corresponds to spatial frequency (number of cycles per unit distance) or wavenumber k, and the vertical axis corresponds to the phase of the detected response signal. Each cell contains a complex number that defines a different amplitude component of the corresponding response signal. The k-space image is converted to a corresponding complex image by applying an inverse Fourier transform. The complex image is defined by a cell matrix of corresponding voxels, where each cell contains a complex number that represents the response signal received from that location. Finally, the complex image is converted to a sample image in the corresponding amplitude format by setting the absolute value of the corresponding complex number in the complex image as the value of each voxel.

[0068] In block 404, the process enters a loop, and the preprocessor processes the sample images for the (current) imaging procedure (starting from the first one in any order from the corresponding repository). In block 405, the processing flow branches depending on whether it is necessary to reduce the number of sample images. For example, this operation may be performed to ensure a minimum quality level of sample images, to select a single zero-dose sample image, or to equalize the number of zero-dose sample images and full-dose sample images (and low-dose sample images if necessary) (e.g., reducing the number of zero-dose sample images relative to full-dose / low-dose sample images). If necessary, the analysis engine in block 406 discards one or more sample images (removes them from the corresponding repository). For example, it is possible to calculate a quality index (such as the signal-to-noise ratio) for each sample image and discard all sample images whose quality index is below an acceptable value (or even strictly below), discard all zero-dose sample images except for the one with the highest quality index, or discard zero-dose sample images with low quality indexes that exceed the number of full-dose / low-dose sample images. If there is no need to reduce the number of sample images, proceed directly from block 405 or via block 406 to block 407.

[0069] Next, the processing flow branches depending on whether it is necessary to increase the number of sample images at this point. For example, this operation may be performed to ensure a minimum number of sample images, to prepare multiple full-dose sample images (in clinical imaging procedures, only one full-dose sample image may be obtained), or to match the number of zero-dose sample images and full-dose sample images (and low-dose sample images as needed) (for example, increasing the number of full-dose / low-dose sample images relative to the number of zero-dose sample images). If necessary, the preprocessor in block 408 generates one or more (generated) sample images and adds them to the corresponding repository. For example, for full-dose sample images (similar concepts apply to zero-dose and low-dose sample images), it is possible to generate them by linearly combining two or more (already acquired) full-dose sample images, generate them using an autoencoder with the corresponding (already acquired) full-dose sample image as input, or add random noise to the corresponding (already acquired) full-dose sample image. If it is not necessary to increase the number of sample images, the process proceeds to block 409 either directly from block 407 or via block 408.

[0070] Next, the processing flow branches depending on the type of sample image for the imaging diagnostic procedure (which may have decreased or increased in number). Specifically, if the sample image is incomplete, blocks 410-446 are executed, and if the sample image is complete, block 447 is executed. In either case, the process then proceeds to block 448.

[0071] In particular with respect to block 410 (incomplete sample images), the creator creates two or more incomplete sample sets for the image diagnostic procedure (each set consists of at least one zero-dose sample image and at least one full-dose sample image extracted from the sample image repository) and saves them to the corresponding repository (with links to the acquisition parameters in the sample image repository). For example, the creator creates incomplete sample sets by combining one zero-dose sample image with multiple full-dose sample images, by creating a one-to-one correspondence between the same number of zero-dose sample images and full-dose sample images, or by combining zero-dose sample images and full-dose sample images in any combination.

[0072] Next, in block 411, the loop is entered, and the analysis engine processes the (current) incomplete sample set of the imaging procedure (starting from the first in any order from the corresponding repository). If necessary, the noise corrector in block 412 calculates the noise of the zero-dose sample image (the difference between the image at acquisition and the image after denoising) and determines its standard deviation at zero dose. Similarly, it calculates the noise of the full-dose sample image (the difference between the image at acquisition and the image after denoising) and determines its standard deviation at full dose. In either case, the sample (zero-dose / full-dose) images can be denoised with the autoencoder as described above. The noise corrector determines the reference standard deviation, for example, by taking the average of the standard deviations at zero dose and full dose. The noise corrector calculates the standard deviation of the artificial noise to be injected additively into the corresponding sample source image (for example, by applying a noise addition formula to the reference standard deviation and increasing the result by a correction coefficient) or the standard deviation of the artificial noise to be injected multiplicatively / convolutionally (for example, by multiplying this value by a transformation coefficient).

[0073] In block 413, the processing flow branches depending on the configuration of the analysis engine (for example, if manually selected by the operator via the user interface, defined by default, or if it is the only available configuration). Specifically, if the analysis engine is not configured to operate in k-space, blocks 414-431 are executed; otherwise, blocks 432-444 are executed. In either case, the processing flow rejoins in block 445.

[0074] In particular, for block 414 (non-k-space), the analysis engine optionally calculates a modulation coefficient to modulate the downscaling coefficient used when applying the simulation formula obtained from the corresponding repository (e.g., if manually selected by the operator via the user interface, defined by default, or in the only available configuration). In practice, the simulation formula may introduce approximations, and the higher the local concentration of the contrast agent, the greater the approximation error. Specifically, in the absence of contrast agent, the signal intensity value obtained by the simulation formula (simulated value) is approximately equal to the measured signal intensity obtained when a low dose of contrast agent is actually administered to the corresponding body part of the subject and a low-dose sample image is obtained. However, as the local concentration of the contrast agent increases, the simulated value gradually becomes lower than the measured value. To compensate for this decrease in the simulated value against the measured value, the decrease in the simulated value for the corresponding dose can be suppressed by increasing the value of the downscaling coefficient used in the simulation formula (i.e., decreasing its denominator). More specifically, by solving an equation that makes the ratio of the signal law and its approximation equal to 1 with respect to the downscaling coefficient, it is obtained that the value of the downscaling coefficient should increase linearly as a function of the local concentration of the contrast agent according to a proportionality coefficient (modulation coefficient) that depends on the acquisition parameters. This modulation coefficient is given by a correction formula that is analytically determined as a function of the acquisition parameters, or by empirically determined values ​​corresponding to the acquisition parameters. Thus, the analysis engine obtains the acquisition parameters of an incomplete sample set from an incomplete sample set repository (via links to them in the sample image repository), applies a correction formula to the acquisition parameters, or obtains values ​​corresponding to the acquisition parameters from a predefined table to calculate the modulation coefficient.

[0075] The processing flow further branches in block 415 depending on the configuration of the analysis engine. In particular, if the analysis engine is configured to operate with amplitude-formatted sample images, it enters a loop in block 416, and the analysis engine processes the (current) voxels of the full-dose sample image (starting from the first one in any order from the corresponding repository). In block 417, the analysis engine modulates the downscaling coefficient used when applying the simulation formula to that voxel. To do this, the analysis engine calculates the contrast enhancement of the voxel as the difference between the voxel value of the full-dose sample image and the voxel value of the zero-dose sample image, and multiplies the downscaling coefficient by the product of the modulation coefficient and the contrast enhancement to obtain the modulated downscaling coefficient. In block 418, the analysis engine uses the (modulated) downscaling coefficient to apply the simulation formula to the voxel values ​​of the zero-dose sample image and the full-dose sample image to calculate the voxel values ​​of the low-dose image. Therefore, in this example, the analysis engine subtracts the voxel value of the sample zero-dose image from the voxel value of the sample full-dose image, multiplies the difference by a downscaling factor, and adds the result to the voxel value of the sample zero-dose image. The analysis engine then adds the resulting voxel value to the sample low-dose image being built in the corresponding repository. In block 419, the analysis engine checks whether the last voxel has been processed. If there are any unprocessed voxels remaining, the process returns to block 416 and repeats the same process for the next voxel. Conversely (if all voxels have been processed), the corresponding loop ends and proceeds to block 420.

[0076] At this point, the noise corrector additively injects artificial noise into the obtained sample low-dose images. To do this, the noise corrector generates artificial noise as a cell matrix (noise matrix) of the same size as the sample low-dose images. This noise matrix contains random values ​​from a normal distribution with a zero mean and a standard deviation equal to the standard deviation of the artificial noise. In block 421, the noise corrector adds this noise matrix voxel by voxel to the sample low-dose images in the corresponding repository. The process then proceeds to block 445.

[0077] Returning to block 415, if the analysis engine is configured to work with complex-format sample images, the processing flow branches in block 422 depending on their availability. If the zero-dose and full-dose sample images are already available in complex format, in block 423 the analysis engine performs phase correction, rotating the vectors representing the complex numbers in each cell to cancel out their angles (while maintaining the absolute values). This operation allows the same results to be obtained for applying the simulation formula to the complex-format zero-dose and full-dose sample images (because all operations performed on the corresponding complex numbers without an imaginary part are equivalent to those performed on their absolute values). The processing then proceeds to block 424. The same point is reached directly from block 422 if the zero-dose and full-dose sample images are available in amplitude format. In this case, the zero-dose and full-dose sample images are treated directly as complex format, and their respective voxel values ​​(real numbers) are considered complex numbers with an imaginary part of 0.

[0078] The same processing as described above is performed to handle the sample zero-dose image and the sample full-dose image in complex format and generate the sample low-dose image from them. Specifically, entering a loop, the analysis engine processes the (current) voxel of the sample full-dose image (starting from the first one in any order from the corresponding repository). In block 425, the analysis engine calculates the contrast enhancement of that voxel (the difference between the absolute value of the voxel in the sample full-dose image and the absolute value of the voxel in the sample zero-dose image), and multiplies the product of the modulation coefficient and the contrast enhancement by the downscaling coefficient to obtain the downscaling coefficient after modulation. In block 426, the analysis engine uses the (modulated) downscaling coefficient to apply a simulation formula to the voxel values ​​of the sample zero-dose image and the voxel values ​​of the sample full-dose image to calculate the voxel values ​​of the sample low-dose image. Then, it adds the obtained voxel values ​​to the sample low-dose image being constructed in the corresponding repository. In block 427, the analysis engine checks whether the last voxel has been processed. If there are any unprocessed voxels remaining, the process returns to block 424 and repeats the same process for the next voxel. Conversely (if all voxels have been processed), the corresponding loop terminates and the process proceeds to block 428.

[0079] At this point, the noise corrector injects artificial noise into the obtained low-dose sample images in a convolutional manner. For this purpose, the noise corrector generates artificial noise as a cell matrix (noise matrix) of the same size as the low-dose sample images. This noise matrix contains complex random values ​​of a normal distribution whose unitary mean and standard deviation are equal to the standard deviation of the artificial noise. In block 429, the noise corrector uses this noise matrix to perform a convolution operation on the low-dose sample images in the corresponding repository (for example, by cyclically moving the noise matrix one stride at a time across the entire low-dose sample image, wrapping around in all directions). In block 430, the analysis engine converts the thus obtained low-dose sample images into amplitude format. For this purpose, each voxel value (generally a complex number) in the low-dose sample image is replaced with its absolute value. In block 431, the processing flow further branches depending on the configuration of the analysis engine. In particular, if the analysis engine is configured to also inject artificial noise into low-dose sample images in an additive manner, the process proceeds to block 420, performs the same process as described above, and then proceeds to block 445. Otherwise, the process proceeds directly to block 445.

[0080] For block 432 (k-space), the analysis engine processes the complex-form sample zero-dose image and the sample full-dose image (either directly, if available, or by applying an inverse Fourier transform from the k-space form). As mentioned above, in block 433, the analysis engine performs phase correction, rotating the vectors representing the complex numbers in each cell of the complex-form sample zero-dose image and sample full-dose image to cancel out their angles (the absolute values ​​are preserved). In block 434, the analysis engine converts the sample zero-dose image and sample full-dose image from complex form to k-space form by applying a Fourier transform.

[0081] Next, a sample low-dose image is generated using the sample zero-dose image and the sample full-dose image in k-space format. Specifically, in block 435, the loop is entered and the analysis engine processes the (current) cell of the sample full-dose image (starting from the first cell in any order). In block 436, the analysis engine uses the (original) downscaling coefficient to apply a simulation formula to the cell values ​​of the sample zero-dose image and the sample full-dose image to calculate the cell values ​​of the sample low-dose image. The obtained cell values ​​are then added to the sample low-dose image being built in the corresponding repository. In block 437, it is checked whether the last cell has been processed, and if there are any unprocessed cells, the process returns to block 435 and the same process is performed on the next cell. Once all cells have been processed, the loop ends and the process proceeds to block 438.

[0082] At this point, the noise corrector injects artificial noise into the obtained sample low-dose image in a multiplicative manner. For this purpose, the noise corrector generates artificial noise as a cell matrix (noise matrix) the same size as the sample low-dose image. This noise matrix contains complex random values ​​of a normal distribution with a unitary mean and a standard deviation equal to the standard deviation of the artificial noise. In block 439, the noise corrector multiplies the sample low-dose image by the noise matrix cell by cell in the corresponding repository. In block 440, the process branches depending on the configuration of the analysis engine. In particular, if the analysis engine is configured to inject artificial noise in an additive manner as well, the process proceeds to block 441, where the noise corrector generates an (additional) noise matrix. This matrix is ​​the same size as the sample low-dose image and contains complex random values ​​of a normal distribution with a zero mean and a standard deviation equal to the standard deviation of the artificial noise. In block 442, the noise corrector adds this noise matrix cell by cell to the sample low-dose image in the corresponding repository. The process then proceeds to block 443. Furthermore, if the analysis engine does not perform additive artificial noise injection, the process proceeds directly from block 440 to block 443. At this point, the analysis engine converts the sample low-dose image from k-space format to complex format by applying an inverse Fourier transform. In block 444, the analysis engine converts the sample low-dose image from complex format to amplitude format, replacing each voxel value with its absolute value. The process then proceeds to block 445. Although not shown in the figure, alternatively, if necessary, the process can proceed from block 444 to block 420, perform the same processing as described above, and then proceed to block 445 to inject additive artificial noise into the amplitude format sample low-dose image.

[0083] In block 445, the analysis engine checks whether the last incomplete sample set of the imaging procedure has been processed. If there are any unprocessed incomplete sample sets, it returns to block 411 and processes the next incomplete sample set in the same way. Once all incomplete sample sets (or selected portions) have been processed, the loop ends and proceeds to block 446.

[0084] At this point, the combiner generates one or more complete sample sets for the imaging procedure (each set consisting of at least one zero-dose sample image, a full-dose sample image, and a low-dose sample image) (and stores them in the corresponding repository), and combines the incomplete sample sets with low-dose sample images generated from other incomplete sample sets (obtained from the corresponding repository). This result can be achieved in various ways. In particular, in one embodiment, half of the incomplete sample sets (in this case, it is not necessary to simulate the corresponding low-dose sample images) are combined in a one-to-one correspondence with the low-dose sample images simulated from the remaining half of the incomplete sample sets. Thus, if there are N incomplete sample sets, INT(N / 2) complete sample sets are obtained. For example, if there are four incomplete sample sets B1-F1, B2-F2, B3-F3, and B4-F4 (each consisting of corresponding zero-dose sample images B1, B2, B3, B4 and full-dose sample images F1, F2, F3, F4) and four low-dose sample images R1, R2, R3, R4 that can be simulated from them, then using the incomplete sample sets B1-F1 and B2-F2 with the low-dose sample images R3 and R4, the complete sample sets (INT(4 / 2)=2) are B1-F1-R3 and B2-F2-R4. This embodiment increases the diversity of complete sample sets used for training neural networks in operation. In another embodiment, incomplete sample sets are combined with low-dose sample images simulated from other incomplete sample sets in any combination. Thus, if there are N incomplete sample sets and corresponding low-dose sample images, N × (N-1) complete sample sets can be obtained. In the same example, the complete sample set (4 x 3 = 12) would be B1-F1-R2, B1-F1-R3, B1-F1-R4, B2-F2-R3, B2-F2-R4, B2-F2-R1, B3-F3-R4, B3-F3-R1, B3-F3-R2, B4-F4-R1, B4-F4-R2, and B4-F4-R3. This method allows for a larger number of complete sample sets to be used for training the neural network during operation.In yet another embodiment, an incomplete sample set is combined with a one-to-one correspondence of low-dose sample images simulated from other incomplete sample sets. Thus, if there are N incomplete sample sets and corresponding low-dose sample images, N complete sample sets are obtained. In the same example, the complete sample sets (4) are B1-F1-R2, B2-F2-R3, B3-F3-R4, and B4-F4-R1. This embodiment provides a configuration that balances diversity with the number of complete sample sets. The process then proceeds to block 448.

[0085] Regarding block 447 (complete sample images), the combiner directly generates one or more complete sample sets (each set consisting of at least one zero-dose sample image, a full-dose sample image, and a low-dose sample image) by combining sample images (obtained from the corresponding repository) for the image diagnostic procedure, and stores them in the corresponding repository. This result can also be achieved in various ways. In particular, there are methods such as one-to-one correspondence between an equal number of full-dose and low-dose sample images and combining each pair with the same (single) zero-dose sample image; one-to-one correspondence between an equal number of zero-dose, full-dose, and low-dose sample images; combination of full-dose and low-dose sample images in any combination and combining each pair with the same (single) zero-dose sample image; and combination of zero-dose, full-dose, and low-dose sample images in any combination. The process then proceeds to block 448.

[0086] Block 448 checks whether the last imaging procedure has been processed. If there are any unprocessed imaging procedures, the process returns to Block 404 and repeats for the next imaging procedure. Conversely (if all imaging procedures have been processed), the loop terminates and the process proceeds to Block 449. As a result, the complete sample set repository may contain a mixture of complete sample sets where low-dose sample images were simulated (simulated) and complete sample sets where low-dose sample images were acquired (acquired). For example, acquired complete sample sets may account for 1-20%, preferably 5-15%, more preferably 6-12%, for example 10%, of the total number of (simulated / acquired) complete sample sets. This allows for further improvement of the training quality of the operational neural network while minimizing additional effort (especially when acquired complete sample sets are obtained from preclinical imaging procedures).

[0087] Regarding block 449, the processing flow branches depending on the operating mode of the configuration computer. If a training neural network is available to simulate the low-dose sample images used to train the operational neural network, in block 450 the training engine trains the training neural network using the complete sample set obtained from the corresponding repository. For example, it performs the same processing as training the operational neural network described later, but the difference is that the training neural network is optimized to generate low-dose sample images from the corresponding zero-dose and full-dose sample images. In this case, it is also possible to improve the performance of the training neural network using a more complex loss function, such as an approach using generative adversarial networks (GANs). The training engine saves the configuration of the training neural network thus obtained to the corresponding repository, and at the same time deletes the analytically simulated low-dose sample images and their corresponding complete sample sets from each repository. Next, in block 451, it enters a loop and simulates refined low-dose sample images for the image diagnostic procedure that provides incomplete sample images. To this end, the analysis engine processes the image diagnostic procedure that provides the (current) incomplete sample images (starting from the first one in any order from within the sample image repository), and in block 452 processes the (current) incomplete sample set of that image diagnostic procedure (starting from the first one in any order from the corresponding repository). In block 453, the analysis engine inputs the zero-dose and full-dose sample images of the incomplete sample set into the training neural network. Moving on to block 454, the training neural network outputs the corresponding low-dose sample image and saves it to the corresponding repository. In block 455, the analysis engine checks if the last incomplete sample set has been processed. If there are any unprocessed incomplete sample sets, processing returns to block 452 and repeats the same process for the next incomplete sample set.Conversely (if all incomplete sample sets, or selected portions, have been processed), the loop terminates and the process proceeds to block 456, where the analysis engine checks if the last imaging procedure has been processed. If there are any unprocessed imaging procedures, the process returns to block 451 and repeats the same process for the next imaging procedure. Conversely (if all imaging procedures have been processed), the loop terminates and the process proceeds to block 457. At this point, the combiner combines the incomplete sample sets with low-dose sample images generated from other incomplete sample sets (obtained from the corresponding repository) to generate one or more complete sample sets for that imaging procedure (storing them in the corresponding repository) and processes them in the manner described above.

[0088] Subsequently, the process proceeds to block 458, where the operational neural network is trained using the complete sample set thus obtained. This implementation improves the accuracy of low-sample-dose images, and consequently, the performance of the operational neural network trained on the corresponding complete sample set also improves. Furthermore, if a training neural network is unavailable, the process proceeds directly to this point from block 449, and the complete sample set generated by the analysis engine is used directly to train the operational neural network. This implementation is particularly simple and fast, and at the same time, the accuracy of the analytically simulated low-sample-dose images is sufficient for the purpose of training the operational neural network with acceptable performance.

[0089] In either case, the training engine performs a process to find the optimal weights to optimize the performance of the production neural network. First, the training engine may post-process the sample images of each complete sample set as needed. For example, the training engine scales and normalizes the voxel values ​​of the sample images to a (common) predefined range. Furthermore, to reduce overfitting in training the production neural network, it performs data augmentation to generate (new) complete sample sets from each (original) complete sample set. For example, a new complete sample set is generated by rotating (from 0° to 90° in increments of 1-5°) and / or flipping the sample images of the original complete sample set horizontally and vertically. In either case, in block 459, the training engine selects multiple training sets by sampling a certain percentage (e.g., 50% randomly) from the complete sample sets in the corresponding repository. In block 460, the training engine randomly initializes the weights of the production neural network. Next, in block 461, it enters a loop, and the training engine inputs the zero-dose and low-dose sample images of each training set into the production neural network. Accordingly, in block 462, the operational neural network generates a corresponding output image, which should match the full sample dose image (ground truth data) of the training set. In block 463, the training engine calculates a loss value based on the difference between the output image and the full sample dose image. For example, the loss value is given by the mean absolute error (MAE), which is calculated as the average of the absolute differences in the corresponding voxel values ​​between the output image and the full sample dose image. In block 464, it is checked whether the loss value is unacceptable and whether there has been significant improvement. This determination can be made in either an iterative mode, which evaluates the loss value after each training set processing, or a batch mode, which evaluates the cumulative loss value (e.g., its average value) after all training sets have been processed. If the conditions are met, in block 465, the training engine updates the weights of the operational neural network to attempt to improve performance.For example, stochastic gradient descent (SGD) based on the ADAM method is applied to determine the direction and amount of change based on the gradient of the loss function (the loss value is approximated as a function of the weights using a backpropagation algorithm and updated according to a predefined learning rate). The process then returns to block 461 and repeats the same process. It returns to block 464 again, and the loop terminates when the loss value reaches an acceptable range or when significant improvement is no longer obtained by changing the weights (meaning that a minimum value of the loss function, at least a local minimum, or a flat region has been found). The above loop is repeated, for example, 100 to 300 epochs, searching for different (potentially better) local minimums and identifying flat regions of the loss function by adding random noise to the weights or changing the initialization of the operational neural network.

[0090] Once a configuration for a production-time neural network that yields the optimal minimum value of the loss function is found, processing proceeds to block 466, where the training engine validates the performance of the resulting production-time neural network. For this purpose, the training engine selects several validation sets from the sample sets in the corresponding repository (e.g., sets different from the training set). Entering a loop in block 467, the training engine inputs the zero-dose and low-dose sample images of the (current) validation set into the production-time neural network (starting from the first in any order). Accordingly, in block 468, the production-time neural network generates a corresponding output image, which should match the full-dose sample image of the validation set. In block 469, the loss value is calculated as described above, based on the difference between the output image and the full-dose sample image. In block 470, it is checked whether the last validation set has been processed. If there are any unprocessed validation sets, processing returns to block 467 and repeats the same process for the next validation set. Once all validation sets have been processed, the loop terminates and the process proceeds to block 471. At this point, the training engine calculates the global loss for the validation (e.g., the average of the loss values ​​across all validation sets). In block 472, the process branches depending on this global loss. If the global loss exceeds (and in some cases strictly) the acceptable limit, this means that the generalization ability of the production neural network (its ability to apply the configuration learned from the training set to the validation set) is insufficient. In this case, the process returns to block 459 and repeats the same process with different training sets and learning parameters (learning rate, number of epochs, etc.). Conversely, if the global loss is (and in some cases strictly) below the acceptable limit, it means that the generalization ability of the production neural network is at a satisfactory level. In this case, in block 473, the training engine approves the resulting configuration of the production neural network and saves it to the corresponding repository along with its upscaling coefficient value.

[0091] In block 474, the analysis engine verifies whether the operational neural network configuration is complete. If not, the process returns to block 402 and repeats the same process to configure the operational neural network for different upscaling coefficients. Conversely, if the operational neural network configuration is complete, the configuration obtained in block 475 is deployed in bulk to multiple instances of the control computer of the corresponding medical imaging system (for example, pre-loaded at the factory during the initial delivery of the medical imaging system, and uploaded via the network or removable storage device during upgrades). The process then terminates with concentric white / black stop symbols 476.

[0092] Next, with respect to Figures 5A to 5C, representative examples of experimental results related to a solution according to one embodiment of the present disclosure are shown.

[0093] In particular, dedicated preclinical studies were conducted on rats with the following two types of brain lesions: C6 glioma tumors (n=36) and cerebral ischemic lesions (n=42). All animals underwent surgery to induce the lesions. Animals that survived the surgery and showed little to no clinical symptoms for the following two weeks (i.e., the period required for pathological development of the lesion) were enrolled in MRI-type imaging procedures (usually twice per animal, three times in limited cases). The imaging procedures were performed using gadolinium-based contrast agents and a Bruker™ Pharmascan preclinical scanner / spectrometer (7T operation, 2-channel rat head volume coil). The CE-MR protocols used in each imaging procedure are as follows: • First pre-contrast acquisition using standard T1-weighted sequencing (first sample zero-dose image) • Second pre-contrast acquisition using standard T1-weighted sequencing (second sample zero-dose image) • Administer contrast agent intravenously at a low sample dose of 0.01 mmol Gd / kg. • Acquisition of T1-weighted sequencing after contrast enhancement (low-dose sample images) Immediately after the previous dose, administer an additional 0.04 mmol Gd / kg of contrast agent intravenously to bring the total sample full dose to 0.05 mmol Gd / kg. • First acquisition using T1-weighted sequencing after contrast enhancement (first image of full sample dose) • Second acquisition using T1-weighted sequencing after contrast enhancement (second image showing full sample dose)

[0094] This study yielded a total of 130 3D MRI sample images (61 from rats with brain tumors (gliomas) and 69 from ischemic model rats), each consisting of 24 slices. The acquired sample images were used to construct the following two types of datasets. • Acquired dataset: For each imaging diagnostic procedure, the dataset includes the first zero-dose image, the low-dose sample image, and the first full-dose sample image (all acquired images). • Simulation dataset: For each imaging diagnostic procedure, in addition to the first sample zero-dose image and the first sample full-dose image (acquired image), the dataset includes the second sample zero-dose image and a low-dose image simulated from the second sample full-dose image with a downscaling factor d=1 / 5.

[0095] Using full-dose sample images as ground truth, a convolutional neural network (described in "Deep artifact learning for compressed sensing and parallel MRI" by Dongwook Lee, Jaejun Yoo, and Jong Chul Ye [https: / / arxiv.org / abs / 1703.01120]) was trained with the following hyperparameters. • Learning rate = 0.01 • Decay rate = 0.001 • Composite Loss (composite LOSS) = Mean Absolute Error (MAE) + Mean Absolute Error in the Fourier Transform Domain (fftMAE) + Perceived Loss (PL, using up to layer 4 of a VGG19 network pre-trained on the ImageNET dataset (listed at [https: / / www.image-net.org / ])) • Relative weights of composite losses: a=b=c=1 and a=b=0,c=1 (a=MAE, b=fftMAE, c=PL) • Levels of artificial noise (σ_artificial) to inject into simulated low-dose sample images: 0.01, 0.0125, 0.015

[0096] Different instances of the operational neural network were trained using the acquired dataset and the simulation dataset, respectively, with corresponding full-dose images as ground truth data. These neural networks were then applied to sample low-dose images included in the acquired dataset (to verify the ability to restore the full-dose contrast agent effect) and operational full-dose images acquired using full-dose contrast agents in a medical image processing application (to verify the ability to enhance the contrast agent effect).

[0097] Beginning with Figure 5A, three representative examples are shown of acquired full-dose sample images and corresponding full-dose sample images simulated by operational neural networks trained on both the acquired and simulated datasets. As you can see, the simulated full-dose sample images are very similar to the acquired full-dose sample images. This is true whether the operational neural network was trained on the acquired dataset or the simulated dataset. In particular, using an operational neural network trained on the simulated dataset introduces virtually no artifacts into the simulated full-dose sample images.

[0098] Moving on to Figure 5B, representative examples of acquired full-dose sample images and simulated full-dose sample images generated by a field-of-use neural network trained on a simulation dataset are shown. Here, the results of using different hyperparameters (i.e., relative weights a=b=c=1 and a=b=0,c=1, noise levels 0.01, 0.0125, and 0.015) are compared. As you can see, by adjusting the hyperparameters, the quality of the simulated full-dose sample images can be further improved, increasing their similarity to the acquired full-dose sample images. This can be done based on various evaluation criteria, such as the enhancement of areas perfused with contrast agent and the signal intensity of enhanced and unenhanced areas.

[0099] Moving on to Figure 5C, two representative examples are shown: a zero-dose image at operation, a full-dose image at operation acquired during the imaging procedure of a (human) patient, and the corresponding boosted dose image (upscaling coefficient k=5) simulated by an operational neural network trained on the acquired dataset and the simulation dataset (both based on animal imaging procedures). As you can see, the boosted dose image simulated by the operational neural network trained on the simulation dataset can be superimposed on the image simulated by the operational neural network trained on the acquired dataset, and in both cases, contrast enhancement is improved without introducing substantial artifacts.

[0100] Regarding corrections

[0101] To meet regional or specific requirements, a person skilled in the art can make numerous logical and / or physical modifications and changes to this disclosure. More specifically, while this disclosure describes one or more embodiments in some degree of detail, it should be understood that forms and details can be omitted, replaced, modified, or otherwise implemented. In particular, different embodiments of this disclosure can be implemented without certain details (such as numerical values) described above, and these can be omitted for the sake of understanding. Conversely, well-known features may be omitted or simplified to avoid complicating the description with unnecessary details. Furthermore, features described in each sentence are explicitly intended to be implementable independently of features described in other sentences (unless functionally strictly required). In any case, certain features described in relation to any embodiment of this disclosure can be incorporated into any other embodiment as a design choice. Also, items, different embodiments, examples, and alternatives presented within the same group should not be considered substantially equivalent, but rather as independent and autonomous elements. In any case, each numerical value should be interpreted as modified according to applicable tolerances. In particular, unless otherwise stated, terms such as “substantially,” “about,” and “approximately” should be understood as being within ±10%, preferably ±5%, and even more preferably ±1%. Numerical ranges should be interpreted as explicitly specifying any continuous value (including endpoints) within that range. Ordinal numbers and other modifiers are used solely as labels to distinguish elements with the same name and do not imply priority or order. Terms such as “include,” “equip,” “possess,” “contain,” and “involve” should be interpreted in an open and non-restrictive sense (i.e., not limited to the listed items). Terms such as “based on,” “dependent on,” “followed by,” and “function of” should be interpreted as a non-exclusive relationship (i.e., other variables may be involved). The terms “a” and “an” should mean one or more items unless explicitly stated to mean only one. Furthermore, “means for” or similar functional expressions should mean any structure adapted or configured to perform the relevant function.

[0102] For example, in one embodiment, a method for training a machine learning model is provided. However, this machine learning model may be of any kind (e.g., fully connected neural networks, convolutional neural networks, generative adversarial networks, linear and nonlinear regression models [such as polynomial regression, support vector regression, nearest neighbor regression, Gaussian process regression, etc.], probabilistic models [such as Bayesian networks, Markov random fields, etc.], and may be used in combination with optimization methods, including advanced techniques such as gradient descent or genetic algorithms, as needed).

[0103] In one embodiment, this machine learning model is used in a medical imaging application. However, the medical imaging application can be of any kind (e.g., diagnostic, therapeutic, or surgical applications based on MRI, CT, fluoroscopy, fluorescence, ultrasound, etc.).

[0104] In one embodiment, this method includes the following steps under the control of a computing system, which may be of any type (details below).

[0105] In one embodiment, the method includes the step of providing multiple incomplete sample sets (to a computing system) for each of several imaging procedures, provided that the imaging procedures can be any number and of any type (e.g., preclinical, clinical, etc.), and the number of incomplete sample sets for each imaging procedure can also be arbitrary (they can be the same or different between imaging procedures). Furthermore, the incomplete sample sets can be provided in any way (e.g., already formed, created from corresponding sample images, collected from any source such as medical / research facilities, etc. The collection method can also be downloading from a central server at the facility via the Internet, automatically acquiring them via a corresponding LAN, manually acquiring them using removable storage media, manually reading them from removable storage media copied from a central server or (standalone) imaging system, etc.). In any case, this is a computer-implemented data processing method that is performed independently of the acquisition of sample images (without requiring interaction with the corresponding subjects).

[0106] In one embodiment, the imaging procedure relates to a corresponding body part of a subject. However, the body parts may be any number or type (e.g., organs, their regions, tissues, bones, joints, etc., and the machine learning model may be the same as or different from the subject used in the medical imaging procedure), and may be in any state (e.g., healthy, pathological state with lesions, etc.). Furthermore, the body parts may belong to any number or type of subject (e.g., animals, humans, etc.).

[0107] In one embodiment, each incomplete sample set includes at least one sample target image. However, the sample target images may be of any number (e.g., axial, coronal, sagittal images, images corresponding to different contrast agent doses, etc.) and of any type (e.g., magnitude format, complex format, k-space format, etc., with arbitrary size, resolution, chromaticity, bit depth, etc.) and may relate to any location of a body part (e.g., voxels in the case of 3D images, pixels in the case of 2D images, etc.).

[0108] In one embodiment, the sample target image represents the corresponding body part of a subject to which a contrast agent has been administered at a sample target dose. However, the contrast agent can be of any type (e.g., targeted contrast agents based on specific or nonspecific interactions, non-targeted contrast agents, etc.) and may be administered in any way that ensures perfusion to the body part (e.g., intravenously, intramuscularly, or orally, at any time well before or immediately before the imaging procedure). The sample target dose may also be arbitrary (e.g., the same as the full dose, less than the full dose, or more than the full dose, etc.).

[0109] In one embodiment, each incomplete sample set includes at least one sample baseline image. However, the sample baseline images may be any number (e.g., axial, coronal, sagittal images, images corresponding to zero dose and / or different contrast agent doses, etc.) and of any type (e.g., the same as or different from the sample target images).

[0110] In one embodiment, the sample baseline image represents the corresponding body part of the subject without contrast agent. However, the sample baseline image may be obtained in any way that ensures that the contrast agent does not significantly affect its content (e.g., obtained well before contrast agent administration, or obtained a sufficient time after previous contrast agent administration).

[0111] In one embodiment, the sample baseline image represents the corresponding body part of a subject to which the contrast agent was administered at a sample baseline dose lower than the sample target dose. However, the sample baseline dose may be any value (e.g., lower or higher than the sample source dose).

[0112] In one embodiment, the method includes the step of simulating multiple sample source images (by a computing system), simulating at least one sample source image from at least a portion of the incomplete sample set for each imaging diagnostic procedure. However, the sample source images may be simulated from any number (up to all) of the incomplete sample sets, in any number for each set (e.g., for each different downscaling coefficient value), and in any way. (For example, it may operate in any domain, such as magnitude form, complex form, k-space form, with or without pre-processing such as alignment, normalization, denoising with autoencoders, block matching, shrinkage fields, wavelet transforms, smoothing filters, distortion correction, filtering of anomalous sample images, and with or without post-processing such as alignment, normalization, noise injection). For example, in one embodiment, the sample source images are generated analytically (e.g., by applying a simulation formula for a single sample baseline / target image, or by interpolation for multiple sample baseline / target images). In another embodiment, a preliminary version of the sample source image is first generated analytically, an additional machine learning model is trained on a preliminary version of the complete sample set based on that preliminary version, and then the refined sample source image is generated by the trained model. In yet another embodiment, the sample source image is generated by an additional machine learning model trained on an additional sample set acquired separately.

[0113] In one embodiment, the sample source image is simulated to represent the corresponding body part, simulating the administration of a contrast agent at a sample source dose lower than the sample target dose to the corresponding subject (the ratio of the sample source dose to the sample target dose is equal to the downscaling factor). However, the sample source dose can take any value, either absolute or relative (for example, the sample source dose may be lower than, equal to, or higher than the full dose, or the sample source dose and sample target dose may define an arbitrary downscaling factor).

[0114] In one embodiment, this method includes the step of generating one or more complete sample sets for each imaging procedure (by a computing system), provided that the number of complete sample sets for each imaging procedure is arbitrary.

[0115] In one embodiment, each complete sample set includes a sample baseline image and a sample target image from one of the incomplete sample sets of the imaging procedure, and a sample source image simulated from another incomplete sample set of the same imaging procedure. However, the incomplete sample sets and sample source images may be combined in any way (for example, combining each incomplete sample set with one or more sample source images from other incomplete sample sets, combining each sample source image with one or more other incomplete sample sets, using all or part of an incomplete sample set or sample source image one or more times, etc.). Also, each complete sample set may include any number of sample source images simulated from one or more other incomplete sample sets.

[0116] In one embodiment, the method includes training a machine learning model (by a computing system) and optimizing its ability to generate sample target images for each complete sample set from sample baseline images and sample source images for that complete sample set. However, this machine learning model can be trained in any way (e.g., selecting any training / validation set from the complete sample set, using any algorithm such as stochastic gradient descent (SGD), real-time recursive learning (RTRL), higher-order gradient descent, augmented Kalman filtering, using any loss function such as mean absolute error (MAE), mean squared error (MSE), perceived loss, adversarial loss. The loss function may be defined per location or per group of locations, for a single upscaling coefficient, for multiple upscaling coefficients corresponding to the sample source images for each complete sample set, or for variable upscaling coefficients as parameters of the operational machine learning model). Furthermore, learning can be performed with or without additional information (for example, any number of complete sample sets consisting of all acquired sample images, already formed ones, or created from corresponding sample images, or additional sample images acquired and / or simulated within one or more complete sample sets [corresponding to different contrast agent dosages or acquisition conditions, health status of body parts, type of subject, etc.]).

[0117] In one embodiment, this method includes the step of deploying a trained machine learning model (by a computing system). However, this machine learning model can be deployed in any way to any number and type of medical imaging systems (e.g., distributing it with a corresponding new medical imaging system, upgrading an already installed medical imaging system, bringing it online, etc.).

[0118] In one embodiment, the deployed machine learning model is used in a medical imaging application to simulate an increase in the dose of contrast agent administered to a corresponding patient, according to an upscaling coefficient that corresponds to the reciprocal of the downscaling coefficient. However, the patient may be of any type (e.g., human, animal, etc.), and the machine learning model can simulate the increase in contrast agent dose in any way (e.g., real-time, offline, local, remote, etc.). Furthermore, the way in which the upscaling coefficient corresponds to the reciprocal of the downscaling coefficient is also arbitrary and not limited (e.g., equal to, less than, greater than, based on a corresponding multiplication coefficient, etc.).

[0119] Further embodiments offer additional advantageous features, which may be omitted in the basic implementation. In this regard, the features of each of the following embodiments are explicitly intended to be able to be used alone or in combination with any number of other embodiments, or in combination with the features described above.

[0120] In one embodiment, the sample source dose lies between the sample baseline dose and the sample target dose, where the sample baseline dose can take any value, either absolute or relative.

[0121] In one embodiment, the sample target dose is the full dose of contrast agent that is standard in clinical practice. However, this full dose can be of any type (for example, it may be fixed for each type of medical imaging application, or it may be determined according to the type of body part, patient type, weight, age, etc.).

[0122] In one embodiment, at least a portion of the subjects are animals, however, the proportion of animals can be any proportion (from zero to the whole) of the total subjects, and the species can also be any (e.g., rats, pigs, etc.).

[0123] In one embodiment, the patient is a human being. However, the human being may be of any type, including gender, age, health condition, etc.

[0124] In one embodiment, this method includes the step of receiving multiple sample images for each imaging procedure (by a computing system), provided that the number of sample images is arbitrary and the method of receiving them is also arbitrary (see the preceding description of incomplete sample sets).

[0125] In one embodiment, the sample images for each imaging procedure include one or more sample baseline images and one or more sample target images. However, the sample images may include any number of sample baseline images and sample target images.

[0126] In one embodiment, the method includes the step of creating an incomplete sample set of imaging procedures based on sample images of each imaging procedure (by a computing system). However, the incomplete sample set can be created in any way (for example, by using the received sample images as they are or adding or subtracting them, by combining each sample baseline image with one or more sample target images, by combining each sample target image with one or more sample baseline images, by using all or part of the sample baseline images or sample target images one or more times, etc.).

[0127] In one embodiment, the method includes the step of generating one or more new sample baseline images and / or sample target images, respectively, from corresponding sample baseline images and / or sample target images received (by a computing system) for one or more imaging procedures. However, any number of new sample images can be generated (only sample baseline images, only sample target images, or both), and the method is also arbitrary (e.g., combining two or more corresponding sample images linearly / nonlinearly, adding arbitrary noise to the corresponding sample images, using any autoencoder, etc.). The imaging procedures to be addressed are also arbitrary (from zero to all).

[0128] In one embodiment, this method includes the step of generating new sample baseline images and / or new sample target images for each imaging procedure (by a computing system) in order to match the number of sample baseline images and sample target images for each imaging procedure. However, the new sample (baseline / target) images can be generated for any purpose (e.g., to increase the number of the sample baseline images or sample target images that are fewer in number, or both, to match the number of both, to obtain the required number of sample baseline / target images from a single image, etc.).

[0129] In one embodiment, this method includes the step of discarding one or more sample baseline images and / or sample target images for one or more imaging procedures (by a computing system). However, any number of sample images (only sample baseline images, only sample target images, or both) can be discarded for any imaging procedure (from zero to all) in any way (e.g., based on quality level, diversity, etc.).

[0130] In one embodiment, the method includes the step of discarding sample baseline images and / or sample target images in each imaging procedure (by a computing system) to equalize the number of sample baseline images and sample target images in that imaging procedure. However, sample (baseline / target) images may be discarded for any purpose (e.g., to reduce the larger number of sample baseline images or sample target images, or both; to equalize the number of both; to obtain the required number of sample baseline / target images; to ensure a minimum quality level or diversity of sample images, etc.).

[0131] In one embodiment, this method includes the step of creating an incomplete sample set for one or more imaging procedures by combining one sample baseline image (by a computing system) with each of several sample target images for that imaging procedure. However, this procedure can be applied to any number of imaging procedures (from zero to all).

[0132] In one embodiment, this method includes the step of creating an incomplete sample set of one or more imaging procedures by combining an equal number of sample baseline images and sample target images of the imaging procedures in a one-to-one correspondence (by a computing system). However, this step can be applied to any number of imaging procedures (from zero to all).

[0133] In one embodiment, this method includes the step of creating an incomplete sample set of one or more imaging procedures by combining sample baseline images and sample target images of the imaging procedures in any combination (by a computing system). However, this step can be applied to any number of imaging procedures (from zero to all).

[0134] In one embodiment, this method includes the step of generating a complete sample set for one or more imaging procedures by combining (by a computing system) half of the incomplete sample set of the imaging procedure with a one-to-one correspondence of sample source images simulated from the other half of the incomplete sample set. However, this procedure can be applied to any number of imaging procedures (from zero to all).

[0135] In one embodiment, this method includes the step of generating a complete sample set for one or more imaging procedures by combining (by a computing system) in any combination of incomplete sample sets of imaging procedures and simulated sample source images from other incomplete sample sets. However, this step can be applied to any number of imaging procedures (from zero to all).

[0136] In one embodiment, this method includes the step of generating a complete sample set for one or more imaging procedures by combining (by a computing system) an incomplete sample set of imaging procedures with a one-to-one correspondence of sample source images simulated from other incomplete sample sets. However, this step can be applied to any number of imaging procedures (from zero to all).

[0137] In one embodiment, each sample baseline image, each sample target image, and each sample source image includes multiple sample baseline values, sample target values, and sample source values, respectively. However, the sample baseline / source / target values ​​may be any number and of any type (e.g., real / complex, any range, grayscale or color, etc.).

[0138] In one embodiment, the method includes the step of calculating each sample source value for each sample source image by applying a simulation formula that depends on a downscaling factor (by a computing system), provided that the simulation formula can be of any kind (e.g., linear, quadratic, cubic, a function of the corresponding sample baseline value and / or sample dose value).

[0139] In one embodiment, the simulation formula is derived from a signal rule that represents the magnitude of the response signal of a body part as a function of the local concentration of the contrast agent. However, the signal rule may be of any kind (e.g., based on any external / internal acquired parameters), and the simulation formula may be derived from the signal rule in any way (e.g., from any approximation of the signal rule, the actual signal rule, etc.).

[0140] In one embodiment, the simulation equation is derived from a signal rule linearized with respect to the local concentration of the contrast agent. However, the signal rule may be linearized in any way (e.g., any series expansion, any approximation, or assuming any linear / nonlinear relationship between the local concentration and the contrast agent dose).

[0141] In one embodiment, the simulation formula is derived from a signaling law that assumes a direct proportional relationship between local concentration and contrast agent dose. However, this direct proportional relationship between local concentration and contrast agent dose may be based on an arbitrary proportionality constant.

[0142] In one embodiment, the sample baseline value, sample target value, and sample source value represent the response signal at the location of the corresponding body part, respectively. However, the response signal may be expressed in any way (e.g., magnitude format, complex format, positive / negative format, etc.).

[0143] In one embodiment, the method includes the step of modulating a downscaling coefficient (by a computing system) used to calculate each sample source value for each sample source image according to an index of the local concentration of the contrast agent at a corresponding location, derived from the corresponding sample target value. However, the local concentration may be derived in any way (e.g., by setting it as the corresponding local contrast enhancement amount, by calculating it from the sample target value according to a signaling law, etc.). The downscaling coefficient may also be modulated according to any linear / nonlinear function based on its local concentration (e.g., by calculating it using the mean / local values ​​of any acquisition parameter according to an empirically determined modulation coefficient, etc.), or it may always be kept at the same value.

[0144] In one embodiment, the method includes the step of injecting artificial noise into each sample source image (by a computing system). However, the artificial noise may be of any kind (e.g., dependent on a downscaling factor, a fixed value, etc.) and may be injected into the sample source image in any way (e.g., additive or multiplicative form [per cell, per group of cells, etc.], convolutional form [circular or non-circular, arbitrary stride and padding, etc.], magnitude-form sample source image, complex form, k-space form, entire area, only in the area where contrast agent is present, etc.), or may not be injected at all.

[0145] In one embodiment, the artificial noise has a statistical distribution that depends on a downscaling coefficient. However, the statistical distribution of the artificial noise may be of any kind (e.g., a normal distribution, Rayleigh distribution, Rice distribution, etc., with any mean), and may be obtained in any way (e.g., calculating artificial values ​​of one or more statistical parameters such as standard deviation, variance, skewness; applying an arbitrary linear / nonlinear function to a baseline value of the statistical parameter obtained from the noise of the sample baseline image and the noise of the sample target image, or to the noise of the sample baseline image only, or to the noise of the sample target image only, etc.). Furthermore, the artificial values ​​of the statistical parameters may be heuristically corrected (e.g., applying the same correction to all statistical parameters, or correcting each statistical parameter in a different way, increasing or decreasing it, or correcting it according to an arbitrary linear / nonlinear function, etc.).

[0146] In one embodiment, the method includes the step of training an additional machine learning model (by a computing system) to optimize its ability to generate sample source images of at least a portion of the complete sample set from corresponding sample baseline images and sample target images. However, this additional machine learning model may be of any kind and may be trained in any way (for example, using all sample sets after analytically completing an incomplete sample set, or using only already completed sample sets, in the same or different ways as the aforementioned machine learning model).

[0147] In one embodiment, this method includes the step of inputting sample baseline images and sample target images of an incomplete sample set corresponding to an additional machine learning model being trained (by a computing system) and generating refined versions of each sample source image. However, the possibility of using this trained machine learning model in another way is not ruled out (e.g., refining or directly generating sample source images of an incomplete sample set).

[0148] In one embodiment, this method (by a computing system) includes repeating the steps of simulating sample source images, generating a complete sample set, and training a machine learning model for multiple downscaling coefficient values, provided that the downscaling coefficients can be any number or type (e.g., uniformly distributed, distributed with a variable pitch, decreasing as the value increases, etc.), and these steps can be repeated in any way (e.g., consecutively, at different points in time, etc.).

[0149] In one embodiment, this method includes the step of deploying a machine learning model of a corresponding configuration, trained with downscaling coefficient values, in each medical image processing application (by a computing system), and enabling the selection of one or more corresponding upscaling coefficient values. However, the different configurations can be deployed in any way (e.g., in batch, added over time, etc.) and may be in any form (e.g., corresponding configurations of a single operational machine learning model, corresponding instances of an operational machine learning model, etc.). Furthermore, they can be used to select the upscaling coefficient values ​​in any number and way (e.g., discrete mode, continuous mode, using the same or different values ​​as the downscaling coefficients, etc.).

[0150] In one embodiment, the machine learning model is a neural network. However, this neural network can be of any type (e.g., autoencoders, multilayer perceptron networks, recurrent networks, generative adversarial networks (GANs), etc., with any number of layers from shallow to deep, arbitrary methods of connecting layers, receptive fields, stride, padding, activation functions, etc.).

[0151] In one embodiment, a method is provided for using the machine learning model trained as described above in a medical image processing application that images a body part of a patient. However, this body part may be of any type or condition, and may belong to any patient (see above). In any case, this method is intended to assist the physician's work and only provides intermediate results to support the physician's judgment; the medical act itself is always performed by the physician.

[0152] In one embodiment, this method includes the following steps under the control of a computing system, which may be of any type (see below).

[0153] In one embodiment, the method includes the step of receiving one or more operational administration images (by a computing system) representing body parts of a patient to which an operational dose of contrast agent has been administered. However, the number and types of operational administration images may be arbitrary (e.g., identical or different from sample images), and the method of reception may also be arbitrary (real-time, offline, local, remote, etc.). Furthermore, the operational dose may be arbitrary (e.g., the same as or different from the sample source dose, less than, equal to, or greater than the full dose of contrast agent, etc.). In any case, the contrast agent may be administered to the patient in any way, but including non-invasive methods (e.g., oral administration for gastrointestinal imaging, nebulizer administration into the airway, topical spray application, etc.) and not involving significant physical interventions (e.g., intramuscular injection) that require specialized medical knowledge or involve health risks.

[0154] In one embodiment, the method includes the step of receiving at least one operational baseline image (by a computing system) representing a body part of a patient in which no contrast agent has been administered or has been administered at an operational baseline dose less than the operational dose. However, the operational baseline image may be acquired in a manner that ensures the contrast agent does not significantly affect its content, or it may be acquired after the contrast agent has been administered at any operational baseline dose (for example, it may be the same as or different from the source baseline image).

[0155] In one embodiment, this method includes the step of generating (simulating) corresponding operational simulation images from operational baseline images and operational administration images using a machine learning model (by a computing system). However, the operational simulation images may be simulated in any way (e.g., operating in any domain such as magnitude format, complex format, k-space format, etc., in real time, offline, local, remote, etc.).

[0156] In one embodiment, the operational simulation image represents a simulation of what would happen if the contrast agent were administered to a patient's body part at an operational simulation dose higher than the operational dose (the ratio of the operational simulation dose to the operational dose is equal to the upscaling coefficient, which is the reciprocal of the downscaling coefficient of the machine learning model). However, the operational simulation dose may be any value (e.g., the same as or different from the sample target dose, less than, equal to, or greater than the full dose).

[0157] In one embodiment, the method includes the step of outputting a representation of a body part based on an operational simulation image (by a computing system). However, this representation of the body part may be of any kind (e.g., an operational simulation image, a corresponding operational composite image, etc.) and may be output in any way (e.g., displayed on any device such as a monitor or VR goggles, or more generally output by printing, remote transmission, etc., in real time or offline).

[0158] Generally, the same considerations apply within the scope of the claims when the same solution is implemented in an equivalent manner (e.g., using similar steps or parts thereof with the same functionality, removing non-essential steps, adding additional optional steps, etc.). Furthermore, the steps may be performed in different orders, simultaneously, or (at least partially) alternately.

[0159] In one embodiment, a computer program is provided, which, when executed on a computing system, is configured to perform the method described above. In another embodiment, a computer program product is provided, which includes one or more non-volatile computer-readable storage media in which program instructions are stored together, which are read by a computing system and used to perform the same method. However, this (computer) program can be executed on any computing system (see below). Furthermore, this program may be implemented as a standalone module, as a plug-in to an existing software program (e.g., a configuration application or an image processing application), or directly embedded therein.

[0160] Generally, similar considerations apply even when the program configuration differs or when additional modules or functions are provided (as long as they are within the scope of the claims). In particular, a program can take any form usable by a computing system and is configured to perform a desired operation as a result. A program may be in the form of external or resident software, firmware, or microcode (either object code or source code) and may be executed, for example, by compilation or an interpreter. Furthermore, a program may be provided to any computer-readable storage medium. A storage medium refers to a tangible medium that can hold and store instructions used by a computing system (different from the temporary signals themselves). For example, a storage medium may be electronic, magnetic, optical, electromagnetic, infrared, or semiconductor, and examples include fixed disks (which can be preloaded with programs), removable disks, and memory keys (such as USB). A program may be downloaded from a storage medium to a computing system, or it may be downloaded via a network (such as the Internet, a wide area network, or a local network including transmission cables, fiber optics, wireless connections, and network equipment). One or more network adapters in the computing system may receive a program from the network and transfer and store it in one or more storage devices of the computing system. In any case, the solution according to one embodiment of the present disclosure can be implemented by a hardware structure (e.g., an electronic circuit integrated on one or more chips of semiconductor material) or by a combination of appropriately programmed or configured software and hardware.

[0161] In one embodiment, a computing system is provided comprising means configured to perform the steps of the above method. In another embodiment, a computing system is provided comprising circuits (i.e., any hardware appropriately configured, for example, by software) for performing each step of the same method. However, this computing system may be of any kind (e.g., a configuration computer implemented as a server, virtual machine, or cloud service for training a machine learning model, a control computer for each scanner implemented as a PC for using the trained machine learning model, a control unit for each scanner, etc.).

[0162] Generally, similar considerations apply (as long as they are within the scope of the claims) even when computing systems have different structures, equivalent components, or other operating characteristics. In any case, each component may be divided into multiple elements, or two or more components may be combined into a single element. Furthermore, each component may be replicated to allow corresponding processing to be executed in parallel. Also, unless otherwise specified, interactions between different components do not necessarily have to be continuous, but may occur directly or indirectly through one or more intermediaries.

[0163] In one embodiment, a medical method is provided that is applied to a body part of a patient. However, this medical method may be applied to any body part of any patient (see above).

[0164] In one embodiment, the medical method includes the step of acquiring an operational baseline image representing the body part. However, the operational baseline image may be acquired in any way, and is not limited to any way (e.g., before administration of the contrast agent, or after administration of the contrast agent at an operational baseline dose lower than the operational dose).

[0165] In one embodiment, this medical method includes the step of administering a contrast agent to a patient in an operational dose. However, the method of administering the contrast agent is optional and not limited to (e.g., using a syringe, an infusion pump, administering it in advance, administering it immediately before acquiring operational images, administering it continuously during acquisition, etc.).

[0166] In one embodiment, the medical method includes the step of acquiring one or more operational administration images representing a body part in response to the administration of a contrast agent to a patient. (A corresponding operational simulation image is simulated from the operational baseline image and the operational administration image, and a representation of the body part based on the operational simulation image is output according to the method described above.) However, any number of operational administration images can be acquired, and the method of acquisition is also arbitrary (e.g., acquired at any delay time after contrast agent administration, acquired during administration, acquired continuously, acquired at a specific point in time, etc.).

[0167] In one embodiment, the medical method includes the step of performing a medical procedure related to a body part based on a representation of that body part. However, this medical procedure may be of any type (e.g., a diagnostic procedure, a therapeutic procedure, a surgical procedure, etc.) and is not limited to such procedures.

Claims

1. A method (400) for training a machine learning model (305) used in a medical image processing application, wherein the method (400) is controlled by a computing system (130). The computing system (130) is provided with a plurality of incomplete sample sets for each plurality of imaging diagnostic procedures related to a plurality of body parts of a plurality of subjects (403-410), wherein each incomplete sample set includes at least one sample target image representing the corresponding body part of the subject to which a contrast agent was administered at a sample target dose, and at least one sample baseline image representing the corresponding body part of the subject to which no contrast agent was administered, or to which a contrast agent was administered at a sample baseline dose lower than the sample target dose (403-410), The computing system (130) simulates a plurality of sample source images (411-445, 450-456), wherein from each of at least a portion of the incomplete sample set of each imaging diagnostic procedure, it simulates at least one sample source image representing the corresponding body part, such that it simulates the administration of a contrast agent to the corresponding subject at a sample source dose lower than the sample target dose, and in this case, the ratio of the sample source dose to the sample target dose is equal to the downscaling coefficient (411-445, 450-456), The computing system (130) generates one or more complete sample sets for each imaging diagnostic procedure (446, 457), wherein each complete sample set includes a sample baseline image and a sample target image obtained from one of the incomplete sample sets of the imaging diagnostic procedure, and a sample source image simulated from another incomplete sample set. The computing system (130) is used to train a machine learning model (305) (458-473) to optimize its ability to generate a sample target image from the sample baseline image and sample source image of each complete sample set. The computing system (130) deploys the trained machine learning model (305) to a medical image processing application (475), thereby mimicking an increase in the dose of contrast agent administered to a corresponding patient based on an upscaling coefficient equivalent to the reciprocal of the downscaling coefficient. including, Method (400).

2. The sample source dose is located between the sample baseline dose and the sample target dose. The method according to claim 1 (400).

3. The aforementioned sample target dose is the full dose of a standard contrast agent used in clinical practice. The method according to claim 1 or 2 (400).

4. At least some of the subjects are animals, and the patients are humans. The method according to any one of claims 1 to 3 (400).

5. The above method (400) further, The computing system (130) receives multiple sample images (403) for each image diagnostic procedure, including one or more sample baseline images and one or more sample target images. The computing system (130) creates an incomplete sample set for each image diagnostic procedure based on the received sample images of the image diagnostic procedure (404-410) and including, The method according to any one of claims 1 to 4 (400).

6. The above method (400) further, The computing system (130) generates a new sample baseline image and / or a new sample target image from the received corresponding sample baseline image and / or sample target image for each image diagnostic procedure (407-408). including, The method according to claim 5 (400).

7. The above method (400) further, The computing system (130) generates new sample baseline images and / or new sample target images in each image diagnostic procedure in order to equalize the number of sample baseline images and sample target images in the image diagnostic procedure (407-408). including, The method according to claim 6 (400).

8. The above method (400) further, The computing system (130) discards one or more sample baseline images and / or sample target images for each image diagnostic procedure (405-406). The method according to any one of claims 5 to 7 (400), including the method according to any one of claims 5 to 7.

9. The above method (400) further, The computing system (130) discards sample baseline images and / or sample target images for each image diagnostic procedure (405-406), thereby equalizing the number of sample baseline images and sample target images in the image diagnostic procedure. The method according to claim 8 (400), including the method according to claim 8.

10. The above method (400) further, The computing system (130) creates an incomplete sample set (410) for each of one or more imaging diagnostic procedures. Includes, The above-mentioned creation (410) is, Combining a single sample baseline image with multiple sample target images from the imaging diagnostic procedure, The imaging diagnostic procedure involves establishing a one-to-one correspondence between a corresponding number of sample baseline images and sample target images, or To generate all possible combinations of sample baseline images and sample target images in the said imaging diagnostic procedure. The method according to any one of claims 1 to 9 (400), carried out by...

11. The above method (400) further, The computing system (130) creates a complete sample set for each of one or more of the image diagnostic procedures (446; 457) Includes, The act of creating the aforementioned (446; 447) is, The aforementioned image diagnostic procedure involves creating a one-to-one correspondence between half of the incomplete sample set and the sample source images simulated from the remaining half. The aforementioned image diagnostic procedure generates all possible combinations of an incomplete sample set and sample source images simulated from other incomplete sample sets, or The aforementioned image diagnostic procedure involves creating a one-to-one correspondence between an incomplete sample set and a sample source image simulated from another incomplete sample set. The method according to any one of claims 1 to 10, carried out by (400).

12. Each sample baseline image, each sample target image, and each sample source image contains multiple sample baseline values, multiple sample target values, and multiple sample source values, respectively. The above method (400) further, The computing system (130) calculates each sample source value (418; 426; 436) for each sample source image by applying a simulation formula that depends on the downscaling coefficient. Includes, The simulation formula is derived based on a signal law that represents the magnitude of the response signal of a biological site, with the local concentration of the contrast agent as a function. The method according to any one of claims 1 to 11 (400).

13. The aforementioned simulation formula is derived from a signaling law that has been linearized with respect to the local concentration of the contrast agent. The method according to claim 12 (400).

14. The aforementioned simulation formula is derived from a signaling law that assumes a direct proportional relationship between local concentration and contrast agent dose. The method according to claim 13 (400).

15. The sample baseline value, sample target value, and sample source value represent the response signal at the corresponding biological site location. The above method (400) further, The computing system (130) modulates the downscaling coefficient used to calculate each sample source value for each sample source image based on the suggestion of the local concentration of the contrast agent at the corresponding location derived from the corresponding sample target value (417; 425). including, The method according to any one of claims 12 to 14 (400).

16. The above method (400) further, The computing system (130) injects artificial noise having a statistical distribution dependent on a downscaling coefficient into each sample source image (420-421; 428-429; 438-442). including, The method according to any one of claims 1 to 15 (400).

17. The above method (400) further, The computing system (130) trains an additional machine learning model (370) (450) to optimize its ability to generate sample source images of at least a portion of the complete sample set from corresponding sample baseline images and sample target images, The computing system (130) applies the sample baseline images and sample target images of the corresponding incomplete sample set to the additional machine learning model (370) that has been trained, and generates improved versions of each sample source image (451-456). including, The method according to any one of claims 1 to 16 (400).

18. The above method (400) further, The computing system (130) repeatedly simulates the sample source images for multiple downscaling coefficient values ​​(411-445; 450-456), generates the complete sample set (446; 457), and trains the machine learning model (305) (458-473) (474), and The computing system (130) deploys the machine learning model (305) in a corresponding configuration trained with the downscaling coefficient values ​​so that one or more corresponding values ​​of the downscaling coefficient can be selected when used in each medical image processing application (475). including, The method according to any one of claims 1 to 17 (400).

19. The aforementioned machine learning model (305) is a neural network. The method according to any one of claims 1 to 18 (400).

20. A method for using the machine learning model (305) trained by the method (400) of any one of claims 1 to 18 to image a patient's body part in a medical image processing application, This method is performed under the control of a computing system (115), The computing system (115) receives one or more operational administration images representing body parts of patients in which a contrast agent has been administered at a predetermined operational dose, and at least one operational baseline image representing body parts of patients in which no contrast agent has been administered, or in which the contrast agent has been administered at an operational baseline dose lower than the operational dose. The computing system (115) uses the machine learning model to simulate corresponding operational simulation images that represent the patient's body parts as if a contrast agent had been administered at an operational simulation dose higher than the operational dose, wherein the ratio of this operational simulation dose to the operational dose is equal to the upscaling coefficient corresponding to the reciprocal of the downscaling coefficient of the machine learning model, and The computing system (115) outputs a representation of body parts based on the simulation image during operation. To do method.

21. A computer program (300) for causing a computing system (130; 115) to execute the method according to any one of claims 1 to 20 when the computing system (130; 115) is executed on said computing system.

22. A computer program product comprising one or more non-temporary computer-readable storage media, wherein the non-temporary computer-readable storage media stores program instructions together, and when the program instructions are read by a computing system, the computing system causes the computing system to execute the method according to any one of claims 1 to 20. Computer program product.

23. A computing system (130; 115) comprising means (500) configured to perform each step of the method according to any one of claims 1 to 20.

24. A computing system including a circuit for performing each step of the method according to any one of claims 1 to 20.

25. A medical method applied to a part of a patient's body, This medical method is To obtain at least one operational baseline image representing the aforementioned body part, The patient is administered a contrast agent at a predetermined operational dose, In response to administering the contrast agent to the patient, one or more operational administration images representing the body parts are acquired. The method according to claim 20 involves simulating a corresponding operational simulation image from the operational baseline image and the operational administration image, outputting a representation of a body part based on the operational simulation image, and Performing medical procedures related to the body part in accordance with the expression of the body part: including, Medical methods.

Images