Image processing device, image processing method, and magnetic resonance imaging device

Abstract

To accurately enhance the image quality of an inputted image regardless of the type of the inputted image.SOLUTION: An image processing device generates a learning model obtained by learning so as to enhance the image quality of a first image. The device accepts the first image and a second image regarding an identical object, and enhances the image quality of the first image by using a learned model to obtain a first high-quality image. Using the first high-quality image and the second image as inputted image, the device generates a high-quality image of the second image having an image quality of the first high-quality image while keeping the contrast of the second image.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an image processing technique for improving the quality of images obtained by a medical imaging device such as a magnetic resonance imaging (hereinafter referred to as an MRI) device. [Background technology]

[0002] Generally, MRI examinations execute multiple imaging sequences to acquire multiple types of images. An imaging sequence describes the timing of application of radio frequency magnetic fields (RF) and gradient magnetic fields (Gs, Gp, Gr) in each axial direction, and various sequences are used depending on the type of image to be acquired. In MRI examinations, diagnosis is made based on the multiple types of images acquired by executing multiple sequences.

[0003] The parameters that determine the imaging sequence (repetition time TR, echo time TE, inversion time TI, flip angle FA, etc.) are called imaging parameters, and the degree of enhancement of the image obtained by imaging is determined by the type of sequence (spin echo, gradient echo, EPI, etc.) and the imaging parameters. The imaging parameters are adjusted in various ways depending on the target area, disease, etc.

[0004] As mentioned above, examinations that acquire multiple types of images take a long time and place a great burden on both the patient and the examiner. For this reason, in MRI examinations, imaging is sometimes performed with reduced resolution to shorten the examination time. In this case, ringing artifacts (also known as truncation artifacts or truncation artifacts) occur due to the high-frequency components of the echo signal being cut off. Ringing artifacts are artifacts that appear as fine stripes on the periphery of the image, and a common method of suppressing them is to apply a low-pass filter to the image. However, applying a low-pass filter to the image has the problem of blurring.

[0005] Meanwhile, in recent years, technologies for improving the quality of low-quality images using machine learning have been developed, and are being widely applied to medical images such as MR images. For example, Patent Document 1 discloses a high-quality image improvement technology that uses a model trained to input a low-frequency component image and output a corrected image that reduces the influence of non-uniformity in high-frequency magnetic fields. Furthermore, Patent Document 2 discloses a method using a two-stage neural network (NN) to obtain a third resolution image with even higher resolution from a low-resolution first resolution image via a higher-resolution second resolution image. By using such machine learning to estimate a high-resolution image from a low-resolution image, it is expected that the ringing and blurring described above can be reduced. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2020-121032 [Patent Document 2] Japanese Patent Application Publication No. 2018-151747 DISCLOSURE OF THE INVENTION [Problem to be solved by the invention]

[0007] However, while high-quality image processing using conventional learning models or machine learning with neural networks can produce highly accurate results for images captured under the same imaging conditions (contrast and resolution) as those used for training, there is a problem in that accuracy drops for images captured under different imaging conditions. Furthermore, it is difficult to train images under all imaging conditions in advance due to the time and amount of data required.

[0008] The present invention has been made in view of the above circumstances, and has as its object to perform high-precision image quality improvement processing regardless of the type of input image. [Means for solving the problem]

[0009] The present invention solves the above problem by performing image quality improvement processing on one image and then using the results to perform a second image quality improvement processing on another image.

[0010] That is, the image processing device of the present invention comprises an image receiving unit that receives a first image of the same object and a second image of a different type from the first image, a first image quality improvement processing unit that improves the image quality of the first image received by the receiving unit using an image quality improvement function that has been trained to improve the image quality of the first image, and a second image quality improvement processing unit that improves the image quality of the second image using the first image quality improvement generated by the first image quality improvement processing unit and the second image.

[0011] Here, "images of different types" means images that differ in at least one of the following: type of device (modality) used for imaging, imaging conditions (degree of enhancement (contrast) of biological tissue or physical quantities, imaging parameters, imaging sequence, etc.), and imaging time (date and time of imaging, time elapsed after administration of contrast agent, respiratory phase, cardiac phase, etc.).

[0012] The image processing method of the present invention also includes a learning step of generating a first image quality improvement function that has been trained to improve the image quality of a first image; an image acceptance step of accepting a first image and a second image of the same object; a first image quality improvement step of improving the image quality of the first image accepted in the image acceptance step using the first image quality improvement function; and a second image quality improvement step of improving the image quality of the second image using the first high-image quality image obtained in the first image quality improvement step and the second image accepted in the image acceptance step as inputs.

[0013] The MRI apparatus of the present invention also includes an imaging unit that generates nuclear magnetic resonance signals in the subject and collects the nuclear magnetic resonance signals generated from the subject, and a computer that processes the nuclear magnetic resonance signals to generate an image, and the computer has the functions of the image processing device of the present invention described above. [Effects of the Invention]

[0014] According to the present invention, high image quality can be achieved for various images without preparing a high image quality function for each type of image. Furthermore, according to the present invention, it is possible to reduce the time and achieve high image quality in MR examinations in which multiple images are acquired at one time. [Brief explanation of the drawings]

[0015] [Figure 1] A block diagram illustrating an embodiment of an image processing device and a medical imaging device. [Figure 2] FIG. 2 is a diagram showing the flow of operations of the image processing device of FIG. 1; [Figure 3] FIG. 1 is a diagram showing an overview of processing performed by the image processing apparatus according to the first embodiment; [Figure 4] FIG. 10 is a diagram showing an example of the second image quality improvement process according to the first embodiment. [Figure 5] FIG. 10 is a diagram showing another example of the second image quality improvement process of the first embodiment. [Figure 6] FIG. 10 is a diagram showing yet another example of the second image quality improvement processing of the first embodiment. [Figure 7] FIG. 10 is a block diagram showing the overall configuration of an image processing apparatus according to a second embodiment. [Figure 8] FIG. 10 is a diagram showing an outline of processing performed by the image processing apparatus according to the second embodiment; [Figure 9] FIG. 10 is a diagram showing an example of a second image quality improvement process according to the second embodiment. [Figure 10] FIG. 10 is a diagram showing another example of the second image quality improvement process according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0017] First, the overall configuration of the image processing device will be described with reference to Fig. 1. The image processing device 200 is a device that receives images captured by the medical imaging device 100 and performs image quality improvement processing, and includes an image receiving unit 210 that receives images, image quality improvement processing units (first image quality improvement processing unit 230, second image quality improvement processing unit 240) that perform image quality improvement processing on the images received by the image receiving unit 210, and a learning unit 220 that generates image quality improvement functions (e.g., learning models) used by the image quality improvement processing units 230 and 240.

[0018] The image receiving unit 210 receives at least two different types of images. These two types of images may be directly transmitted from the medical imaging device 100 or may be images stored in a medical image database 500 such as a PACS. While FIG. 1 illustrates a single medical imaging device 100, the medical imaging device 100 may be multiple imaging devices with different modalities, such as MRI devices and CT devices, or multiple imaging devices installed in different locations. The two types of images include images acquired by different medical imaging devices, as described above, as well as images acquired by the same medical imaging device but with different contrasts, imaging parameters, and imaging times. When the medical imaging device 100 is an MRI device, various images with different contrasts, such as T1W images, T2W images, and FLAIR images, can be obtained by varying the imaging sequence and imaging parameters. These images may be acquired simultaneously within a single examination or at different times. In this embodiment, these images are processed as "different types of images" (first image, second image).

[0019] The image quality improvement processing unit includes a first image quality improvement processing unit 230 that performs processing using as input the first image received by the image receiving unit 210, and a second image quality improvement processing unit 240 that performs processing using as input both the high-quality image that is the output of the first image quality improvement processing unit 230 and the second image received by the image receiving unit 210. As an image quality improvement function (algorithm) for the image quality improvement processing, the first image quality improvement processing unit 230 can employ a convolutional neural network (CNN) or a known machine learning algorithm, or a method of creating a dictionary using only high-quality images (sparse modeling or sparse coding).

[0020] The second image quality improvement processing unit 240 improves the image quality of an image (second image) of a type different from the first image using a high-quality image (first high-quality image) of the first image as a guide image, and can use, as its algorithm, a filter that smooths while preserving edges, such as a joint bilateral filter or a guided filter. An unsupervised learning model such as a GAN (Generative Adversarial Networks) may also be used. Details of the processing will be described later.

[0021] The learning unit 220 generates an image quality improvement function (learning model) to be used in the image quality improvement processing unit 230. Specifically, a set of a large number of first images and their high-quality images is used as learning images, and an unlearned model is trained to output a high-quality image using the first images as input. The high-quality image of the first image is, for example, an image of the same type as the first image but with a higher resolution, or an image from which artifacts such as rigging artifacts and noise have been removed. The image used may be an image acquired under high-resolution imaging conditions or an image acquired under imaging conditions in which artifacts do not occur. The first image and its high-quality image are stored as a large number of sets, for example, in a database 500, and are accepted by the image accepting unit 210 and passed to the learning unit 220. When one type of image (first image) is input to a learning model (such as a CNN), the learning unit 220 determines weights and coefficients so that the learning model outputs a high-quality image of that image. Note that, although a case where a set of low-quality images and high-quality images is used as a learning method for the learning model, depending on the content of the image quality improvement processing, only low-quality images may be used.

[0022] The operation of the image processing device of this embodiment with the above configuration will be described with reference to Fig. 2. Here, it is assumed that the learning model used by the first image quality improvement processing unit 230 has been learned by the learning unit 220.

[0023] First, the image receiving unit 210 receives a plurality of types of images (first and second images) to be processed (S1). The first image quality improvement processing unit 230 reads the results of learning by the learning unit 220 (for example, CNN weighting coefficients) (S2) and performs image quality improvement processing on the first image (S3). Next, the second image quality improvement processing unit 240 uses the first high-quality image, which is the processing result of the first image quality improvement processing unit 230, as a guide image and performs image quality improvement processing on the second image (S4). Through the above processing, a high-quality image is obtained for each of the plurality of types of images that have been input.

[0024] Through the above processing, it is possible to highly accurately improve the image quality of each type of image without constructing a learning model using a large amount of training data for each type of image. Note that although the case where there are two different types of images has been described here, similar processing is possible when three or more types of images are input by providing multiple second image quality improvement processors 240 or by performing processing by the second image quality improvement processor 240 in multiple stages. When multiple second image quality improvement processors 240 are provided, the algorithm used may be changed as appropriate depending on the type of image.

[0025] <Embodiment 1> Next, an embodiment will be described in which the medical imaging device 100 is an MRI device and a plurality of images acquired by the MRI device are processed.

[0026] As shown in Fig. 1, the medical imaging device 100 is broadly composed of an imaging unit 110 and a computer 120. The configuration of the imaging unit 110 and the function of the computer 120 vary depending on the type of medical imaging device 100. In the case of an MRI device, although not shown, the imaging unit 110 includes a static magnetic field magnet that generates a static magnetic field, a gradient magnetic field coil that generates a gradient magnetic field in a static magnetic field space, an RF transmitting coil that applies a pulsed high-frequency magnetic field to a subject placed in the static magnetic field space, and an RF receiving coil that detects nuclear magnetic resonance signals (echo signals) generated from atomic nuclei (such as protons) that constitute the subject's tissues by the application of the high-frequency magnetic field. The imaging unit 110 further includes a power supply or high-frequency magnetic field generator that drives the gradient magnetic field coil and the RF transmitting coil, a signal processing unit that processes signals received by the RF receiving coil, and a sequencer that controls the application of high-frequency magnetic field pulses and gradient magnetic field pulses and the measurement of echo signals according to a predetermined pulse sequence.

[0027] The computer 120 can be configured as a general-purpose computer or workstation equipped with a CPU, GPU, and memory, and includes a control unit that controls the operation of the entire device and a calculation unit that performs calculations such as image reconstruction using signals processed by the signal processing unit, and further includes a user interface unit (including a display device and input device) for displaying the processing results and for the user to input data and commands.

[0028] 1 may be built in such a computer 120, or may be a device independent of the MRI device. Also, some of the functions (e.g., some of the calculations) performed by the computer 120 or the image processing device 200 can be realized by hardware such as a programmable logic device (PLD).

[0029] In this embodiment, an example will be described in which an image processing device 200 (including a case where it is built in an MRI device) learns a T2 weighted image (T2W image), which is essential in many MRI examinations, as a first image. Since the configuration of the image processing device shown in Fig. 1 and the processing flow shown in Fig. 2 are common to this embodiment as well, these drawings will be referenced in the following description. Also, Fig. 3 shows an overview of the processing of this embodiment.

[0030] [Learning Steps] As shown in Fig. 3, first, the learning unit 220 uses a set 400 of training T2W image data to learn an image quality improvement function to be used in the first image quality improvement processing unit 230. In this embodiment, a CNN is used as the image quality improvement function. The set 400 of training T2W image data includes one low-quality T2W image and the other high-quality T2W image. The learning unit 220 uses a large number of training image data to determine the weights, activation function, etc. of the CNN so that the output of the CNN will be a high-quality T2W image in response to an input of a low-quality T2W image.

[0031] As is well known, CNNs have multiple convolutional layers and are designed to obtain the desired output (an output similar to the training data) by optimizing parameters such as the weight coefficients and biases of the convolutional layers. Various layer configurations have been proposed for CNNs, and it is also possible to configure them to include layers with different properties, such as pooling layers, in addition to convolutional layers. CNN layer configurations and activation functions can be predetermined, but can also be selected appropriately depending on the target image and the image quality improvement processing content. It is also possible to prepare multiple CNNs and select the appropriate one depending on the processing content.

[0032] For example, when the image quality improvement by the first image quality improvement processor 230 is a high-resolution process, the learning unit 220 may prepare several patterns with different R / M ratios depending on the ratio of the number of reconstruction matrices (image size) R of the first image to the number of measurement matrices M (either the number of phase encodings or the number of frequency encodings) in the configuration and learning of the CNN (layer structure and activation function). It is known that the interval between ringings caused by zero filling is roughly proportional to the number of reconstruction matrices R / the number of measurement matrices M. Therefore, even if the number of matrices R and M of the images obtained during inspection (images to be processed) vary, it is believed that ringing can be removed with high accuracy as long as the R / M is the same. Therefore, by selecting and using a CNN that matches the R / M of the target image, high-accuracy high-resolution processing can be performed even if the number of reconstruction matrices R and the number of measurement matrices of the target image vary. Furthermore, since it is sufficient to learn using several patterns with different R / M ratios, less training data is required.

[0033] [First high-definition processing] The image processing device 200 improves the image quality of a plurality of images to be processed, assuming that the CNN has been learned by the learning unit 220. The image receiving unit 210 receives a plurality of types of images 401, 402 (S1). One of the plurality of types of images is a T2W image 401, and the other 402 is any image such as a PDW (proton density weighted) image or a FLAIR image, and both are low-quality images, for example, low-resolution images obtained by high-speed imaging.

[0034] Of the received multiple images, the first image quality improvement processor 230 first processes a T2W image 401 and outputs a high-quality T2W image (for example, a high-resolution image) 403. At this time, if the first image quality improvement processor 230 has multiple CNNs corresponding to the reconstruction matrix / measurement matrix (R / M) of the image, it selects a CNN corresponding to the R / M of the input T2W image and performs processing (S2, S3).

[0035] The processing of the first image quality improvement processor 230 has been explained above using CNN as an example, but it is also possible to use other machine learning techniques, sparse modeling, etc., in addition to CNN as the image quality improvement function.

[0036] [Second high-resolution processing] The second image quality improvement processing unit 240 receives the high quality T2W image 403 output from the first image quality improvement processing unit 230 and an image other than a T2W image (for example, a PDW image) 402 received by the image receiving unit 210, and performs image quality improvement processing (S4). The second image quality improvement processing is processing that improves the image quality of the processing target image using the high quality T2W image as a guide image or reference image, and uses a guided filter, joint bilateral filter, GAN, or the like as an image quality improvement function.

[0037] Hereinafter, with reference to FIG. 4, the image quality improvement process when using a guided filter will be described.

[0038] 4A shows an image to be processed (e.g., a PDW image) 402, (B) shows a T2W high-quality image 403, and (C) shows an output image 404 from the second image quality improvement processor 240. In processing using a guided filter, pixels within predetermined patches of two images, image 402 and image 403, are first extracted. When the pixel value of image 402 is y and the pixel value of image 403 is x, the following equation (1) is used: y=ax+b (1) The coefficient a and intercept b are calculated by approximating the above equation, and the following equation (2) is used to calculate the coefficient a and intercept b. z=ax+b (2) The value z calculated in the above is set as the output pixel value for this patch.

[0039] This process is performed for all patches while moving the patch position. At this time, the patches are moved so that adjacent patches overlap. For pixels in the overlapping area of ​​the patches, the average pixel value calculated for each patch is used as the pixel value of the corresponding pixel in the final output image 404. By performing the above process, an output image 404 is obtained in which noise (ringing artifacts) contained in the target image 402 has been removed and the edges of the target image 402 have been preserved. In other words, an image of the same type as the target image 402 but with higher image quality is obtained.

[0040] 4 again, a process using a joint bilateral filter will be described as another example of a filter used in the second image quality improvement processing unit 240. Like the guided filter, the joint bilateral filter is a filter that smoothes while preserving edges, but it calculates pixel values ​​of the output image 404 using pixel values ​​of neighboring pixels according to the following equation (3).

[0041]

number

[0042] "1 / k(p)" is a coefficient that makes the sum of the weights 1, and k(p) is expressed by the following formula.

number

[0043] In this way, the joint bilateral filter not only gives a spatial weight but also gives a large weight to pixels having similar brightness values ​​in the guide image 403, so that an output image that reflects the edges of the guide image can be obtained.

[0044] It is also possible to use a neural network (NN) as the image quality improvement function of the second image quality improvement processor 240 instead of the filter described above. Processing using a neural network will be described with reference to FIG. 5. (A) in FIG. 5 shows a first high-image-quality image (referred to herein as a source image) 402, (B) shows a second image (referred to herein as a target image) 403, and (C) shows an output image 404 of the NN. In this processing, a process is performed to convert the appearance (appearance characteristics such as contrast) of the target image 403 while maintaining the structure of the source image 402. In the example shown in the figure, multiple types of images are used as target images, and the appearance characteristics of each are specified to obtain multiple images that have the structure of the source image and the appearance of each target image.

[0045] For this reason, during training, a pair of a first image (T2W image) and various other images (PDW image, FLAIR image, etc.) is used as input to perform training, and an NN is constructed that converts the source image into the appearance (contrast, etc.) specified in the target image. In other words, the conversion process is learned. During use, the first high-quality image output from the first image quality improvement processing unit 230 is used as the source image 403, and the second image accepted by the image accepting unit 210 is input as the target image 402, thereby obtaining an image in which the first high-quality image has been converted into the appearance of the second image, i.e., a second high-quality image 404.

[0046] In this high image quality processing, the conversion processing itself is learned, so even if an image captured under different conditions than that used during learning is input as the target image for the second image 402, a high image quality image can be output.

[0047] As described above, according to this embodiment, it is possible to perform high-precision image quality improvement processing without depending on the imaging conditions of the second image.

[0048] <Modification of the First Embodiment> In the first embodiment, an example of performing image quality improvement processing using a first image and a second image as input has been described, but it is also possible to further use a third image or a third high-quality image to improve the accuracy of the image quality improvement processing of the second image (second high-quality image processing). In MR examinations, three or more types of images are often acquired, and this can be utilized.

[0049] An example of image quality improvement processing using the third image will be described using a guided filter as an example.

[0050] 6, for example, three images are input to the second image quality improvement processor 240: a second image (image to be processed) 402 to be processed, a first high-quality image 403 obtained by improving the image quality of the first image by the first image quality improvement processor 230, and a third high-quality image 405. The third high-quality image 405 is a different type of image from the first and second images. As in the above example, if the first image is a T2W image and the second image is a PDW image, the third image is a FLAIR image, etc. If the third image is acquired under high-resolution imaging conditions, the third high-quality image 405 may be used as is, or may be one that has been improved in image quality by known image quality improvement processing such as filtering.

[0051] As when using two types of images, the second image quality improvement processing unit 240 uses the pixel values ​​of the patches at corresponding locations (pixel value y of image 402, pixel value x1 of image 403, and pixel value x2 of image 405) to calculate the coefficients a and b and intercept c of equation (5). y=ax1+bx2+c (5) These coefficients a, b and section c The output pixel value z for this patch is calculated using z=ax1+bx2+c (6)

[0052] This is calculated for all patches, and the pixel values ​​z of the overlapping pixel positions are averaged to obtain the pixel value of the second high quality image 404 .

[0053] In this way, by using not only the first high quality image 403 but also the third image and the third high quality image 405, the accuracy of the second high quality image 404 can be improved.

[0054] Although this modified example has been described using a guided filter as an example, the third image or the third high-quality image can also be used when a joint bilateral filter or NN is used as the image quality improvement function.

[0055] <Other variations> In the above explanation, the image quality improvement processing has mainly been described as increasing the resolution of a low-resolution image, but the image quality improvement processing also includes a case where the input image (first image and second image) is a noisy image and the noise is reduced, a case where the k-space is undersampled and the artifacts and noise are removed by undersampling the image, an artifact reduction process for body movement, breathing, etc., and a process that appropriately combines these processes with the high-resolution processing. In any case, this can be achieved by designing and training an image quality improvement function (e.g., CNN) to be used in the first image improvement processing unit 230 for one type of image using pre-processing and post-processing learning data.

[0056] <Embodiment 2> In the first embodiment and its variants, the first high-quality image was used to improve the quality of the second image, but the present embodiment is characterized by performing local processing taking into account noise that may be present in the first image and inconsistencies in the local structures between the first and second images.

[0057] The processing by the second image quality improvement processor in the first embodiment is based on the premise that the same structures are visible between images. However, depending on the imaging conditions, locally different structures may be visible. For example, blood and blood vessels appear black in T2*W images, whereas cerebral infarction areas appear white in DWI images. Therefore, if bleeding or the like is present, the structure of that area will appear different in the two images. Such areas are likely to become blurred after image quality improvement. Furthermore, the accuracy of image quality improvement decreases when the guide image (first high-quality image) contains noise. This embodiment prevents degradation of image quality improvement processing by performing localized processing.

[0058] As shown in Fig. 7, image processing device 200 of this embodiment adds a map calculation unit 250 that creates an adjustment map for adjusting the image quality improvement processing to the configuration shown in Fig. 1. The content of the image processing also differs in that, as shown in Fig. 8, the second image quality improvement processing uses the adjustment map calculated by map calculation unit 250 (addition of adjustment map calculation processing S3-1). The following describes this embodiment, focusing on the differences from embodiment 1.

[0059] In this embodiment, as in the first embodiment, the first image quality improvement processing unit 230 performs image quality improvement processing on the first image, and the image quality improvement function used by the first image quality improvement processing unit 230 (for example, a CNN learned by the learning unit 220) is also the same. The map calculation unit 250 calculates the adjustment map 400 using any of the first image 401, second image 402, or first high-quality image 403 accepted by the image accepting unit 210. Furthermore, if a third image has also been acquired for the same subject in addition to the first and second images, the third image may be used.

[0060] The adjustment map 400 is an image in which pixel values ​​are weighted to weight each pixel value or each patch when the first high-quality image 403 and the second image 402 are used to improve image quality, and the weights are calculated based on the reliability of each pixel in the images used to create the map and the correlation between the images. Specifically, when the adjustment map 400 is calculated using, for example, the first image 401 or the first high-quality image 403 alone, the local variance and entropy of the image are calculated, and a weight w (0≦w≦1) is calculated based on the calculated weights and used as the pixel values ​​of the map. Both the local variance and entropy indicate the variation in pixel values, and the greater the variation, the more likely it is that noise is included (the lower the reliability), and the weight value is set to be such that the local variance and entropy are small. The same applies when a third image is used.

[0061] Furthermore, when using the first image 401 or its high-quality image 403 and the second image 402, the local correlation coefficient, mutual information, etc. between the images are calculated and used as pixel values. Since it can be said that the higher the correlation between the two, the higher the structural similarity, the pixel value that increases the weight w is set. Furthermore, a combination of multiple different maps may be used as the adjustment map.

[0062] The adjustment map 400 does not necessarily have to be a map of the entire region of the image. For example, if there is knowledge of regions where bleeding is expected or regions where noise is likely to be mixed in, an image representing the region of interest, such as a segmentation image or an edge extraction image that extracts a specific region, may be created, and a map of only the region of interest may be created.

[0063] Next, an example of processing by the second image quality improvement processor 240 using the above-mentioned adjustment map will be described. As in the first embodiment, the function used in the image quality improvement processing may be a guided filter, a joint bilateral filter, a GAN, or the like.

[0064] As shown in Fig. 9, the second image quality improvement process S4 includes an inter-image conversion process S41, a single image quality improvement process S42, and an image synthesis process S43. The inter-image conversion process S41 is the same process as the second image quality improvement process S4 shown in Fig. 3, and improves the image quality of the second image using the first high-quality image 403 as a guide image. In the example shown, this is a conversion process of a PDW image using a T2W image as a guide. The output of this process is designated Y1.

[0065] The single image quality improvement process S42 uses the second image 402 as input and performs general image quality improvement processes such as bilateral filtering, sequential reconstruction using sparsification constraints, and CNN trained on multiple types of images. This process is a general process that uses only the second image, and differs from the second image quality improvement process S4 in Figure 3. The output of this process is designated as Y2.

[0066] In the image synthesis process S43, the two high-quality images Y1 and Y2 are synthesized as follows: For adjustment The images are combined using a map 400. The adjustment map 400 is, for example, the absolute value of the correlation coefficient between the first image and the second image. The pixel value Z of the combined image is expressed by the following equation (7), where the pixel value of the adjustment map 400 is used as a weight w. Z=wY1+(1-w)Y2 (7)

[0067] When the weight w is calculated based on the correlation between the two images, if the correlation is high, the accuracy of the results obtained in the inter-image conversion process S41 is considered to be high, so the weight of the output Y1 is increased. As a result, for areas with high correlation, high-precision image quality improvement is performed using a guide image (here, a T2W image), and for areas with low correlation, an image that reflects more of general image quality improvement is obtained. Furthermore, when w is calculated based on the variance of the first image, for example, if the variance is large and the pixel values ​​vary widely, the accuracy of the results obtained in the inter-image conversion process S41 is considered to be low, so synthesis is performed with a small weight Y1.

[0068] According to this embodiment, it is possible to prevent a decrease in the accuracy of the high-quality image processing due to differences in the local structure between the guide image and the image to be processed, or the influence of noise contained in the guide image, and to maintain the accuracy of the high-quality image processing (second high-quality image processing).

[0069] In the above explanation, the adjustment map 400 is applied when combining the outputs Y1 and Y2 of the inter-image conversion process S41 and the single image quality improvement process S42, but as shown in Fig. 10, it is also possible to apply the adjustment map 400 (weighting) to the first high-quality image 403 and second image 402, which are the inputs, in the inter-image conversion process S41. In this case, the process uses the following equation (8) instead of the equation (2) shown above) of the inter-image conversion process S41 (for example, the guided filter). z=wy+(1-w)(ax+b) (8)

[0070] Furthermore, the second image quality improvement processor 240 can be configured so that the image quality improvement function is configured as a CNN, the first high-image quality image, the second image, and the adjustment map are input to the CNN, and a second high-image quality image that has been subjected to local image quality improvement processing is output. As described in the modified example of the first embodiment (FIG. 5), this CNN performs processing to match the appearance characteristics of the image to be processed to the source image 402, and at this time, performs the processing locally in accordance with the adjustment map to achieve image quality improvement.

[0071] The specific processing of embodiment 2 and its variations have been described above. However, the various variations and alternative means described in embodiment 1 can be applied to this embodiment alone or in combination, as long as there is no technical contradiction, and such variations are encompassed by the present invention. [Explanation of symbols]

[0072] 100: Medical imaging device, 110: Imaging unit, 120: Computer, 200: Image processing device, 210: Image receiving unit, 220: Learning unit, 230: First image quality improvement processing unit, 240: Second image quality improvement processing unit, 250: Map calculation unit, 500: Database

Claims

1. an image receiving unit that receives a first image of the same object and a second image that is different from the first image in at least one of modality, contrast, and imaging parameters; a first image quality improvement processing unit that improves the image quality of the first image received by the image receiving unit using an image quality improvement function that has been trained to improve the image quality of the first image; An image processing device characterized by comprising: a second image quality improvement processing unit that improves the image quality of the second image using at least one of a guided filter or a joint bilateral filter that uses the first high-image quality image generated by the first image quality improvement processing unit as a guide image, or a neural network that is trained to convert the appearance of the source image into the target image using the first high-image quality image as a source image and the second image as a target image.

2. 2. The image processing device according to claim 1, The image processing device further comprises a learning unit that learns the image quality improvement function.

3. 2. The image processing device according to claim 1, The image processing device is characterized in that the image quality improvement function is configured by a convolutional neural network.

4. 2. The image processing device according to claim 1, The image processing device is characterized in that the image quality improvement function is a learning model learned using a first image without artifacts as learning data, and the first image quality improvement processing unit generates an image without artifacts as the first high-quality image.

5. 2. The image processing device according to claim 1, An image processing device characterized in that the image quality improvement function is a learning model learned using a high-resolution image of a first image as learning data, and the first image quality improvement processing unit generates a high-resolution image as the first high-quality image.

6. 6. The image processing device according to claim 5, the first image and the second image are images captured by a magnetic resonance imaging apparatus, the first image quality improvement processor includes a plurality of image quality improvement functions, each having a different configuration or a different learning process, depending on a ratio between a reconstruction matrix and a measurement matrix of the first image; an image processing device that selects an image quality improvement function to be used from the plurality of image quality improvement functions based on a ratio between a reconstruction matrix and a measurement matrix of a first image received by the image receiving unit;

7. 2. The image processing device according to claim 1, The image processing device further comprising a map calculation unit that calculates an adjustment map for adjusting the image quality improvement processing in the second image quality improvement processing unit.

8. 8. The image processing device according to claim 7, The image processing device is characterized in that the map calculation unit calculates the adjustment map based on at least one of the first image and second image accepted by the image accepting unit, and the first high-quality image generated by the first high-quality image processing unit.

9. 8. The image processing device according to claim 7, The image processing device, wherein the map calculation unit calculates a correlation between the first image and the second image, and calculates the adjustment map using the correlation.

10. 8. The image processing device according to claim 7, a third image quality improvement processing unit that independently improves the image quality of the second image; An image processing device further comprising an image synthesis unit that uses the adjustment map to synthesize a high-quality image of the second image generated by the second image quality improvement processing unit and a high-quality image generated by the third image quality improvement processing unit.

11. 8. The image processing device according to claim 7, The image processing device is characterized in that the second image quality improvement processing unit performs local image quality improvement processing using the adjustment map in image quality improvement processing using the first high-image quality image and the second image.

12. a training step of generating a first image enhancement function trained to enhance the first image; an image receiving step of receiving a first image of the same object and a second image that is different from the first image in at least one of modality, contrast, and imaging parameters; a first image quality improvement step of improving image quality of the first image received in the image receiving step by using the first image quality improvement function; and a second image quality improvement step of improving the image quality of the second image using at least one of a guided filter or a joint bilateral filter with the first high-quality image as a guide image, or a neural network trained to convert the appearance of the source image into the target image with the first high-quality image as a source image and the second image as a target image, using the first high-quality image obtained in the first image quality improvement step and the second image as input.

13. an imaging unit that generates a nuclear magnetic resonance signal in an object to be examined and collects the nuclear magnetic resonance signal generated from the object to be examined, and a computer that processes the nuclear magnetic resonance signal to generate an image, 12. A magnetic resonance imaging apparatus, wherein the computer comprises the image processing device according to claim 1.

Images