Image processing apparatus, image processing method, and magnetic resonance imaging apparatus

Abstract

The present application relates to an image processing apparatus, an image processing method, and a magnetic resonance imaging apparatus. High-precision high-quality processing is performed regardless of the kind of an input image. A learning model learned in a manner of performing high-quality processing on a first image is generated. A first high-quality image is obtained by performing high-quality processing on the first image using the learned model with respect to a second image of the same subject as the first image. The first high-quality image and the second image are input, and a high-quality image of the second image having the quality of the first high-quality image is generated while maintaining the contrast of the second image.

Description

Technical Field

[0001] This invention relates to image processing techniques for improving the image quality of images obtained in medical imaging devices such as magnetic resonance imaging (MRI) devices. Background Technology

[0002] Generally, in an MRI examination, multiple imaging sequences are performed to acquire multiple image types. The imaging sequences describe the timing of the application of high-frequency magnetic fields (RF) and tilting magnetic fields (Gs, Gp, Gr) in each axis, with various imaging sequences used corresponding to the type of image desired. Diagnosis in an MRI examination is based on the multiple images obtained by performing multiple sequences.

[0003] The parameters that determine the imaging sequence (repetition time TR, echo time TE, reversal time TI, flip angle FA, etc.) are called imaging parameters. The emphasis of the resulting image depends on the type of sequence (spin echo, gradient echo, EPI, etc.) and the imaging parameters. Imaging parameters are adjusted in various ways depending on the subject area, disease, etc.

[0004] As mentioned above, obtaining multiple images during an MRI examination takes a long time, placing a significant burden on both the patient and the examiner. Therefore, in order to shorten the examination time, the resolution is sometimes reduced during imaging in MRI examinations. In this case, ringing artifacts (also called truncation artifacts) occur due to the cutoff of high-frequency components of the echo signal. Ringing artifacts appear as fine stripes around the edges of the image, and applying a low-pass filter is a common method to suppress them. However, applying a low-pass filter can lead to blurring.

[0005] On the other hand, in recent years, techniques for improving the image quality of low-resolution images using machine learning have been developed and are being widely applied to medical images such as MR images. For example, Patent Document 1 discloses a model-based image quality improvement technique that learns by taking a low-frequency component image as input and outputting a corrected image that reduces the influence of inhomogeneities in the high-frequency magnetic field. Furthermore, Patent Document 2 discloses a method that uses a two-stage neural network (NN) to obtain a high-resolution third image from a low-resolution first-resolution image through a second-resolution image with a higher resolution than the first-resolution image. By using such machine learning to infer a high-resolution image from a low-resolution image, it is expected that the aforementioned ringing and blurring can be reduced.

[0006] Existing technical documents

[0007] Patent documents

[0008] Patent Document 1: JP 2020-121032

[0009] Patent Document 2: JP 2018-151747

[0010] However, in high-resolution image enhancement based on machine learning using existing learning models and neural networks, there is a challenge: while high-precision results can be obtained on images with the same camera conditions (contrast / resolution) as the images used for learning, the accuracy decreases on images with different camera conditions. Furthermore, from a time / data volume perspective, it is difficult to learn from images under all camera conditions beforehand. Summary of the Invention

[0011] The present invention is proposed in view of the above-mentioned matters, with the aim of performing high-precision, high-quality image enhancement processing regardless of the type of the input image.

[0012] This invention performs high-quality enhancement processing on one image and uses the result to perform a second high-quality enhancement processing on other images, thereby solving the aforementioned problem.

[0013] That is, the image processing apparatus of the present invention includes: an image receiving unit that receives a first image of the same object and a second image of a different type from the first image; a first high-definition processing unit that uses a high-definition function learned in a manner for high-definition processing of the first image to perform high-definition processing on the first image received by the image receiving unit; and a second high-definition processing unit that uses the first high-definition image generated by the first high-definition processing unit and the second image to perform high-definition processing on the second image.

[0014] Here, "different types of images" means images that differ in at least one of the following: the type of device used in the imaging (modality), the conditions during imaging (organism tissue, degree of emphasis on physical quantities (contrast), imaging parameters, imaging sequence, etc.), and the time of imaging (date and time of imaging, time elapsed after contrast agent administration, respiratory phase, heartbeat phase, etc.).

[0015] Furthermore, the image processing method of the present invention includes: a learning step, generating a first high-quality enhancement function learned in a manner for enhancing a first image; an image receiving step, receiving a first image and a second image of the same object; a first high-quality enhancement step, using the first high-quality enhancement function to enhance the first image received through the image receiving step; and a second high-quality enhancement step, using the first high-quality image obtained through the first high-quality enhancement step and the second image received through the image receiving step as inputs to enhance the second image.

[0016] Furthermore, the MRI apparatus of the present invention includes: an imaging unit that generates magnetic resonance signals from the subject of examination and collects the magnetic resonance signals generated from the subject of examination; and a computer that processes the magnetic resonance signals to generate an image, wherein the computer has the functions of the image processing apparatus of the present invention described above.

[0017] Invention Effects

[0018] According to the present invention, high-quality enhancement can be achieved for various types of images without preparing high-quality enhancement functions for each type of image. Furthermore, according to the present invention, in MR inspections that acquire multiple images at once, both time reduction and high-quality imaging can be achieved. Attached Figure Description

[0019] Figure 1 This is a block diagram illustrating one embodiment of an image processing apparatus and a medical imaging device.

[0020] Figure 2 It means Figure 1 A flowchart illustrating the operation of the image processing device.

[0021] Figure 3 This is a diagram illustrating the general outline of the processing of the image processing apparatus in Embodiment 1.

[0022] Figure 4 This diagram illustrates an example of the second high-definition processing in Implementation Method 1.

[0023] Figure 5 This is a diagram illustrating another example of the second high-definition processing of Implementation 1.

[0024] Figure 6 This is a diagram illustrating another example of the second high-definition processing of Implementation 1.

[0025] Figure 7 This is a block diagram showing the overall structure of the image processing apparatus in Embodiment 2.

[0026] Figure 8 This is a diagram illustrating the general outline of the processing of the image processing apparatus in Embodiment 2.

[0027] Figure 9 This diagram illustrates an example of the second high-definition processing in Implementation Method 2.

[0028] Figure 10 This is a diagram illustrating another example of the second high-definition processing in Implementation 2.

[0029] Explanation of reference numerals in the attached figures

[0030] 100: Medical imaging device; 110: Imaging unit; 120: Computer; 200: Image processing device; 210: Image receiving unit; 220: Learning unit; 230: First high-definition processing unit; 240: Second high-definition processing unit; 250: Map calculation unit; 500: Database. Detailed Implementation

[0031] The embodiments of the present invention will be described below with reference to the accompanying drawings.

[0032] First refer to Figure 1 The overall structure of the image processing apparatus will be explained below. The image processing apparatus 200 is an apparatus that receives images captured by the medical imaging device 100 and performs high-quality processing on them. It includes: an image receiving unit 210 that receives images; high-quality processing units (first high-quality processing unit 230 and second high-quality processing unit 240) that perform high-quality processing on the images received by the image receiving unit 210; and a learning unit 220 that generates high-quality functions (e.g., learning models) used by the high-quality processing units 230 and 240.

[0033] The image receiving unit 210 receives at least two different types of images. These two types of images can be images directly transmitted from the medical imaging device 100, or images stored in a medical image database 500 such as a PACS. The medical imaging device 100... Figure 1 The diagram shows one device, but it could also be multiple imaging devices of different modes, such as MRI or CT, or multiple imaging devices located in different places. The two types of images include not only images different from those acquired by the medical imaging device, as shown here, but also images acquired by the same medical imaging device, and images with different contrast, imaging parameters, and imaging time. When the medical imaging device 100 is an MRI device, various images with different contrasts, such as T1W images, T2W images, and FLAIR images, can be obtained by differentiating the imaging sequence and imaging parameters. Furthermore, these images are sometimes acquired simultaneously within a single examination, and sometimes not simultaneously. In this embodiment, these images are designated as "images of different types" (first image, second image) and are treated as processing objects.

[0034] The high-resolution processing unit includes: a first high-resolution processing unit 230 that processes a first image received by the image receiving unit 210 as input; and a second high-resolution processing unit 240 that processes a high-resolution image, the output of the first high-resolution processing unit 230, and the second image received by the image receiving unit 210, as input. The first high-resolution processing unit 230 can employ a convolutional neural network (CNN), a known machine learning algorithm, or a method of creating a dictionary using only high-resolution images (sparse modeling, sparse coding) as the high-resolution processing function (algorithm).

[0035] The second image enhancement processing unit 240 uses the high-quality image of the first image (the first high-quality image) as a guide image to enhance the image (the second image) of a different type than the first image. Its algorithm can use filters such as joint bilateral filters and guided filters to smooth the image while preserving edges. Alternatively, unsupervised learning models such as GANs (Generative Adversarial Networks) can be used. Details of the processing will be described later.

[0036] The learning unit 220 generates the high-resolution function (learning model) used in the high-resolution processing unit 230. Specifically, the unlearned model is learned in the following manner: a large set of first images and their high-resolution images is used as the learning images; the first images are used as input, and the high-resolution images are output. The high-resolution images of the first images are, for example, images of the same type as the first images but with higher resolution, or images with artifacts such as ringing artifacts and noise removed; images obtained under high-resolution camera conditions or under camera conditions that do not produce artifacts are used. The first images and their high-resolution images are stored as a large set, for example, in the database 500, and are received by the image receiving unit 210 and handed over to the learning unit 220. When an image of a certain type (the first image) is input into the learning model (CNN, etc.), the learning unit 220 determines the weights and coefficients of the high-resolution images of that image so that the learning model outputs the high-resolution images of that image. In addition, the learning method for the learning model describes the case of using a combination of low-quality and high-quality images, but depending on the content to be processed in high-quality mode, sometimes only low-quality images are used.

[0037] refer to Figure 2 The operation of the image processing apparatus of this embodiment with the above structure will be explained. Here, the learning model used by the first high-definition processing unit 230 is the model learned by the learning unit 220.

[0038] First, the image receiving unit 210 receives multiple images (a first image and a second image) as the objects of processing (S1). The first high-quality image processing unit 230 reads the learning result (e.g., the weight coefficients of a CNN) performed by the learning unit 220 and performs high-quality image processing on the first image (S3). Next, the second high-quality image processing unit 240 uses the processing result of the first high-quality image processing unit 230, i.e., the first high-quality image, as a guide image and performs high-quality image processing on the second image (S4). Through the above processing, high-quality images can be obtained from the multiple input images respectively.

[0039] Through the above processing, high-precision high-quality enhancement of each image can be achieved without constructing a learning model using a large amount of teaching data for each image of different types. Furthermore, while this describes the case of two different types of images, by setting up multiple second high-quality enhancement processing units 240, or by performing processing in multiple stages with the second high-quality enhancement processing units 240, the same processing can be achieved even when three or more types of images are input. When multiple second high-quality enhancement processing units 240 are set up, the algorithm used can be appropriately varied according to the type of image.

[0040] <Implementation Method 1>

[0041] The following describes an embodiment in which the medical imaging device 100 is an MRI device and processes multiple images acquired by the MRI device.

[0042] like Figure 1 As shown, the medical imaging device 100 generally includes an imaging unit 110 and a computer 120. The structure of the imaging unit 110 and the function of the computer 120 vary depending on the type of medical imaging device 100. In the case of an MRI device, the imaging unit 110 includes: a static magnetic field magnet that generates a static magnetic field; a tilting magnetic field coil that generates a tilting magnetic field in the static magnetic field space; an RF transmitting coil that applies a pulsed high-frequency magnetic field to a subject disposed in the static magnetic field space; and an RF receiving coil that detects nuclear magnetic resonance signals (echo signals) generated from the atomic nuclei (protons, etc.) of the tissue constituting the subject by the application of the high-frequency magnetic field. Furthermore, the imaging unit 110 includes a power supply or a high-frequency magnetic field generator for driving the tilting magnetic field coil and the RF transmitting coil, a signal processing unit for processing the signals received by the RF receiving coil, and a sequence generator for controlling the application of high-frequency magnetic field pulses, tilting magnetic field pulses, and the measurement of echo signals according to a given pulse sequence.

[0043] The computer 120 can be composed of a general-purpose computer or workstation equipped with a CPU, GPU and memory, including a control unit for controlling the overall operation of the control device and an arithmetic unit for performing operations such as image reconstruction using signals processed by the signal processing unit, and further includes a user interface unit (including a display device and an input device) for displaying processing results or for receiving data or instructions from the user.

[0044] Figure 1 The image processing device 200 shown can be built within such a computer 120, or it can be a device independent of the MRI device. Furthermore, a portion of the functions performed by the computer 120 and the image processing device 200 (e.g., a portion of the computation) can be implemented using hardware such as a PLD (programmable logic device).

[0045] In this embodiment, the example will be described where the image processing apparatus 200 (including those built into an MRI apparatus) learns T2-emphasis images (T2W images), which are essential in most MRI examinations, as the first image. In this embodiment, also because... Figure 1 The structure of the image processing device shown and Figure 2 The processing flow shown is common, therefore, these figures are also referenced in the following description. Furthermore, in Figure 3 The outline of the processing in this embodiment is shown below.

[0046] [Learning Steps]

[0047] like Figure 3 As shown, firstly, the learning unit 220 uses the set 400 of T2W image data for learning to learn the high-definition function used in the first high-definition processing unit 230. In this embodiment, a CNN is used as the high-definition function. The set 400 of T2W image data for learning includes both low-definition T2W images and high-definition T2W images. The learning unit 220 uses a large amount of learning image data to determine the weights, activation functions, etc., of the CNN, so that for input of a low-definition T2W image, the output of the CNN becomes a high-definition T2W image.

[0048] As is widely known, CNNs have multiple convolutional layers. The desired output (the same output as the teaching data) is obtained by optimizing the parameters such as the weights and biases of these convolutional layers. Various layer structures have been proposed for CNNs, including structures that include pooling layers and other layers with different properties in addition to convolutional layers. The layer structure and activation functions of a CNN can use predetermined structures and activation functions, but they can also be appropriately selected according to the image being processed or the high-quality content. Furthermore, multiple CNNs can be prepared, and selection can be made based on the processing content.

[0049] For example, if the high-quality processing in the first high-quality processing unit 230 is high-resolution processing, the learning unit 220 can also prepare several different patterns of R / M corresponding to the ratio of the number of reconstruction matrices (image size) R to the number of measurement matrices M (either the number of phase codes or the number of frequency codes) of the first image during the structure and learning of the CNN (layer structure, activation function). The ringing interval caused by zero-padding is known to be approximately proportional to the number of reconstruction matrices R / the number of measurement matrices M, and it is assumed that even if the number of matrices R and M of the image obtained during inspection (the image becoming the processing target) varies, as long as R / M is the same, ringing removal can be performed with high precision. Therefore, by selecting a CNN that matches the R / M of the processing target, high-precision high-resolution processing can be performed even if the number of reconstruction matrices R and the number of measurement matrices of the processing target image vary. Furthermore, since only a few different patterns of R / M need to be used for learning, a small amount of learning data is not a problem.

[0050] [First-class high-definition processing]

[0051] The image processing apparatus 200 performs high-quality enhancement on multiple images as the processing targets, based on the CNN learning of the learning unit 220 described above. The image receiving unit 210 receives multiple images 401 and 402 (S1). One of the multiple images is the T2W image 401, and the others 402 are arbitrary images such as PDW (proton density emphasis) images and FLAIR images, all of which are low-quality images, such as low-resolution images obtained by high-speed photography.

[0052] The first high-quality image processing unit 230 first processes the T2W image 401 among the received multiple images and outputs a high-quality T2W image (e.g., a high-resolution image) 403. At this time, if the first high-quality image processing unit 230 has multiple CNNs corresponding to the reconstruction matrix / measurement matrix (R / M) of the image, it selects the CNN corresponding to the R / M of the input T2W image for processing (S2, S3).

[0053] The above uses CNN as an example to illustrate the processing of the first high-definition processing unit 230. However, as a high-definition function, in addition to CNN, other machine learning methods, sparse modeling, etc. can also be used.

[0054] [Second High-Quality Image Processing]

[0055] The second image enhancement processing unit 240 takes as input the high-quality T2W image 403, which is the output of the first image enhancement processing unit 230, and an image other than the T2W image (e.g., a PDW image) 402 received by the image receiving unit 210, and performs image enhancement processing (S4). The second image enhancement processing uses the high-quality T2W image as a guide image or reference image to enhance the image to be processed. As the image enhancement function, a guided filter, a joint bilateral filter, GAN, etc., can be used.

[0056] The following is for reference Figure 4 This illustrates the high-quality processing using a guided filter.

[0057] Figure 4 (A) represents the image (e.g., PDW image) 402 of the object being processed, (B) represents the high-quality image 403 of the T2W, and (C) represents the output image 404 of the second high-quality processing unit 240. In the processing using the guided filter, pixels within a given patch of the two images 402 and 403 are first extracted. When the pixel value of image 402 is set to y and the pixel value of image 403 is set to x, the following equation (1) is used.

[0058] y = ax + b (1)

[0059] To approximate the result, we calculate the coefficients a and the intercept b, and then use these coefficients a and the intercept b from the following equation (2).

[0060] z = ax + b (2)

[0061] The calculated value z is used as the output pixel value in that block.

[0062] This processing is performed on all blocks while moving their positions. At this time, the blocks are moved in a manner that makes adjacent blocks repeat each other. For the pixels of the repeated portions of a block, the average of the pixel values ​​obtained for each block is used as the pixel value of the corresponding pixel in the final output image 404. Through the above processing, an output image 404 is obtained that removes noise (ringing artifacts) contained in the object image 402 while preserving the edges of the object image 402. That is, a high-quality image of the same type as the object image 402 is obtained.

[0063] As another example of the filters used in the second high-definition processing unit 240, see again Figure 4 To illustrate the processing using the joint bilateral filter. The joint bilateral filter, like the guided filter, is a filter that smooths the edges without changing them. The pixel values ​​of the output image 404 are calculated using the pixel values ​​of the neighboring pixels by the following formula (3).

[0064]

Mathematical Expression 3

[0065]

[0066] In the formula, p is the coordinate of a pixel, p' is the coordinate of the pixel within the nearest neighbor (Ω) of p, X is the pixel value of the guiding image 403, and Y is the pixel value of the processing object image 402. The subscripts represent the pixel coordinates. Furthermore, in the above formula, g(gd, gr) is a Gaussian function, and gd(pp') represents the spatial weight; closer pixels have greater weight. Additionally, gr(Xp - Xp') is a weight based on pixel values; closer pixel values ​​have greater weight.

[0067] “l / k(p)” is the coefficient that makes the sum of the weights equal to 1, and k(p) is represented by the following formula.

[0068]

Mathematical Expression 4

[0069] k(p)=∑ p′∈Ω g d (p′-p)g r (X p′ -X p′ (4)

[0070] In this way, the joint bilateral filter can produce an output image that reflects the edges of the guide image because, in addition to spatial weights, the weights of pixels with similar brightness values ​​in the guide image 403 are also increased.

[0071] The high-definition function of the second high-definition processing unit 240 can also use a neural network (NN) instead of the filters described above. (See reference) Figure 5 This will illustrate the processing using NN. Figure 5 (A) is the first high-quality image (here referring to the source image) 402, (B) is the second image (here referring to the target image) 403, and (C) is the output image 404 of the NN. In this process, the source image 402 is transformed into the visual appearance (appearance characteristics such as contrast) of the target image 403 while maintaining its original structure. In the illustrated example, multiple images are used as target images, and their respective visual appearance characteristics are specified, resulting in multiple images whose structure is the source image and whose visual appearance is the target image.

[0072] Therefore, during learning, pairs of the first image (T2W image) and various images different from it (PDW image, FLAIR image, etc.) are used as input to construct a neural network that transforms the source image into a visually displayed image (contrast, etc.) specified by the target image. That is, the transformation process is learned. In use, the output of the first high-definition processing unit 230, i.e., the first high-definition image, is used as the source image 403, and the second image received by the image receiving unit 210 is used as the target image 402 as input, thereby obtaining a visually displayed image that transforms the first high-definition image into the second image, i.e., the second high-definition image 404.

[0073] In this high-quality image enhancement process, since the transformation process itself has been learned, even if an image with different shooting conditions than during the learning process is used as the target image, a high-quality image can be output as the second image 402.

[0074] As explained above, according to this embodiment, high-precision, high-quality image processing can be performed without relying on the imaging conditions of the second image.

[0075] <Modification of Implementation Method 1>

[0076] In Implementation 1, an example of high-quality processing using the first and second images as inputs was described. However, a third image or a third high-quality image can be further used to improve the accuracy of the high-quality processing of the second image (second high-quality processing). In MR inspection, more than three images are often obtained, so this can be taken advantage of.

[0077] The example of using a guided filter illustrates the use of high-quality processing of a third image.

[0078] In this variation, such as Figure 6 As shown, the second high-resolution processing unit 240 receives, for example, three images: a second image (processing target image) 402, a first high-resolution image 403 obtained by the first high-resolution processing unit 230 after re-resolution of the first image, and a third high-resolution image 405. The third high-resolution image 405 is an image of a different type than the first and second images. Similar to the previous example, if the first image is a T2W image and the second image is a PDW image, then the third image is a FLAIR image, etc. When the third image is obtained under high-resolution imaging conditions, the high-resolution image 405 of the third image can be used as is, or it can be an image obtained by re-resolution processing such as known filter processing.

[0079] The second high-definition processing unit 240, in the same manner as when using two images, uses the pixel values ​​of the corresponding blocks (pixel value y of image 402, pixel value x1 of image 403, and pixel value x2 of image 405) to obtain the coefficients a, b and intercept c of equation (5).

[0080] y = ax1 + bx2 + c (5)

[0081] Use these coefficients a, b, and the intercept b to calculate the output pixel value z in the block.

[0082] z = ax₁ + bx₂ + c (6)

[0083] The pixel value z is calculated across all blocks and averaged over the repeated pixel locations to set as the pixel value of the second high-quality image 404.

[0084] In this way, by using not only the first high-quality image 403, but also the third image and the third high-quality image 405, the accuracy of the second high-quality image 404 can be improved.

[0085] Furthermore, this variation was illustrated using the case of a guided filter, but as a high-quality enhancement function, a third image or a third high-quality image can also be used when using a joint bilateral filter and a neural network.

[0086] <Other variations>

[0087] The above explanation primarily describes the case of high-resolution image enhancement, which is the process of resolving low-resolution images to high-resolution ones. However, when the input images (the first and second images) are noisy and noise reduction is performed, or when the input images are downsampled from k-space and artifacts / noise removal is performed as a result, high-resolution image enhancement also includes artifact reduction processing for body motion, breathing, etc., and further processing that appropriately combines these processing with high-resolution enhancement. In any case, this can be achieved by designing and learning the high-resolution function (e.g., CNN) used in the first high-resolution image enhancement unit 230 using pre-processing and post-processing training data for an image.

[0088] <Implementation Method 2>

[0089] In Embodiment 1 and its variations, the second image is enhanced using the first high-quality image. However, this embodiment is characterized by considering the noise that may exist in the first image, the mismatch in the local structure of the first image and the second image, and performing local processing.

[0090] The second high-resolution processing unit in Embodiment 1 assumes that the same structures can be seen between images. However, depending on the imaging conditions, sometimes locally different structures may be observed. For example, in a T2*W image, blood and blood vessels are depicted as black, while in a DWI image, a cerebral infarction site is depicted as white. Therefore, if there is bleeding, the structure of that area will appear different in the two images. In such areas, the likelihood of further blurring after high-resolution processing is high. Furthermore, if there is noise in the guiding image (the first high-resolution image), the accuracy of high-resolution processing will also decrease. This embodiment prevents the degradation of high-resolution processing by performing localized processing.

[0091] The image processing apparatus 200 of this embodiment is as follows: Figure 7 As shown, in Figure 1 The structure shown includes an additional image calculation unit 250 for generating adjustment images for high-resolution processing. Furthermore, regarding image processing, such as... Figure 8 As shown, the adjustment image calculated by the image calculation unit 250 in the second high-definition processing (an addition to the adjustment image calculation process S3-1) is different in this respect. The following description focuses on the differences from Embodiment 1.

[0092] In this embodiment, the first high-definition processing unit 230 performs high-definition processing on the first image, just as in Embodiment 1. The high-definition function used by the first high-definition processing unit 230 (e.g., a CNN learned by the learning unit 220) is also the same. The image calculation unit 250 calculates the adjustment image 400 using any one of the first image 401, the second image 402, or the first high-definition image 403 received by the image receiving unit 210. Furthermore, if a third image is obtained in addition to the first and second images for the same subject, the third image can also be used.

[0093] The adjustment image 400 is an image whose pixel values ​​are weighted using weights applied to each pixel value or each block during high-quality enhancement using the first high-quality image 403 and the second image 402. These weights are calculated based on the reliability of each pixel in the images used in image creation, the correlation between images, etc. Specifically, for example, when the adjustment image 400 is calculated using only the first image 401 or the first high-quality image 403, the local variance and entropy of the image are calculated, and a weight w (0 ≤ w ≤ 1) is calculated based on these and used as the pixel value of the image. Local variance and entropy both represent the deviation of pixel values; a larger deviation is considered to indicate a higher probability of noise (lower reliability), and the weight is set to a smaller pixel value. The same applies when using a third image.

[0094] Furthermore, when using the first image 401 or its high-resolution image 403 and the second image 402, the local correlation coefficient and mutual information between the images are calculated as pixel values. It can be said that the higher the correlation between the two, the higher the similarity of the constructed image; therefore, the pixel value that increases the weight w is set as such. Furthermore, an image synthesized from multiple different images can also be used as an adjustment image.

[0095] The adjustment of Figure 400 does not necessarily have to be a map of the entire image region. For example, if there is an idea of ​​a bleeding area or an area that is prone to noise, then it is also possible to create an image representing the region of interest, such as a segmented image or an edge extraction image from which a specific region has been extracted, and to create a map of only the region of interest.

[0096] Next, an example of the processing of the second high-definition processing unit 240 using the adjustment diagram described above will be explained. The functions used in the high-definition processing are the same as in Embodiment 1, and can include guided filters, joint bilateral filters, GANs, etc.

[0097] Second high-definition processing S4, such as Figure 9 As shown, it includes image transformation processing S41, single-image enhancement processing S42, and image compositing processing S43. Image transformation processing S41 is related to... Figure 3 The second high-quality enhancement process S4 shown is the same as the previous process, using the first high-quality image 403 as a guide image for high-quality enhancement of the second image. In the illustrated example, the transformation process uses the T2W image as the guide image for the PDW image. The output of this process is set as Y1.

[0098] The single-image high-quality enhancement process S42 takes the second image 402 as input and performs general high-quality enhancement processes such as bilateral filters, successive reconstruction with sparsity constraints, and CNNs learned from multiple images. This process is a general process that uses only the second image, and... Figure 3 The second high-resolution enhancement process, S4, is different. The output of this process is set to Y2.

[0099] In the image synthesis process S43, the modulation diagram 400 is used to synthesize two high-quality images Y1 and Y2. The modulation diagram 400 is, for example, the absolute value of the correlation coefficient between the first image and the second image. The pixel value Z of the synthesized image is represented by the pixel value of the modulation diagram 400 as the weight w, using the following equation (7).

[0100] Z = wY1 + (1 - w)Y2 (7)

[0101] When calculating the weight w based on the correlation of two images, it is considered that the accuracy of the result obtained through the inter-image transformation process S41 is high when the correlation is high, so the weight of the output Y1 is increased. Thus, for regions with high correlation, high-precision image enhancement is performed using the guide image (here, the T2W image), while for regions with low correlation, a generally high-quality image is obtained. Furthermore, when calculating w based on, for example, the variance of the first image, it is considered that the accuracy of the result obtained through the inter-image transformation process S41 is low when the variance is large and the pixel value deviation is large, so the weight of Y1 is reduced during synthesis.

[0102] Thus, according to this embodiment, the accuracy of high-quality processing can be prevented from decreasing due to differences in the local structure of the guide image and the processing target image, noise contained in the guide image, etc., and the accuracy of high-quality processing (second high-quality processing) can be ensured.

[0103] In the above description, the adjustment diagram 400 is used when combining the outputs Y1 and Y2 of the image transformation process S41 and the single-unit high-quality enhancement process S42, but it can still be done as described above. Figure 10 As shown, in the image transformation process S41, the first high-quality image 403 and the second image 402, which are inputs, are adjusted using the weighting formula 400. In this case, instead of the formula (e.g., the guide filter) of the image transformation process S41, the process is performed using the following formula (8).

[0104] z = wy + (1 - w)(ax + b) (8)

[0105] Furthermore, the second high-definition processing unit 240 can also construct a high-definition function using a CNN, taking the first high-definition image, the second image, and the adjustment image as input to the CNN, and outputting a second high-definition image that has undergone local high-definition processing. This CNN is a variation of Embodiment 1 (…). Figure 5 As explained in the document, although processing is performed to match the visual display characteristics of the image to the source image 402, high-quality enhancement is achieved by locally processing the image according to the adjustment diagram.

[0106] The above describes the specific processing and its variations in Embodiment 2. However, various variations and alternatives described in Embodiment 1 can also be used individually or in combination in this embodiment as long as they do not contradict each other technically. Such variations are included in this invention.

Claims

1. An image processing apparatus, characterized in that, have: The image receiving unit receives a first image of the same object and a second image of a different kind from the first image. The first high-definition processing unit uses a high-definition function learned in a way that high-definitions the first image to high-definition the first image received by the image receiving unit. The second high-definition processing unit uses the first high-definition image generated by the first high-definition processing unit and the second image to perform high-definition processing on the second image; and The image calculation unit calculates the adjustment image used to adjust the high-definition processing in the second high-definition processing unit. The image calculation unit calculates the adjustment image based on at least one of the first image and the second image received by the image receiving unit, and the first high-definition image generated by the first high-definition processing unit. The first image and the second image are MRI images or CT images.

2. The image processing apparatus according to claim 1, characterized in that, The image processing device also includes: The learning department will study the aforementioned high-definition function.

3. The image processing apparatus according to claim 1, characterized in that, The high-resolution function is constructed from a convolutional neural network.

4. The image processing apparatus according to claim 1, characterized in that, The high-quality enhancement function is a learning model that uses a first image without artifacts as learning data, and the first high-quality enhancement processing unit generates an image without artifacts as the first high-quality image.

5. The image processing apparatus according to claim 1, characterized in that, The high-resolution function is a learning model that uses a high-resolution image of the first image as learning data, and the first high-resolution processing unit generates a high-resolution image as the first high-resolution image.

6. The image processing apparatus according to claim 5, characterized in that, The first image and the second image are images captured by a magnetic resonance imaging device. The first high-definition processing unit has a ratio structure corresponding to the reconstruction matrix and measurement matrix of the first image, and multiple high-definition functions that differ in either of the learned components. Based on the ratio of the reconstruction matrix to the measurement matrix of the first image received by the image receiving unit, the high-definition function to be used is selected from the plurality of high-definition functions.

7. The image processing apparatus according to claim 1, characterized in that, The second high-definition processing unit uses one or more high-definition functions selected from the guided filter, the joint bilateral filter, and the neural network to perform high-definition processing.

8. The image processing apparatus according to claim 1, characterized in that, The image calculation unit calculates the correlation between the first image and the second image, and uses the correlation to calculate the adjustment image.

9. The image processing apparatus according to claim 1, characterized in that, The image processing device also includes: The third high-definition processing unit performs high-definition processing on the second image separately; and The image compositing unit uses the adjustment image to composite the high-quality image of the second image generated by the second high-quality processing unit and the high-quality image generated by the third high-quality processing unit.

10. The image processing apparatus according to claim 1, characterized in that, The second high-definition processing unit uses the adjustment map to perform local high-definition processing in the high-definition processing of the first high-definition image and the second image.

11. The image processing apparatus according to claim 1, characterized in that, The first image and the second image differ in at least one of the following: mode, contrast, camera parameters, and camera timing.

12. An image processing method, characterized in that, include: The learning steps generate a first high-resolution function that has been learned in a way that improves the resolution of the first image. The image receiving step involves receiving a first image of the same object and a second image of a different kind from the first image. The first high-definition step involves using the first high-definition function to enhance the first image received through the image receiving step. The second high-definition enhancement step takes the first high-definition image obtained through the first high-definition enhancement step and the second image received through the image receiving step as inputs to enhance the second image to high-definition quality. and The calculation steps involve calculating the adjustment diagram used to adjust the high-definition processing in the second high-definition step. In the image calculation step, the adjustment image is calculated based on at least one of the first image and the second image received through the image receiving step, and the first high-definition image obtained through the first high-definition step. The first image and the second image are MRI images or CT images.

13. A magnetic resonance imaging device, characterized in that, have: The imaging unit generates nuclear magnetic resonance signals from the object being examined and collects the nuclear magnetic resonance signals generated from the object being examined; and The computer processes the nuclear magnetic resonance signal to generate an image. The computer has the image processing apparatus according to any one of claims 1 to 11.

Images