Medical information processing device, medical information processing method, and medical information processing program
The medical information processing device addresses the lack of data consistency in machine learning models by implementing noise reduction and data consistency processes, enhancing image quality in medical imaging applications.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- CANON MEDICAL SYST CORP
- Filing Date
- 2022-06-23
- Publication Date
- 2026-05-15
AI Technical Summary
Existing machine learning models for medical imaging lack processes to improve data consistency across multiple resolutions, leading to suboptimal image quality.
A medical information processing device that includes an acquisition unit for multiple resolution data and a processing unit performing noise reduction and data consistency processes to generate high-quality output data, utilizing neural networks for noise reduction and data consistency improvement.
Enhances image quality by reducing noise and improving data consistency, thereby improving the performance of machine learning models in medical imaging.
Smart Images
Figure 0007859885000003 
Figure 0007859885000004 
Figure 0007859885000005
Abstract
Description
Technical Field
[0001] The embodiments disclosed in this specification and the drawings relate to a medical information processing apparatus, a medical information processing method, and a medical information processing program.
Background Art
[0002] With the progress of machine learning, the application of machine learning models has also advanced in the medical field. For example, there is a method of performing image reconstruction processing by applying a machine learning model to MR (Magnetic Resonance) images having a plurality of different resolutions. Generally, when applying a machine learning model to a medical image, it is known that the image quality is improved by improving the data consistency between the input data and the output data after applying the machine learning model. However, there has been no consideration of how to introduce a process for improving the data consistency into a machine learning model that processes a plurality of resolutions.
Prior Art Documents
Non-Patent Documents
[0003]
Non-Patent Document 1
Non-Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0004] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is to improve image quality. However, the problems that the embodiments disclosed herein and in the drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]
[0005] The medical information processing device according to this embodiment includes an acquisition unit and a processing unit. The acquisition unit acquires a plurality of resolution data, each having a different resolution. The processing unit performs one or more first noise reduction processes to reduce noise contained in the data and one or more first DC processes to improve the degree of data agreement for each of the plurality of resolution data, and generates output data corresponding to each of the plurality of resolution data. When processing the second and subsequent resolution data, the processing unit performs one or more second noise reduction processes and one or more second DC processes on the resolution data to be processed and one or more output data generated before the resolution data to be processed, and generates the output data. [Brief explanation of the drawing]
[0006] [Figure 1] Figure 1 is a block diagram showing a medical information processing device according to this embodiment. [Figure 2] Figure 2 shows a network model according to the first embodiment. [Figure 3] Figure 3 shows a network model according to the second embodiment. [Figure 4] Figure 4 is a flowchart showing an example of the learning process of the medical information processing device according to the second embodiment. [Figure 5] Figure 5 is a flowchart showing an example of the learning process for each subset network according to the second embodiment. [Figure 6A] Figure 6A shows an example of training for the first subset network. [Figure 6B] Figure 6B shows an example of training for the second subset network. [Figure 6C] Figure 6C shows an example of training for the third subnetwork. [Figure 7] Figure 7 shows a first modified example of the network model according to the second embodiment. [Figure 8] Figure 8 shows a second modified example of the network model according to the second embodiment. [Figure 9] Figure 9 shows a third modified example of the network model according to the second embodiment. [Modes for carrying out the invention]
[0007] Hereinafter, a medical information processing device, a medical information processing method, and a medical information processing program according to this embodiment will be described with reference to the drawings. In the following embodiments, parts with the same reference numerals perform similar operations, and redundant explanations will be omitted as appropriate. Hereinafter, one embodiment will be described with reference to the drawings.
[0008] (First Embodiment) The medical information processing device according to this embodiment will be described with reference to the block diagram in Figure 1. The medical information processing device 1 according to this embodiment includes a memory 11, an input interface 12, a communication interface 13, and a processing circuit 14.
[0009] The medical information processing device 1 according to the embodiment described herein may be included in a console, workstation, etc., or in a medical imaging diagnostic device such as an MRI (Magnetic Resonance Imaging) device or a CT (Computed Tomography) device.
[0010] The memory 11 stores various data, pre-trained models, etc. to be described later. The memory 11 is a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, a hard disk drive (HDD), a solid state drive (SSD), an optical disk, etc. Further, the memory 11 may be a drive device that reads and writes various information to and from a portable storage medium such as a CD-ROM drive, a DVD drive, a flash memory, etc.
[0011] The input interface 12 has a circuit that receives various instructions and information inputs from the user. The input interface 12 has, for example, a circuit related to a pointing device such as a mouse or an input device such as a keyboard. Note that the circuit included in the input interface 12 is not limited to a circuit related to physical operation components such as a mouse and a keyboard. For example, the input interface 12 may have an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the medical information processing device 1 and outputs the received electrical signal to various circuits within the medical information processing device 1.
[0012] The communication interface 13 exchanges data with an external device by wire or wirelessly. Since general communication means may be used for the communication method and the structure of the interface, the description here is omitted.
[0013] The processing circuit 14 includes an acquisition function 141, a noise reduction function 142, a DC function 143, a learning function 144, a model execution function 145, and a display control function 146. The noise reduction function 142 and the DC function 143 are also collectively referred to as a processing function. The processing circuit 14 has a processor not shown as a hardware resource.
[0014] The acquisition function 141 acquires a plurality of input data. The plurality of input data are medical data having different resolutions. The medical data may be collected at different resolutions respectively, or may be generated from data collected so as to include information of different resolutions respectively. Examples of the type of medical data include, for example, k-space data collected by an MRI (Magnetic Resonance Imaging) device, MR images, projection data collected by a CT (Computed Tomography) device, sinogram data, or CT images. In this embodiment, "resolution" assumes temporal resolution. Specifically, it is data having different temporal resolutions, such as data collected at 3 seconds per frame and data collected at 6 seconds per frame. Also, for example, k-space data radially collected by an MRI device can adjust the temporal resolution of the data according to the number of spokes used for image reconstruction, so it can be said to be data including information of different resolutions. Specifically, data reconstructed in units of 3 spokes and data reconstructed in units of 6 spokes are data having different temporal resolutions. Note that the resolution is not limited to the temporal resolution, and may be other data resolutions such as spatial resolution, and the processing of the medical information processing apparatus 1 according to this embodiment can be similarly applied to other data resolutions. Hereinafter, for convenience of explanation, a plurality of input data having different temporal resolutions may be distinguished and referred to as first-resolution data, second-resolution data, and the like.
[0015] The noise reduction function 142 executes a process of reducing noise included in the data. The data consistency (DC) function 143 executes a process of improving the degree of consistency (data consistency) of the data, for example, a process of reducing an error with the input data. In other words, the processing function including the noise reduction function 142 and the DC function 143 executes a plurality of noise reduction processes by the noise reduction function 142 and a plurality of DC processes by the DC function 143 for each of the plurality of input data, and generates output data corresponding to each of the plurality of input data.
[0016] The learning function 144 incorporates the noise reduction function 142 and the DC function 143 as part of the network model and is executed during the training of the network model. The learning function 144 trains the network model by executing processing functions using training data that includes multiple input data acquired by the acquisition function 141. Upon completion of training, a trained model is generated.
[0017] The model execution function 145 is executed when a pre-trained model is used. The model execution function 145 applies the pre-trained model to multiple input data acquired by the acquisition function 141, and outputs high-quality data as an execution result, for example, with improved noise reduction and data consistency.
[0018] The display control function 146 displays, for example, the learning status of the network model by the learning function 144 and the execution results by the model execution function 145 on an external display or the like.
[0019] Furthermore, the various functions of the processing circuit 14 may be stored in memory 11 in the form of programs that can be executed by a computer. In this case, the processing circuit 14 can be said to be a processor that realizes the functions corresponding to each program by reading and executing the programs corresponding to these various functions from memory 11. In other words, the processing circuit 14 in the state in which each program has been read will have multiple functions, etc., as shown in the processing circuit 14 of Figure 1.
[0020] In Figure 1, these various functions are explained as being realized by a single processing circuit 14, but it is also possible to configure the processing circuit 14 by combining multiple independent processors, and each processor realizes the functions by executing a program. In other words, each of the above functions may be configured as a program, and one processing circuit may execute each program, or a specific function may be implemented in a dedicated, independent program execution circuit.
[0021] Next, an example of a network model used in the medical information processing device 1 according to the first embodiment will be explained with reference to the conceptual diagram in Figure 2. The network model 20 used in the medical information processing device 1 according to the first embodiment includes a plurality of processing blocks 21 and an output block 22. Specifically, the plurality of processing blocks 21 are linked together, and the output block 22 is placed at the final stage as the output layer of the network model 20. The number of processing blocks 21 can be predetermined according to the task using the network model 20, for example, by linking together four processing blocks 21.
[0022] Each processing block 21 includes a noise reduction block 23 corresponding to the noise reduction function 142 and a number of DC blocks 24 corresponding to the DC function 143. The number of DC blocks 24 connected to the noise reduction block 23 should correspond to the number of different resolution data input to the network model 20. In the example in Figure 2, the DC blocks are placed after the noise reduction block 23, but the order of connection between the noise reduction block 23 and the DC blocks 24 is not important, and a configuration in which the noise reduction block 23 is placed after the DC blocks 24 is also acceptable.
[0023] The noise reduction block 23 is a neural network block composed of a deep neural network, such as a deep convolutional neural network. The noise reduction block 23 is a type of function and typically has learnable parameters. The noise reduction block 23 may be formed by a MoDL-type or unroll-type neural network disclosed in Non-Patent Literature 1, or by a recurrent neural network such as LSTM (Long Short Term Memory) or GRU (Gated Recurrent Unit). The noise reduction block 23 receives input data and outputs noise-reduced data from which the noise in the input data has been reduced.
[0024] The noise to be reduced includes, but is not limited to, noise caused by static magnetic field inhomogeneity, noise caused by data acquisition, noise caused by signal processing on the MR signal, and noise caused by data processing on k-space data and spectra, if the input data is MR data. Furthermore, if the input data is CT data, it includes, but is not limited to, noise caused by data acquisition such as metal artifacts and low-count artifacts.
[0025] DC block 24 is a block that reduces data errors in order to improve data matching. Like noise reduction block 23, DC block 24 is a type of neural network or function and typically has learnable parameters. DC block 24 should, for example, if the collected data is y, then perform the process of searching for an image x that minimizes equation (1) below. That is, if the collected data y is k-space data, it reduces the error of the k-space data Ax with respect to the k-space data y.
[0026]
number
[0027] A represents the Fourier transform, and "Ax" transforms the image x into k-space data. λR(x,x denoise ) represents the error term with respect to the output value of the noise reduction block 23. λ is a parameter related to the error term and is a value learned during the training of the network model 20. For example, the output value of the noise reduction block 23 x denoise If so, R(x) can be expressed as shown in equation (2) below. In this case, the minimization problem of equation (1) can be solved, for example, by the conjugate gradient method.
[0028]
number
[0029] DC block 24 receives noise-reduced data as input and outputs DC-processed data in which the error between the acquired data and the noise-reduced data has been reduced.
[0030] The output block 22 includes one noise reduction block 23 and one DC block 24. The DC block 24 is positioned to correspond to the time resolution of the resolution data to be output from the network model 20. In the example in Figure 2, one DC block 24 is positioned to output one high-quality data corresponding to the third resolution of the third resolution data from the network model 20, but the number of DC blocks 24 should be adjusted according to the number of data outputs from the network model 20.
[0031] In the network model 20, some or all of the noise reduction blocks 23 may have a common configuration, or they may all have different configurations. Similarly, some or all of the DC blocks 24 may have a common configuration, or they may all have different configurations.
[0032] During training of the network model 20, multiple resolution data, each with a different time resolution, are input to the first processing block 21 of the network model 20. In the example in Figure 2, first resolution data 25 with a first time resolution, second resolution data 26 with a second time resolution, and third resolution data 27 with a third time resolution are input to the processing block. For the sake of explanation, in this embodiment, we assume that the first time resolution has the lowest resolution, and the resolution data with the highest ordinal number among the input resolution data has the highest time resolution. That is, in the example in Figure 2, we assume that the first time resolution has the lowest time resolution, and the third time resolution has the highest time resolution.
[0033] Furthermore, in addition to the three different resolution data, an nth resolution data having an nth time resolution (where n is a natural number greater than or equal to 4) may also be input to the processing block 21. In this case, the processing block 21 should have DC blocks 24 corresponding to the n resolution data. Of course, the network model 20 may also be configured to have two DC blocks 24 for each processing block, processing two resolution data with different time resolutions.
[0034] On the other hand, in this embodiment, as the ground truth data during training, for example, ground truth data corresponding to the third resolution data with the highest temporal resolution can be used. The ground truth data may be pre-imaged data or historical data. The learning function 144 causes the processing circuit 14 to apply the network model 20 to multiple resolution data, thereby performing noise reduction processing and DC processing, and the output block 22 outputs output data in which the DC processed data from the first temporal resolution to the nth temporal resolution has been integrated.
[0035] The learning function 144 causes the processing circuit 14 to update the learnable parameters such as weights and biases of the noise reduction block 23 and DC block 24 of the network model 20, using methods such as stochastic gradient descent and backpropagation, in order to minimize the loss value between the ground truth data and the output data calculated by the loss function. Any learning method used in so-called supervised learning can be applied. Upon completion of learning, a trained model of the network model 20 is generated.
[0036] When using the network model 20, the processing circuit 14 inputs the first resolution data to the third resolution data into the network model 20 via the model execution function 145, and outputs high-quality data 28 that corresponds to the third time resolution and has reduced noise and error (improved data consistency) compared to the third resolution data. Note that while the noise reduction block 23 and DC block 24 are shown as being arranged alternately, they do not necessarily have to be arranged alternately. Also, the DC block 24 does not necessarily have to be placed after the noise reduction block 23. For example, there may be places where the noise reduction block 23 is placed consecutively.
[0037] According to the first embodiment described above, a network model is trained using multiple resolution data sets, each having a different time resolution. This model consists of one or more noise reduction blocks and one or more DC blocks, each corresponding to a different resolution data set. This improves the accuracy of network model training. Therefore, when using the trained model, high-quality data with reduced noise can be generated while error reduction considering data consistency is performed for each input data set with a different time resolution. This improves the performance of the trained model and enhances the output image quality.
[0038] (Second Embodiment) In the network model shown in Figure 2 according to the first embodiment, since multiple resolution data are processed in a single processing block, the number of DC blocks 24 increases as the number of processing blocks 21 increases, and the amount of computation required to obtain the output value tends to increase. Therefore, the second embodiment differs from the first embodiment in that it designs a subset network that performs noise reduction processing and DC processing a predetermined number of times for each resolution data. As a result, when calculating the parameters of a DC block 24 for a given resolution data, it is not necessary to calculate the parameter information of other resolution data placed in the preceding stage, thus reducing the computational cost related to DC processing. The functional configuration of the medical information processing device 1 according to the second embodiment is the same as that of the first embodiment, so its description is omitted here.
[0039] Next, the network model according to the second embodiment will be described with reference to Figure 3. The network model 30 shown in Figure 3 includes multiple subset networks 31 to correspond one-to-one with multiple resolution data. Specifically, a first subset network 31-1 is designed for the first resolution data, a second subset network 31-2 for the second resolution data, and a third subset network 31-3 for the third resolution data. The noise reduction block 23 and DC block 24 have similar configurations to those of the first embodiment. For example, the noise reduction block 23 and DC block 24 may have some or all of their blocks in common, or they may have all different configurations.
[0040] Each subset network 31 is assumed to have multiple alternating layers of noise reduction blocks 23 and DC blocks 24, but it is not necessary for them to be arranged in multiple alternating layers; for example, a single DC block 24 may be placed as the final block of the subset network 31. Also, while an example is shown in which sets in which a DC block 24 is placed after a noise reduction block 23 are executed multiple times, sets in which a noise reduction block 23 is placed after a DC block 24 may also be executed multiple times. Furthermore, in network model 30, the data is processed in the order of first resolution data to third resolution data, that is, from the resolution data with the lowest time resolution to the resolution data with the highest time resolution among the multiple resolution data.
[0041] Furthermore, the network model 30 generates output data for each subnetwork. The generated output data is input to each subnetwork corresponding to the subsequent multiple resolution data, along with the subsequent resolution data.
[0042] Next, an example of the learning process of the medical information processing device 1 according to the second embodiment will be described with reference to the flowchart in Figure 4 and the network model 30 in Figure 3. In step S401, the processing circuit 14 acquires the first resolution data using the acquisition function 141.
[0043] In step S402, the processing circuit 14 applies the first subset network 31-1 to the first resolution data using the processing function, and performs noise reduction processing by the noise reduction block 23 and DC processing by the DC block 24 multiple times. As a result, the first output data is output by the first subset network 31-1.
[0044] In step S403, the processing circuit 14 acquires the second resolution data using the acquisition function 141. In step S404, the processing circuit 14 applies the second subset network 31-2 to the second resolution data and the first output data from the first subset network 31-1, and noise reduction processing by the noise reduction block 23 and DC processing by the DC block 24 are performed multiple times. As a result, the second output data is generated by the second subset network 31-2.
[0045] In step S405, the processing circuit 14 acquires the third resolution data using the acquisition function 141. In step S406, the processing circuit 14 applies the third subset network 31-3 to the third resolution data, the first output data from the first subset network 31-1, and the second output data from the second subset network 31-2. That is, noise reduction processing by the noise reduction block 23 and DC processing by the DC block 24 are performed multiple times on the third resolution data, the first output data, and the second output data. As a result, the third output data is generated by the third subset network 31-3.
[0046] In step S407, the processing circuit 14 calculates a loss value between the output data and the ground truth data based on the loss function, using the learning function 144. Specifically, the loss function can be any loss function commonly used in machine learning, such as the mean squared error, mean absolute error, VGG16 loss (e.g., "Very Deep Convolutional Networks for Large-Scale Image Recognition", ICLR 2015, arxiv:1409.1556), or a discriminator. Alternatively, each subset network 31 may calculate a loss value based on the output data, and the weighted sum of these multiple loss values may be set as the loss function.
[0047] In step S408, the processing circuit 14 determines whether learning is complete or not using the learning function 144. The determination of learning completion may be, for example, when a predetermined number of epochs of learning have been completed, or when the loss value is below a threshold. If learning is complete, parameter updates are stopped and the process ends. This generates a trained model. On the other hand, if learning is not complete, the process proceeds to step S409.
[0048] In step S409, the learning function 144 causes the processing circuit 14 to update network parameters such as weights and biases of the noise reduction block 23 and DC block 24 in the network model 30 to minimize the loss value. After updating the parameters, the process returns to step S401 and the same process is repeated.
[0049] When using the trained model of the network model 30 according to the second embodiment, the process is the same as with the network model 20 according to the first embodiment. The processing circuit 14 inputs the first resolution data to the third resolution data to the network model 30 via the model execution function 145, and outputs high-quality data 28 that corresponds to the third time resolution and has reduced noise and error (improved data consistency) compared to the third resolution data.
[0050] Note that while Figure 4 assumes that a single loss value is calculated for the entire network model 30, this is not the only option; the training of the network model 30 may be divided into subsets. Specifically, for example, the parameters of each subset may be updated to minimize the loss value calculated for each subset, and training may be completed for each subset sequentially, starting with the first subset, and the parameters may be fixed.
[0051] An example of the learning process for each subset network will be explained with reference to the flowchart in Figure 5. Similar to the case in Figure 4, in steps S401 and S402, first resolution data is acquired, and after noise reduction processing and DC processing are performed, first output data is generated.
[0052] In step S501, the processing circuit 14 acquires, for example, ground truth data related to pre-prepared first resolution data using the acquisition function 141, and the processing circuit 14 calculates the loss value between the first output data and the ground truth data using the learning function 144. In step S502, similar to step S409 shown in Figure 4, it is determined whether or not training is complete. If training is complete, the parameters such as weights in the first subset network 31-1 are fixed. If training is not complete, the process proceeds to step S503. In step S503, the processing circuit 14 updates the parameters related to the first subset network 31-1 using the learning function 144 so that the loss value is minimized. After updating the parameters, the process returns to step S401 and the same process is repeated.
[0053] Next, in steps S405 and S406, second resolution data is acquired, and after noise reduction processing and DC processing are performed on the second resolution data and the first output data together, the second output data is generated. In step S504, the processing circuit 14 acquires, for example, ground truth data related to pre-prepared second resolution data using the acquisition function 141, and the processing circuit 14 calculates the loss value between the second output data and the ground truth data using the learning function 144. In step S505, it is determined whether or not training is complete. If training is complete, the parameters such as weights in the second subset network 31-2 are fixed. If training is not complete, the process proceeds to step S506. In step S506, the processing circuit 14 updates the parameters related to the second subset network 31-2 using the learning function 144 so that the loss value is minimized. After updating the parameters, the process returns to step S405 and the same process is repeated.
[0054] In steps S409 and S410, third resolution data is acquired, and after noise reduction processing and DC processing are performed on the third resolution data, the first output data and the second output data together, the third output data is generated. In step S507, the processing circuit 14 acquires, for example, ground truth data related to pre-prepared third resolution data using the acquisition function 141, and the processing circuit 14 calculates the loss value between the third output data and the ground truth data using the learning function 144. In step S508, it is determined whether or not training is complete. If training is complete, the parameters such as weights in the third subnetwork 31-3 are fixed. If training is not complete, the process proceeds to step S509. In step S509, the processing circuit 14 updates the parameters related to the third subset network 31-3 using the learning function 144 so that the loss value is minimized. After updating the parameters, the process returns to step S409 and the same process is repeated.
[0055] Next, we will explain the learning process for each of the 31 subset networks, referring to Figures 6A to 6C. Figure 6A shows an example of training for the first subset network 31-1. The first subsystem 31-1 receives the first resolution data 25 as input, and the first output data 61 is output as a result of processing in the first subsystem 31-1. The parameters of the first subsystem 31-1 are updated to minimize the loss value calculated from the first output data 61 and the ground truth data 62 for the first resolution data 25 using a loss function, and training is completed. After training is complete, the parameters of the first subsystem 31-1 are fixed.
[0056] Figure 6B shows an example of training for the second subset network 31-2. During the training of the second subsystem 31-2, the first subsystem 31-1 receives the first resolution data 25 as input, and the second subsystem 31-2 receives the second resolution data 26 and the first output data 61 from the first subsystem 31-1 for the first resolution data 25 as input. The second subsystem 31-2 outputs the second output data 63 as a result of processing. The subsystem is trained to minimize the loss value calculated from the second output data 63 and the ground truth data 64 for the second resolution data 26 using a loss function. Here, the parameters of the first subsystem 31-1 are fixed because training is already complete, and only the parameters of the second subsystem 31-2 are updated. The second subsystem 31-2 is also trained to complete according to predetermined termination conditions, and its parameters are fixed.
[0057] Figure 6C shows an example of training for the third subset network 31-3. During the training of the third subsystem 31-3, the first-resolution data 25 is input to the first subsystem 31-1, the second-resolution data 26 and the first output data 61 are input to the second subsystem 31-2, and the third-resolution data 27 and the second output data 63 and first output data 61 from the second subsystem 31-2 for the second-resolution data 26 are input to the third subsystem 31-3. As a result of processing by the third subsystem 31-3, the third output data 65 is output. The subsystem is trained to minimize the loss value calculated from the third output data 65 and the ground truth data 66 for the third-resolution data 27 using a loss function. Here, the parameters of the first subsystem 31-1 and the second subsystem 31-2 are fixed because training has already been completed, and only the parameters of the third subsystem 31-3 are updated. The third subsystem 31-3 is also trained to complete its training according to predetermined termination conditions, and its parameters are fixed. As a result, the training of network model 30 is completed.
[0058] In this way, by learning each subset network and sequentially fixing the parameters of each subset network, the computational cost related to the parameters of DC block 24 can be reduced.
[0059] Next, a first modified example of the network model according to the second embodiment will be described with reference to Figure 7. The configuration of the subset network 31 described above is not limited to the order in which the blocks are arranged: noise reduction block 23, DC block 24, noise reduction block 23, DC block 24, ... As shown in the network model 70 relating to the first modified example in Figure 7, the last block of the subset network may be a noise reduction block 71. By making the last output layer in the noise reduction block 71 a 1x1 convolutional layer, the loss can be calculated in an image-like state.
[0060] In other words, when training the network model 70, the loss function should be used to train the network model 70 based on the loss value between the output image, which is the output data from the last DC block of each subsystem, and the ground truth image. When using the network model 70, you can use the network model 70 as is, or you can remove the 1x1 convolutional layers of the first and second subsystems 31-2, except for the noise reduction block 71 of the third subsystem that you want to output, and apply the network model 70 to each resolution data. This is because the 1x1 convolutional layers were used to calculate the loss during learning using images, and after the parameters of each subsystem have been determined, the subsystems other than the subsystem that ultimately generates the output data from the network model do not need to use image data.
[0061] Next, a second modified example of the network model according to the second embodiment will be described with reference to Figure 8. When using k-space data as resolution data, the DC block 81 may be placed in the second and subsequent subsystems before the block that receives the output data from the preceding subsystem.
[0062] In the network model 80 related to the second modified example shown in Figure 8, the second resolution data (k-space data) is input to the DC block 81, and DC-processed data (k-space data) is output. The noise reduction block 23, located after the DC block 81, receives the DC-processed data (k-space data) and the first output data, and processes them in the same manner as the network described above.
[0063] Next, a third modified example of the network model according to the second embodiment will be described with reference to Figure 9. In the third modified network model 90, the network configuration is such that the number of DC blocks, i.e., the number of DC processing executions, increases as the input resolution data increases compared to the input resolution data with low time resolution. As shown in Figure 9, for example, the first subsystem 31-1 can be designed to have two DC blocks 24, the second subsystem 31-2 to have three DC blocks 24, and the third subsystem 31-3 to have four DC blocks 24. This allows for concentrated error reduction on high-resolution data that is the output of the trained model, while reducing the processing load on low-resolution data, thereby lowering the overall computational cost.
[0064] In the above embodiment, three different resolutions were described for convenience, but the system can also be used when two resolution data points are input, or when four or more resolution data points are input. In other words, when using n resolution data points (where n is a natural number greater than or equal to 2) with different resolutions, a network model can be designed that includes a corresponding subsystem for each resolution.
[0065] In general, in the processing of the nth subset network, the nth resolution data and the n-1 output data from the first to the (n-1)th output data are input to the nth subset network, and noise reduction processing and DC processing by the DC block 24 are performed multiple times by the nth subset network, so that the nth output data is generated.
[0066] According to the second embodiment described above, a subset network is designed for each different resolution data, training is divided into multiple stages, and each subset network is trained separately. In addition, the input resolution data is processed so that it progresses from data with low temporal resolution to data with high temporal resolution. As a result, similar to the first embodiment, the accuracy of training the network model can be improved, the performance of the trained model is improved, and the output image quality is also improved. Furthermore, the computational cost of parameters during training can be reduced compared to cases where parameters related to DC blocks must be calculated in relation to all of the multiple input resolution data.
[0067] According to at least one embodiment described above, image quality can be improved.
[0068] In addition, each function according to the embodiment can also be realized by installing a program that performs the processing on a computer such as a workstation and loading it into memory. In this case, the program that can cause the computer to execute the method can also be stored and distributed on a storage medium such as a magnetic disk (hard disk, etc.), optical disk (CD-ROM, DVD, etc.), or semiconductor memory.
[0069] In the above explanation, the term "processor" refers to circuits such as CPUs (Central Processing Units), GPUs (Graphics Processing Units), Application Specific Integrated Circuits (ASICs), and Programmable Logic Devices (e.g., Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and Field Programmable Gate Arrays (FPGAs)).
[0070] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.
[0071] With respect to the above embodiments, the following appendix is disclosed as one aspect of the invention and an optional feature. (Appendix 1) An acquisition unit that acquires multiple input data each having a different time resolution, A processing unit that performs one or more noise reduction processes to reduce noise contained in the data and one or more DC processes to improve the degree of data agreement on each of the multiple input data, and generates output data corresponding to each of the multiple input data, A medical information processing device equipped with the following features. (Note 2) The processing unit may alternately perform the noise reduction process and the DC process multiple times. [Explanation of Symbols]
[0072] 1. Medical Information Processing Device 11 memory 12 Input Interfaces 13 Communication Interface 14 Processing Circuit 20, 30, 70, 80, 90 Network Models 21 Processing Blocks 22 Output Blocks 23,71 Noise Reduction Block 24.81 DC Compensation Block 25. First Resolution Data 26. Second resolution data 27. Third Resolution Data 28 High-quality data 31 Subnetwork 31-1 First Part Network 31-2 Second Part Network 31-3 Third Part Network 61 First Output Data 62, 64, 66 Correct data 63 Second Output Data 65 Third Output Data 141 Acquisition function 142 Noise Reduction Function 143 DC function 144 Learning Functions 145 Model Execution Function 146 Display control function
Claims
1. An acquisition unit that acquires multiple resolution data, each having a different resolution, The system comprises a processing unit that performs one or more first noise reduction processes to reduce noise contained in each of the plurality of resolution data and one or more first DC processes to improve the degree of data agreement, and generates output data corresponding to each of the plurality of resolution data, When the processing unit processes the second and subsequent resolution data, it performs one or more second noise reduction processes and one or more second DC processes on each of the resolution data to be processed and one or more output data generated before the resolution data to be processed, and generates the output data. Medical information processing device.
2. The medical information processing apparatus according to claim 1, wherein the processing unit processes the plurality of resolution data in order from the resolution data with the lowest resolution.
3. The medical information processing apparatus according to claim 1, wherein the processing unit alternately performs the first noise reduction process and the first DC process, or the second noise reduction process and the second DC process, multiple times for a single resolution data.
4. The medical information processing apparatus according to claim 1, wherein the processing unit executes a set of executing the first DC processing after the first noise reduction processing, or a set of executing the second DC processing after the second noise reduction processing, multiple times.
5. The medical information processing apparatus according to claim 1, wherein the processing unit executes a set of executing the first noise reduction processing after the first DC processing, or a set of executing the second noise reduction processing after the second DC processing, multiple times.
6. The medical information processing apparatus according to claim 1, wherein the processing unit performs DC processing on second-resolution data having a second resolution higher than the first resolution more times than it performs DC processing on first-resolution data having a first resolution.
7. The medical information processing device according to any one of claims 1 to 6, wherein the types of the plurality of resolution data are k-space data, MR (Magnetic Resonance) images, projection data, sinogram data, or CT (Computed Tomography) images.
8. By acquiring multiple resolution data sets, each with a different resolution, For each of the plurality of resolution data, one or more first noise reduction processes are performed to reduce the noise contained in the data and one or more first DC processes are performed to improve the degree of data agreement, and output data corresponding to each of the plurality of resolution data is generated. A medical information processing method that, when processing the second and subsequent resolution data, performs one or more second noise reduction processes and one or more second DC processes on the resolution data to be processed and on one or more output data generated before the resolution data to be processed, in order to generate the output data.
9. On the computer, An acquisition function that acquires multiple resolution data, each with a different resolution, A medical information processing program for implementing a processing function that performs one or more first noise reduction processes to reduce noise contained in each of the multiple resolution data and one or more first DC processes to improve the degree of data agreement, and generates output data corresponding to each of the multiple resolution data, The processing function is a medical information processing program that, when processing the second or subsequent resolution data, performs one or more second noise reduction processes and one or more second DC processes on the resolution data to be processed and on one or more output data generated before the resolution data to be processed, and generates the output data.