Medical image processing device, learning device, medical image processing method, learning method, and program

Abstract

To provide a technique for properly processing a medical image captured by an imaging apparatus of a different type in analysis using a learned model.SOLUTION: An abnormal accumulation detection apparatus 1 comprises: an input unit 10 which inputs a bone scintigram of a subject imaged by an imaging apparatus; an inverse filtering processing unit 30 which converts the bone scintigram unique to the apparatus applied with noise removal processing unique to the imaging apparatus into a medical image before noise removal; an abnormal accumulation detection processing unit 31 which infers abnormal accumulation by applying the bone scintigram before noise removal to a learned model 41 for abnormal accumulation detection; and an output unit 13 which outputs data indicating an abnormal portion.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for performing image processing such as detecting an abnormal portion or an anatomical structure based on a medical image.

Background Art

[0002] Conventionally, techniques for processing medical images captured by SPECT, PET, CT, MRI, X-ray, etc. are known. For example, in relation to a technique for detecting abnormal accumulation in a bone scintigram, there are techniques for performing a process of recognizing a skeleton and a process of extracting a high-accumulation site. Recently, techniques for performing a skeleton recognition process and a high-accumulation site extraction process using machine learning have also been studied (Non-Patent Document 1). Thus, techniques for analyzing medical images using artificial intelligence to assist in diagnosis have been widely studied regardless of the modality of the equipment used for imaging.

[0003] When performing image processing of a medical image, processing by filtering may be performed for the purpose of noise removal or the like. For example, in a bone scintigram, several techniques for reducing noise have been proposed in order to improve the SN ratio of a planar scintigram (Non-Patent Document 2). In addition, the planar image of SwiftScan has been created by performing Clarity 2D processing on a conventional whole-body image (Non-Patent Document 3). Here, the Clarity 2D processing includes a process of noise removal by a bilateral filter.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

[0005] [Non-Patent Document 2] Carl A. Wesolowski et al. “Improved lesion detection from spatially adaptive, minimally complex, Pixon reconstruction of planar scintigraphic images,” Computerized Medical Imaging and Graphics, 2005 Jan;29(1):65-81

[0006] [Non-Patent Document 3] Takayuki Shibuya et al., "Performance of SwiftScan planar and single photon emission computed tomography technology using low-energy high-resolution and sensitivity collimator," Research Square, 2021 Jul 1;42(7):732-737. [Overview of the project] [Problems that the invention aims to solve]

[0007] Incidentally, recently, there has been an increasing trend of imaging devices being equipped with filtering functions to remove noise and other unwanted elements. However, the content of the filtering process varies depending on the manufacturer of the imaging device, and the medical images captured by each device have their own unique characteristics. Therefore, there was a concern that differences in filtering processes could affect the accuracy of detecting abnormalities. In particular, when performing image analysis processing using machine learning, the processing content is a black box, making it difficult to modify the processing content according to differences in the input image.

[0008] In view of the above background, the present invention aims to provide a technology for appropriately processing medical images acquired by different types of imaging devices in analysis using a trained model. [Means for solving the problem]

[0009] The medical image processing apparatus of the present invention comprises an input unit that inputs a medical image of a subject captured by an imaging device, an image conversion unit that converts a unique medical image of the imaging device, which has undergone processing specific to the imaging device, into a medical image of a predetermined form, an inference unit that applies the medical image of the predetermined form to a first trained model to infer abnormal locations or anatomical structures, and an output unit that outputs data indicating the abnormal locations or anatomical structures.

[0010] By applying the medical image, which has been converted from the imaging device's unique image to a predetermined configuration, to the first trained model and inferring abnormal areas or anatomical structures, abnormal areas or anatomical structures can be detected without being affected by differences in images from the imaging device. In the case of a bone scintigram, the inference unit may detect abnormal accumulations as abnormal areas.

[0011] In the medical image processing apparatus of the present invention, the image conversion unit may generate a medical image of the predetermined form by applying the medical image of the apparatus to a second trained model which has been trained using the apparatus's own medical image and a medical image of the predetermined form corresponding to the apparatus's own medical image as training data.

[0012] By using a second trained model, which is trained using the device's unique medical images and medical images of a predetermined configuration as training data, it is possible to generate medical images of a predetermined configuration from the device's unique medical images, even if the specifications of the device's unique medical images are not disclosed.

[0013] In the medical image processing apparatus of the present invention, the second trained model is a model that takes a medical image unique to the apparatus as input and outputs the difference between the medical image unique to the apparatus and a medical image of a predetermined form, and the image conversion unit may apply the medical image unique to the apparatus to the second trained model to obtain difference data, and generate a medical image of the predetermined form by adding the difference data to the medical image unique to the apparatus.

[0014] In this way, a trained model that outputs the difference between the device's own medical image and a medical image of a predetermined type can accurately determine the difference, and consequently, appropriately determine the medical image of the predetermined type.

[0015] In the medical image processing apparatus of the present invention, the medical image unique to the apparatus is a medical image that has undergone noise reduction processing, while the medical image of the predetermined embodiment may be a medical image that has not undergone noise reduction processing.

[0016] Since undecompressed medical images are easy to collect, a trained model can be prepared to accurately detect abnormal areas from undecompressed medical images. According to the present invention, since the images are converted to undecompressed medical images, the medical image processing device can appropriately detect abnormal areas.

[0017] The learning device of the present invention is a learning device that generates a model for converting a medical image unique to the device into a medical image of a predetermined form by learning, and comprises: an input unit that inputs the device's unique medical image and the medical image of the predetermined form corresponding to the device's unique medical image as training data; a difference calculation unit that calculates difference data between the device's unique medical image and the medical image of the predetermined form; a learning unit that inputs the device's unique medical image into the model to be learned and performs inference, and learns the model to be learned based on the error between the obtained inference result and the difference data; and a storage unit that stores the model to be learned.

[0018] By creating a model that outputs difference data between a medical image unique to the device and a medical image of a predetermined form, the model can be appropriately trained based on the error between the inference result of the difference data and the ground truth data.

[0019] The present invention relates to a medical image processing method for processing a medical image using a medical image processing device, comprising the steps of: inputting a medical image of a subject captured by an imaging device into the medical image processing device; converting a device-specific medical image that has undergone processing specific to the imaging device into a medical image of a predetermined form; applying the medical image of the predetermined form to a first trained model to infer abnormal locations or anatomical structures; and outputting data indicating the abnormal locations or anatomical structures.

[0020] The learning method of the present invention is a method for generating a model for converting a device's own medical image into a predetermined type of medical image by learning, comprising the steps of: inputting the device's own medical image and the predetermined type of medical image corresponding to the device's own medical image as training data; obtaining difference data between the device's own medical image and the predetermined type of medical image; inputting the device's own medical image into the model to be learned and performing inference, and learning the model to be learned based on the error between the obtained inference result and the difference data; and storing the model to be learned in a storage unit.

[0021] The present invention is a program for processing medical images, which causes a computer to perform the following steps: input a medical image of a subject captured by an imaging device; convert a device-specific medical image, which has undergone processing specific to the imaging device, into a medical image of a predetermined form; apply the medical image of the predetermined form to a first trained model to infer abnormal locations or anatomical structures; and output data indicating the abnormal locations or anatomical structures.

[0022] A program according to another aspect of the present invention is a program for generating a model for converting a medical image of a device-specific format into a medical image of a predetermined format by learning. The program causes a computer to perform steps of: inputting, as teacher data, the medical image of the device-specific format and the medical image of the predetermined format corresponding to the medical image of the device-specific format; obtaining difference data between the medical image of the device-specific format and the medical image of the predetermined format; inputting the medical image of the device-specific format into a model to be learned and performing inference, and performing learning of the model to be learned based on an error between the obtained inference result and the difference data; and storing the model to be learned in a storage unit.

Advantages of the Invention

[0023] According to the present invention, regardless of the format of the image of the imaging device, abnormal portions can be appropriately detected from medical images.

Brief Description of the Drawings

[0024] [Figure 1] It is a diagram showing the configuration of an abnormal accumulation detection device according to an embodiment. [Figure 2] (a) It is a schematic diagram showing the configuration of a learned model for inverse filtering processing. (b) It is a schematic diagram showing another example of a learned model for inverse filtering processing. [Figure 3] It is a diagram showing the configuration of a model learning device for inverse filtering. <{\lt}

Modes for Carrying Out the Invention

[0025] Hereinafter, a medical image processing device according to the present embodiment will be described with reference to the drawings. Note that the following description shows only an example of a preferred aspect and is not intended to limit the invention described in the claims.

[0026] Figure 1 shows the configuration of an abnormal accumulation detection device 1 according to an embodiment. The abnormal accumulation detection device 1 is a device that detects abnormal accumulation from a bone scintigram of a subject. In this embodiment, a bone scintigram acquired with a SPECT imaging device is given as an example of a medical image, but the medical image processing device of the present invention can also be applied to medical images acquired with PET, CT, MRI, etc.

[0027] The anomaly accumulation detection device 1 includes an input unit 10 that receives the bone scintigram of the subject to be examined, a calculation unit 11 that performs image processing of the subject's bone scintigram to detect anomalies, a storage unit 12 that stores two trained models 40 and 41, and an output unit 13 that outputs the anomaly accumulation detection result.

[0028] The subject's scintigram input from the input unit 10 undergoes processing unique to the SPECT imaging device. One example of this unique processing is filtering, which is aimed at removing noise. The method of this filtering varies depending on the manufacturer of the SPECT imaging device. In this embodiment, a filtered bone scintigram is used as an example to illustrate the device's unique medical image.

[0029] The memory unit 12 stores a trained model 40 for inverse filtering and a trained model 41 for anomaly accumulation detection. Figure 2(a) is a schematic diagram showing the configuration of the trained model 40 for inverse filtering. The trained model 40 for inverse filtering is an autoencoder that takes the image after filtering as input and outputs the difference data of the image before and after filtering. The autoencoder has an encoder that extracts compressed feature representations and a decoder that restores the original data from the feature representations.

[0030] The pre-trained model 41 for anomaly accumulation detection takes an image before filtering is applied as input and outputs whether or not anomalies are present for each pixel. Here, the pre-trained model 41 for anomaly accumulation detection is a model that has been trained using bone scintigrams that have not undergone filtering such as noise reduction as training data. Images that have not undergone filtering such as noise reduction can be obtained regardless of the type of imaging device, making them suitable as training data.

[0031] The calculation unit 11 includes a reverse filtering processing unit 30 that performs reverse filtering on the input bone scintigram, and an abnormal accumulation detection processing unit 31 that detects abnormal accumulations based on the image before filtering obtained by the reverse filtering process.

[0032] The inverse filtering processing unit 30 reads a trained model 40 for inverse filtering from the memory unit 12, inputs the subject's bone scintigram to the read trained model 40, and infers the difference data between the filtered image (input image) and the image before filtering. The inverse filtering processing unit 30 generates the image before filtering by adding the difference data to the filtered image. The image before filtering is the image before noise reduction processing is performed, and is the original image without any filtering processing unique to the imaging device.

[0033] The anomaly accumulation detection processing unit 31 reads a trained model 40 for anomaly accumulation detection from the storage unit 12, performs inverse filtering on the read trained model 40, and inputs the resulting bone scintigram to detect anomalies by inference.

[0034] The output unit 13 outputs the data of the abnormal accumulation detection result detected by the abnormal accumulation detection processing unit 31. For example, the output unit 13 displays an image on the monitor that identifies the location of the abnormal accumulation.

[0035] An example of the hardware of the anomaly detection device 1 in this embodiment is a computer equipped with a CPU, RAM, ROM, hard disk, display, keyboard, mouse, communication interface, etc. The anomaly detection device 1 described above is realized by storing a program having modules that realize each of the above-mentioned functions in RAM or ROM and executing the program with the CPU. Such a program is also included in the scope of the present invention.

[0036] Figure 3 shows the configuration of the learning device 2 that generates a pre-trained model 40 for inverse filtering. The learning device 2 includes an input unit 20 that receives training data, a calculation unit 21 that generates a pre-trained model 40 for inverse filtering, a storage unit 22 that stores the generated pre-trained model 40, and an output unit 23 that outputs the pre-trained model 40 stored in the storage unit 22 to the outside.

[0037] The input unit 20 accepts filtered bone scintigram data and unfiltered bone scintigram data as training data. The difference data generation unit 32 generates difference data between the bone scintigrams before and after filtering. The learning unit 33 inputs the filtered bone scintigram data into the model to be trained and performs inference. The learning unit trains the model to be trained to reduce the error between the difference data obtained by inference (inference result) and the difference data obtained from the training data (correct answer data). When the error falls below a predetermined threshold, the learning is stopped and the model to be trained is stored in the storage unit 22 as a trained model 40.

[0038] Here, we will explain that in this embodiment, a trained model 40 that outputs differential data is used. Since the differential data is smaller than the data of the bone scintigram itself, the error between the inference result and the ground truth data has a large impact on the inference result of the differential data. In addition, the differential data does not depend on the absolute value of the signal, even for signal values ​​near the edge or signal values ​​with small absolute values, and the size of the prediction target is consistent. In this embodiment, by training to reduce the error in the differential data, a trained model 40 that can perform inference with high accuracy can be generated.

[0039] The anomaly accumulation detection device 1 of this embodiment can appropriately detect anomalies by performing inverse filtering on the input subject's bone scintigram using a trained model 40 for inverse filtering, thereby absorbing differences in images due to differences in SPECT imaging devices.

[0040] Attempting to detect anomaly clusters using SPECT images with different specifications requires a vast amount of training data to generate a pre-trained model 40 for anomaly cluster detection. Furthermore, preparing a separate pre-trained model for anomaly cluster detection for each SPECT image with different specifications is undesirable from the standpoint of efficiency in terms of processing speed and memory. According to this embodiment, by changing to an image that has not undergone filtering, anomaly clusters can be appropriately detected using an anomaly cluster detection model generated using an unfiltered image.

[0041] Although the medical image processing apparatus of the present invention has been described in detail with reference to embodiments, the present invention is not limited to the embodiments described above. In the embodiment described above, the trained model 40 for inverse filtering was described as a model that takes the filtered image as input and outputs the difference data of the image before and after filtering. However, the medical image processing apparatus of the present invention may use a trained model 40 that takes the filtered image as input and outputs the image before filtering, as shown in Figure 2(b). With this configuration, when the filtered image is input to the trained model 40, the image before filtering is output, thus reducing the step of generating the image before filtering.

[0042] In the embodiments described above, an example was given of generating a bone scintigram image before filtering processing such as noise reduction. However, the medical image processing device of the present invention may change the image to a specific manufacturer's image format instead of generating an image before noise reduction.

[0043] In the embodiments described above, an example was given in which anomaly accumulations are detected based on an image generated by inverse filtering. However, the medical image processing device of the present invention can also be applied to the detection of anatomical structures. In this case, a trained model for detecting anatomical structures, which is learned using an image before noise reduction processing, is generated and prepared in advance. As a method for detecting anatomical structures using the trained model, the method described in Non-Patent Literature 1 can be used. The medical image processing device performs inverse filtering on the input bone scintigram of a subject to generate an image before noise reduction processing, and detects anatomical structures based on this image. [Explanation of symbols]

[0044] 1. Anomaly accumulation detection device 2 Learning device 10 Input section 11 Arithmetic section 12 Storage section 13 Output section 20 Input section 21 Arithmetic section 22 Memory section 23 Output section 30 Inverse filtering processing unit 31 Anomaly Accumulation Detection Processing Unit 32 Difference Data Generation Unit 33 Learning Department 40 Pre-trained models for inverse filtering 41. Pre-trained models for detecting anomaly accumulation

Claims

1. An input unit that inputs medical images of a subject captured by an imaging device, which are unique medical images of the device that have undergone filtering processing specific to the imaging device, An image conversion unit converts a medical image unique to the imaging device, which has undergone filtering processing specific to the imaging device, into a medical image that has not undergone filtering processing specific to the imaging device. An inference unit that applies medical images that have not undergone filtering processing specific to the imaging device to a first trained model to infer abnormal locations or anatomical structures, An output unit that outputs data indicating the abnormal location or anatomical structure, A medical image processing device equipped with [a specific feature].

2. The image conversion unit, A medical image processing apparatus according to claim 1, which generates a medical image that has not undergone filtering processing specific to the imaging apparatus by applying the medical image that is unique to the apparatus to a second trained model that has been trained using the medical image that is unique to the apparatus and a medical image that corresponds to the medical image that is unique to the apparatus and has not undergone filtering processing specific to the imaging apparatus as training data.

3. The second trained model is a model that takes a medical image unique to the device as input and outputs the difference between the medical image unique to the device and a medical image that has not undergone the filtering process specific to the imaging device. The medical image processing apparatus according to claim 2, wherein the image conversion unit applies the apparatus's own medical image to the second trained model to obtain difference data, and adds the difference data to the apparatus's own medical image to generate a medical image that has not undergone filtering processing specific to the imaging apparatus.

4. The medical image processing apparatus according to any one of claims 1 to 3, wherein the medical images unique to the apparatus are medical images that have undergone noise reduction processing, and the medical images that have not undergone filtering processing specific to the imaging apparatus are medical images that have not undergone noise reduction processing.

5. The medical image processing apparatus according to any one of claims 1 to 4, wherein the medical image is a bone scintigram, and the inference unit detects an abnormal accumulation as the abnormal area.

6. A learning device that generates a model by learning for converting a medical image unique to the imaging device, which has undergone filtering processing specific to the imaging device, into a medical image which has not undergone filtering processing specific to the imaging device, An input unit that inputs a medical image unique to the said device and a medical image corresponding to the medical image unique to the said device that has not undergone any filtering process specific to the imaging device as training data, A difference calculation unit that obtains difference data between a medical image unique to the aforementioned device and a medical image that has not undergone the filtering process specific to the imaging device, A learning unit inputs a medical image unique to the device into the model to be trained and performs inference, and trains the model based on the error between the obtained inference result and the difference data. A memory unit for storing the model to be learned, A learning device equipped with the following features.

7. A method for processing medical images using a medical image processing device, The medical image processing device receives a medical image of a subject captured by an imaging device, which is a unique medical image of the device that has undergone filtering processing specific to the imaging device. The medical image processing device performs a step of converting a medical image unique to the imaging device, which has undergone filtering processing specific to the imaging device, into a medical image that has not undergone filtering processing specific to the imaging device. The medical image processing device performs the step of applying a medical image that has not undergone filtering processing specific to the imaging device to a first trained model to infer abnormal locations or anatomical structures. The medical image processing device outputs data indicating the abnormal location or anatomical structure, A medical image processing method comprising the following.

8. A method for generating a model by learning for converting a medical image unique to an imaging device, which has undergone filtering processing specific to the imaging device, into a medical image which has not undergone filtering processing specific to the imaging device, The steps include inputting a medical image unique to the said device and a medical image corresponding to the medical image unique to the said device that has not undergone any filtering process specific to the imaging device as training data, The steps include obtaining difference data between a medical image unique to the aforementioned device and a medical image that has not undergone the filtering process specific to the imaging device, The steps include: inputting a medical image unique to the device into the model to be trained and performing inference, and training the model based on the error between the obtained inference result and the difference data; The steps include storing the model to be learned in the memory unit, A learning method that includes [the following features].

9. A program for processing medical images, which is used on a computer. The steps include inputting a medical image of a subject captured by an imaging device, which is a unique medical image of the device that has undergone filtering processing specific to the imaging device, The steps include converting a medical image unique to the imaging device, which has undergone filtering processing specific to the imaging device, into a medical image that has not undergone filtering processing specific to the imaging device, The steps include applying medical images that have not undergone filtering specific to the imaging device to a first trained model to infer abnormal locations or anatomical structures, The steps include outputting data indicating the abnormal location or anatomical structure, A program that executes the command.

10. A program that generates a model by learning for converting a medical image unique to the imaging device, which has undergone filtering processing specific to the imaging device, to a medical image which has not undergone filtering processing specific to the imaging device, wherein the computer The steps include inputting a medical image unique to the said device and a medical image corresponding to the medical image unique to the said device that has not undergone any filtering process specific to the imaging device as training data, The steps include obtaining difference data between a medical image unique to the aforementioned device and a medical image that has not undergone the filtering process specific to the imaging device, The steps include: inputting a medical image unique to the device into the model to be trained and performing inference, and training the model based on the error between the obtained inference result and the difference data; The steps include storing the model to be learned in the memory unit, A program that executes the command.

Images