Medical image learning method and medical image processing device

Abstract

To provide a medical image learning method for generating a trained model that detects a region of interest with high accuracy while reducing the load on a recording storage, in learning using a large amount of collected frame images, and a medical image processing device.SOLUTION: A method of generating a trained second model includes: generating a trained first model through first learning using a first learning image group constituted of a normal image which is a medical image having no region of interest; inputting an input image group including at least a medical image different from the first learning image group to the trained first model to detect anomaly; selecting an extraction image to be used for learning to prevent false recognition according to a result of the anomaly detection; performing second learning using a second learning image group including at least the extraction image; and generating a second model that detects the region of interest through the second learning.SELECTED DRAWING: Figure 15

Description

Technical Field

[0001] The present invention relates to a medical image learning method and a medical image processing apparatus.

Background Art

[0002] Assume a case of performing learning to detect a region of interest including a lesion from a medical image. In order to perform accurate learning, it is necessary to collect not only images including the region of interest but also images not including it with various variations covered. By recording an inspection video, it becomes possible to collect a large number of frame images.

[0003] However, since many images that are not beneficial for learning are included in the video frames during inspection, using the large number of collected frame images as learning data as they are will lead to inefficiency in learning. For example, problems such as a large number of similar images being collected and a large number of images not including the region of interest being collected occur. Since similar images have no difference in the amount of information they have, even if there are a large number of them, their contribution to learning is small. In particular, since many frames during inspection do not include the region of interest, they tend to be collected excessively. Excessively collecting images not including the region of interest not only leads to an undesirable class imbalance problem but also causes compression of the recording storage. Although a method of randomly reducing images after collecting a large amount to avoid storage compression can be considered, there is a possibility of reducing image data beneficial for learning and making it unusable for learning, which may lead to a decrease in accuracy.

[0004] In Patent Document 1, it is described that a diagnostic result image in which an abnormal part is extracted from a medical image is generated using an abnormal part detector that outputs an abnormal part extraction image and a specific part estimator that outputs a specific part estimation image. In Patent Document 2, there are a first learning model learned with identification information corresponding to a polyp and a second learning model learned with identification information corresponding to other than the polyp output by the first learning model, and it is described that the second learning model determines identification information corresponding to other than the polyp.

Prior Art Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-86698 [Patent Document 2] Japanese Patent Publication No. 2021-58464 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Patent Document 1 describes detecting abnormal areas in medical images using a trained abnormality detector and a trained characteristic area estimator, but it does not describe how to distinguish abnormal areas from non-lesion abnormalities such as treatment marks or blisters. Patent Document 2 describes training a second model with information that does not correspond to lesions, output by a first model trained with information corresponding to lesions, and then selecting images according to their features by inputting images into the trained second model. While it describes preventing false detection of normal images that are easily detected as lesions in images near the boundary of the identification process, it does not describe preventing the non-detection of lesion images that are difficult to detect. Furthermore, it does not describe how to prevent bias in the learning content and stabilize the learning process. Based on the above points, it is desirable to accurately train the areas of interest, such as lesions, so that it is possible to select abnormal images that are erroneously not detected or whose detection is prevented from a large number of frame images acquired by video recording.

[0007] The present invention aims to provide a medical image learning method and a medical image processing apparatus that reduce the burden on recording storage and generate a trained model that can detect areas of interest with high accuracy when learning using a large number of collected frame images. [Means for solving the problem]

[0008] The medical image learning method of the present invention involves first learning with a first set of learning images consisting of normal images, which are medical images that do not have a region of interest, to generate a first model. This model is then input to an input image set that includes at least medical images different from the first set of learning images, to perform anomaly detection. Based on the results of the anomaly detection, extracted images to be used for learning to prevent misrecognition are selected. A second model is then generated in a second learning process using a second set of learning images that includes at least the selected extracted images from the input image set, to perform detection of the region of interest.

[0009] In the first learning stage, it is preferable that each medical image in the input first learning image group is generated or restored.

[0010] The selection performed on the input image group consisting of normal images is the first selection, and it is preferable that the first selection selects medical images that have been determined to be abnormal in the abnormality detection results as extracted images.

[0011] The selection process performed on the input image group consisting of abnormal images is the second selection, and it is preferable that the second selection selects medical images that have been determined to be normal in the abnormality detection results as extracted images.

[0012] The first selection is performed on an input image group consisting of normal images, and in the first selection, medical images that are determined to be abnormal in the anomaly detection result are selected as extracted images. The second selection is performed on an input image group consisting of abnormal images, and in the second selection, medical images that are determined to be normal in the anomaly detection result are selected as extracted images. The input image group may consist of normal images or abnormal images, and it is preferable for the first model to switch between the first selection and the second selection depending on the type of input image group.

[0013] The second set of training images preferably includes the extracted images and medical images different from the extracted images.

[0014] For anomaly detection, it is preferable to evaluate the degree of abnormality of each medical image in the input image set, and for the second learning, generate a second model using a second set of learning images in which each medical image is weighted according to its degree of abnormality.

[0015] In weighting, it is preferable to set small weights for extracted images, such as extracted normal images and extracted abnormal images, and large weights for non-extracted images, such as non-extracted normal images and non-extracted abnormal images.

[0016] The first model is an autoencoder, and in the first training phase, the first model performs autoencoding training and calculates reference intermediate features of normal images. Based on each medical image in the input image group input to the first model, an autoencoded image and intermediate features are obtained, and anomaly detection is preferably performed by comparing the image before autoencoding with the autoencoded image and calculating the restoration error, or by comparing the reference intermediate features with the intermediate features, or at least one of these.

[0017] The first model is a GAN, and it is preferable to perform anomaly detection using at least one of a trained judge or a trained generator.

[0018] In the first model, it is preferable to perform image interpolation training in the first training phase, then to intentionally remove missing data from each medical image constituting the input image set input to the first model, interpolate the missing data using the criteria calculated in the first training phase, and perform anomaly detection using the restoration error calculated by comparing the medical image before the data loss with the medical image after interpolation.

[0019] The medical image processing apparatus of the present invention comprises a processor, which inputs an input image group containing at least medical images different from the first learning image group to a first model generated by a first learning process using a first learning image group consisting of normal images which are medical images that do not have a region of interest. The processor performs anomaly detection and selects extracted images to be used for learning to prevent misrecognition according to the result of the anomaly detection. In a second learning process using a second learning image group containing at least the selected extracted images from the input image group, the processor generates a second model that detects regions of interest. [Effect of the Invention]

[0020] In learning using a large number of collected frame images, a learned model can be generated that reduces the burden on the recording storage and detects a target area with high accuracy. [Brief Description of the Drawings]

[0021] [Figure 1] It is an explanatory diagram showing connection devices of a medical image processing apparatus. [Figure 2] It is a block diagram showing functions of a medical image processing apparatus. [Figure 3] It is an explanatory diagram of an image stored in a database used for learning. [Figure 4] It is an explanatory diagram of first learning in which a first group of learning images is input into a first model. [Figure 5] It is an explanatory diagram showing functions of a learned first model. [Figure 6] It is an explanatory diagram of first selection for selecting an extracted image from a normal image. [Figure 7] It is an explanatory diagram of second selection for selecting an extracted image from an abnormal image. [Figure 8] It is an explanatory diagram of the input of first selection and second selection and abnormality detection. [Figure 9] It is an explanatory diagram of a specific example for evaluating the degree of abnormality of an image after selection. [Figure 10] It is an explanatory diagram of second learning in which a second group of learning images is input into a second model. [Figure 11] It is an explanatory diagram for dividing an inspection video frame by frame. <000​​​​​​​​​​​​​[Modes for carrying out the invention]

[0022] Figure 1 shows an example configuration of a medical image processing system 10 that realizes a medical image learning method according to an embodiment of the present invention. The medical image processing system 10 includes a medical image processing device 11, a database 12, an endoscope system 13 having an endoscope 13a, a display 14, and a user interface 15. The medical image processing device 11 is electrically connected to the database 12, the endoscope system 13, the display 14, and the user interface 15.

[0023] The database 12 is a device that stores acquired images and can send and receive data with the medical image processing device 11, and may be a recording medium such as USB (Universal Serial Bus) or HDD (Hard Disk Drive). The medical image processing device 11 acquires images taken during examinations from the endoscope 13a that constitutes the endoscope system 13. The user interface 15 is an input device for inputting settings to the medical image processing device 11, and includes a keyboard and mouse.

[0024] Database 12 stores frame images of examination videos taken during endoscopic examinations and other procedures. Unless otherwise specified during shooting, white light is used as illumination, a video signal of 60 frames per second (60fps) is acquired, and the shooting time is recorded. Furthermore, when the video signal is 60fps, it is preferable to count the time in units of 1 / 100th of a second.

[0025] As shown in Figure 2, in the medical image processing device 11, the functions of the image processing unit 21, display control unit 22, input receiving unit 23, storage memory 24, and learning model management unit 30 are realized by the operation of a program in the program memory by the central control unit 20, which is composed of an image control processor. Furthermore, with the realization of the functions of the learning model management unit 30, the functions of the first learning image group creation unit 31, the input image group creation unit 32, the first model 33, the second learning image group creation unit 34, and the second model 35 are realized. The image processing unit 21 receives data such as images acquired by the endoscopy system 13. The display control unit 22 controls the display of still images or videos on the display 14. The input receiving unit 23 is connected to the user interface 15. The storage memory 24 can realize the storage function on its own, but it can also be connected to the database 12 to send and receive data. In addition, the medical image processing device 11 stores programs related to image processing and other processing in the program memory (not shown). Model 33 (first model) and Model 35 (second model) are computer algorithms consisting of neural networks that learn from input training data, and perform image classification or detection of specific images according to the learned content.

[0026] As shown in Figure 3, the database 12 stores a large number of images as a stored image group 41. The stored image group 41 includes normal images 42 that do not have a region of interest R such as a lesion, and abnormal images 43 that have a region of interest R such as a lesion, as images used for training or input of the first model 33 and the second model 35. Since the first training to train the first model 33 uses normal images 42, it is preferable that normal images 42 and abnormal images 43 can be easily distinguished by means other than visual identification of the images, such as by tagging them for the presence or absence of a region of interest R.

[0027] The learning model management unit 30 creates a pre-trained first model 33 that performs anomaly detection through first learning using normal images 42 extracted from the stored image group 41, and creates a pre-trained second model 35 that detects the region of interest R through second learning using the results of anomaly detection.

[0028] As shown in Figure 4, the first training image group creation unit 31 acquires multiple normal images 42 from the stored image group 41 and creates a first training image group 44 to be used for the first training. The first training image group 44 is then used to train the first model 33, performing the first training to calculate the criteria for anomaly detection. As a result, the first model 33 can select images based on the training content using anomaly detection to obtain extracted images that are highly likely to be useful in preventing misrecognition of the area of ​​interest R during training. The extracted images are, for example, images that represent the discrimination boundary between the detection and non-detection of the area of ​​interest R. The first training image group 44 may include about one anomaly image 43 that can be compared with the normal image 42 regarding the presence or absence of the area of ​​interest R. Anomaly detection using the calculated criteria can have multiple patterns depending on the training method in the first training, such as training an autoencoder, training a GAN (Generative Adversarial Network), or training image interpolation. Details of the training methods will be described later. Furthermore, in the first learning stage, it is preferable that the first model 33 generates or restores the input image.

[0029] As shown in Figure 5, after the first training, it is possible to confirm whether the generation of a trained first model 33 capable of correctly detecting anomalies has been completed. Anomaly detection determines whether an input image is normal or abnormal. Images that are unlikely to contain lesions and do not have a region of interest R similar to the content of the first training are judged as normal. Conversely, images that are likely to contain lesions and have a region of interest R that is not similar to the content of the first training are judged as abnormal. Therefore, a group of confirmation images 45 containing at least one normal image 42 and one abnormal image 43 from the stored image group 41 can be input into the trained first model 33, and the judgment results can be confirmed. However, since anomaly detection is judged based on the first training image group 44, even if it is a normal image 42, treatment marks T or blisters may be identified as abnormal areas, and images that do not contain a region of interest R may be judged as abnormal. Similarly, even if it is an abnormal image 43 with a region of interest R, minute lesions may not be detected and it may be judged as normal. Treatment marks T refer to areas where lesions have been treated in the past.

[0030] The generated, pre-trained first model 33 is used to select images useful for the second training. The second training is the process of creating criteria for detecting the region of interest R for the second model. An input image set consisting of multiple images included in the stored image set 41 is input to the pre-trained first model 33 for anomaly detection, and selection is performed according to the results of the anomaly detection. The selected images are extracted images that are useful for preventing misrecognition in the identification of the region of interest R, and the second training using the extracted images enables the second model to accurately detect the region of interest R. The selection consists of a first selection, in which multiple normal images 42 are input to the pre-trained first model 33 to obtain extracted images, and a second selection, in which multiple abnormal images 43 are input to the pre-trained first model 33 to obtain extracted images. Images that are not selected as extracted images are non-extracted images that are unlikely to be useful for preventing misrecognition.

[0031] In Figure 6, the input image group creation unit 32 creates a first input image group 46 consisting of normal images 42, performs a first selection, and obtains extracted images. Since the normal images 42 do not contain the region of interest R, the majority of the abnormality detection results for the first input image group 46 are determined to be normal, but a small number of images with a large difference from the criteria of the first model 33 are determined to be abnormal. The normal images 42 determined to be normal are non-extracted normal images 47. The normal images 42 determined to be abnormal are extracted images, and are extracted normal images 48 used in the second learning process as images useful for preventing erroneous detection of the region of interest R. If sufficient extracted normal images 48 cannot be obtained through abnormality detection of the first input image group 46, the first selection is performed again using different normal images 42 from the stored image group 41. Note that the first input image group 46 includes at least normal images 42 that were not included in the first learning image group 44.

[0032] In Figure 7, the input image group creation unit 32 creates a second input image group 49 consisting of abnormal images 43, performs a second selection, and obtains extracted images. Since the abnormal images 43 include the region of interest R, the majority of the abnormality detection results for the second input image group 49 are judged as abnormal, but a small number of images with small differences from the criteria of the first model 33 are judged as normal. The abnormal images 43 judged as normal are extracted images and are extracted abnormal images 50 used in the second learning process as images useful for preventing erroneous non-detection of the region of interest R. The abnormal images 43 judged as abnormal are not extracted images and are non-extracted abnormal images 51. If sufficient extracted abnormal images 50 cannot be obtained in the abnormality detection of the first input image group 46, the second selection is performed again using different abnormal images 43 from the stored image group 41.

[0033] As shown in Figure 8, four types of images can be obtained through the first and second selection processes. The first selection process yields a non-extracted normal image 47 and an extracted normal image 48, while the second selection process yields a non-extracted abnormal image 51 and an extracted abnormal image 50. The input image group creation unit 32 creates either a first input image group 46 or a second input image group 49 to be input to the first model, and the trained first model 33 switches between the first and second selection processes depending on the type of input image group. Furthermore, the selection process performs at least one of the first or second selection processes.

[0034] When the trained first model 33 performs selection by determining whether an image is normal or abnormal through anomaly detection, it may evaluate the anomaly score of each image. The anomaly score is evaluated by comparing the value of at least one item related to image information or image features, which clearly changes depending on the presence and size of the region of interest R, with the criteria of the first model. Items used to evaluate the anomaly score include, for example, the average value by density / color, histogram, frequency distribution, edge intensity distribution by edge direction, fractal dimension, brightness distribution, and average brightness. Images with values ​​close to the criteria of the first model are evaluated as having a low anomaly score, while images that are not close are evaluated as having a high anomaly score. A second learning is performed using the anomaly score and learning methods such as weighting.

[0035] The extracted images are saved automatically or manually and retained as training images. Only the extracted images selected through the process are stored in the storage of each facility, such as database 12 or storage memory 24, and the large number of other non-extracted images are deleted, which leads to saving storage capacity. It is preferable to tag the extracted images to determine that they are useful for training to prevent misrecognition.

[0036] Figure 9 shows an example where, in addition to the selection results of the first or second selection, the difference from the criteria of the first model is comprehensively evaluated as an abnormality score on a scale of 0 to 9 and used for the second learning. The case of evaluating the abnormality score for images A-C used in the first selection and images D-F used in the second selection will be explained. The larger the difference, the higher the abnormality score. An abnormality score of 0-2 corresponds to a normal judgment, and 3-9 corresponds to an abnormal judgment. For example, in image A, there is no difference from the criteria of the first model, so the abnormality score is 0. In image B, there is a treatment mark T, and the abnormality score is... 6 Image C shows an area that is in shadow from the illumination, with an abnormality level of 3. Image D shows a small, inconspicuous tumor, with an abnormality level of 2. Image E shows a small area of ​​bleeding, with an abnormality level of 3. 5 Image F shows a large tumor with an abnormality score of 7. A higher abnormality score indicates a potentially more serious lesion, but as mentioned above, extracted normal images 48, including treatment marks, also have a high abnormality score. Therefore, it is preferable to incorporate learning methods such as weighting in addition to abnormality score evaluation when performing the second learning process.

[0037] This section describes how to apply weights to the images that make up the second set of training images. Extracted normal images 48 and extracted abnormal images 50, which are useful for learning to prevent misrecognition, are used infrequently in the decision-making process. Treating them the same as non-extracted normal images 47 or non-extracted abnormal images 51 may lead to a decrease in accuracy, so a difference in priority is given by weighting. For example, extracted normal images 48 and extracted abnormal images 50 are weighted with a value greater than 0, "a" (a>0), non-extracted normal images 47 are weighted with a value greater than a, "b", and non-extracted abnormal images 51 with a region of interest R are weighted with a value equal to or greater than b, "c" (c≧b>a>0).

[0038] As shown in Figure 10, the second learning process, in which the second model 35 is trained using the second set of training images 52, will be described. The second set of training images 52 consists of the extracted normal images 48 and extracted abnormal images 50 obtained in the first and second selection processes, and the non-extracted abnormal images 51. Furthermore, the presence or absence of the region of interest R and the prevention of learning instability are also considered. of Preferably, the large number of acquired images also include randomly selected non-extracted normal images 47. For example, the second training image group 52 is mainly composed of non-extracted abnormal images 51, and is configured to include extracted normal images 48, non-extracted normal images 47, and extracted abnormal images 50 in a certain proportion to the non-extracted abnormal images 51. By training the second model with this configuration, a trained second model 35 that detects the region of interest R is generated.

[0039] In the above explanation, extracted images obtained through selection, along with non-extracted normal images 47 and non-extracted abnormal images 51, are used as the second training image group 52. However, normal images 42 or abnormal images 43, which are likely to be less useful for preventing misrecognition and have been prepared in advance in the stored image group 41, may also be used as non-extracted images for the second training.

[0040] The trained second model 35 detects the region of interest R in a large number of still images framed from the examination video. Here, we will explain using an endoscopic examination video taken with the endoscope 13a as an example. As shown in Figure 11, after the end of the endoscopic examination, the image processing unit 21 receives the examination video 61 recorded by the endoscope 13a from the endoscope system 13, and obtains a group of examination frame images 63 consisting of a large number of examination frame images 62 obtained by dividing the examination video 61 frame by frame. The group of examination frame images 63 is transmitted to the trained second model 35. Although the reception of the examination video 61 is after the end of the endoscopic examination, the video signal may also be framed and processed in real time during the endoscopic examination, before the examination video 61 is created.

[0041] As shown in Figure 12, the inspection frame image group 63 is subjected to detection of the region of interest R based on the input of the trained second model 35, and detected images 64 are acquired. Since the inspection video 61 consists mostly of inspection frame images 62 that do not contain the region of interest R, the number of detected images 64 that contain the detected region of interest R is small compared to the number of input images. The detected images 64 are preferably temporarily stored in the storage memory 24 and displayed on the display 14 via the display control unit 22 for user confirmation. Alternatively, they may be stored in the database 12 instead of being temporarily stored in the storage memory 24. Inspection frame images 62 in which the region of interest R was not detected based on the input are preferably deleted unless otherwise specified.

[0042] As shown in Figure 13, the user can individually diagnose the detected images 64 displayed on the display 14. They can check the area of ​​interest R and the image information displayed in the image information display field 65, and input a title and findings. After diagnosis, they can select the save button 66 to save to the database 12, or delete it by selecting the delete button 67 if it is no longer needed.

[0043] As shown in Figure 14, the detected images 64 may be diagnosed using thumbnail display. In this case, it is preferable to be able to select the detected images 64 whose information is displayed in the image information display area using the cursor or keyboard. Alternatively, the save button 66 or delete button 67 may be used to save multiple detected images 64 at once.

[0044] When real-time framing of video signals and detection of areas of interest are performed during an endoscopic examination, the examination frame image 62 created by framing the video signal is appropriately transmitted from the endoscopy system 13 to the trained second model 35. The trained second model 35 performs detection of areas of interest R each time it receives an examination frame image 62, and displays the detected image 64, which is found to contain the area of ​​interest R, on the display 14. The detected image 64 displayed on the display 14, including thumbnails, can be diagnosed by the user, saved, or deleted, just as if it were acquired after the examination.

[0045] As shown in Figure 15, the flowchart for generating a trained second model 35 that detects the region of interest R will be explained. The first model 33 is trained with a first training image group 44 consisting of normal images 42 that do not have the region of interest R, thereby generating a trained first model 33. The trained first model 33 performs anomaly detection, determining whether the input image is positive or negative based on the content of the first training. Selection is performed to obtain extracted images using the anomaly detection results. There are first and second selections, and at least one of them is performed. In the first selection, the first input image group 46 is used to select non-extracted normal images 47 and extracted normal images 48, and this is repeated until a sufficient number of extracted normal images 48 are obtained. In the second selection, the second input image group 49 is used to select non-extracted abnormal images 51 and extracted abnormal images 50, and this is repeated until a sufficient number of extracted abnormal images 50 are obtained. A second set of training images 52 is created, comprising extracted normal images 48 and extracted abnormal images 50 obtained through selection, and normal images 42 and abnormal images 43, which are not intended to prevent misrecognition in anomaly detection. The second model is trained using the second set of training images to generate a trained second model 35 that detects inspection frame images 62 having a region of interest R.

[0046] When the acquired inspection video 61's set of inspection frame images 63 is input to the generated, pre-trained second model 35, detection images 64 containing the region of interest R are detected from the large number of frame images and temporarily saved. After temporary saving, the detection images 64 are displayed on the display 14, allowing the user to perform a diagnosis. Based on the diagnosis, the image name and findings can be entered and saved.

[0047] The learning method for the first model will be explained. Learning methods include training the autoencoder, training the GAN, and training image interpolation. GANs, also called Generative Adversarial Networks, use a Discriminator and a Generator to generate non-existent images. When used for anomaly detection, there are two patterns: anomaly detection using the Discriminator and anomaly detection using the Generator. Anomaly detection can also be performed by combining the Discriminator and the Generator.

[0048] The first model 33 performs initial training on the first set of training images to generate an autoencoder. The initial training also calculates reference intermediate features in the reference normal image 42. The autoencoder performs autoencoding, encoding each image in the input image set for anomaly detection to calculate intermediate features, and then decoding them based on the content of the initial training to generate an autoencoded image. Anomaly detection is performed using at least one of the following: the reconstruction error value, which is the difference calculated by comparing the image before and after autoencoding, or the comparison result, such as the distributional difference between the intermediate features of the input medical image and the reference intermediate features. If the reconstruction error value or the comparison result such as the distributional difference exceeds a predetermined threshold, it is determined to be an anomaly; if it does not exceed the threshold, it is determined to be normal.

[0049] Model 33 1 performs initial training using the first set of training images to generate a GAN classifier. With the classifier's parameters fixed, the intermediate feature distribution of the classifier is calculated when the data from the first training is input. Anomaly detection is performed by comparing the intermediate features of the input image with the above intermediate feature distribution. If the difference calculated by the comparison exceeds a pre-set threshold, the input image is judged as abnormal; if it does not exceed the threshold, it is judged as normal.

[0050] Model 33 performs initial training on the first set of training images to generate a GAN generator. The generator parameters are fixed, and the input noise to the generator is optimized so that the difference between the input image and the generated image is minimized. Anomaly detection is performed by comparing the optimized generated image with the input image. If the difference calculated from the comparison exceeds a pre-set threshold, it is determined to be an anomaly; if it does not exceed the threshold, it is determined to be normal.

[0051] Model 33 1 performs initial training on the first set of training images and generates an image interpolator. The image interpolator takes the input forFor each medical image in the image set, a random region is deleted from the normal image 42, and a region corresponding to the area of ​​interest R is deleted from the abnormal image 43. The missing portion of the medical image is interpolated based on the criteria calculated in the first learning stage, and an interpolated image is obtained. For each medical image, the image before deletion and the image after interpolation are compared to calculate the restoration error and anomaly detection is performed. If the restoration error exceeds a predetermined threshold, it is determined to be abnormal; if it does not exceed the threshold, it is determined to be normal.

[0052] In this embodiment, the medical image learning method was described using an example where the medical image processing device 11 is connected to the endoscope system 13 and processes examination videos acquired during an endoscopic examination. However, the present invention is not limited to this, and trained models may be generated for videos or frame image groups acquired by other medical devices such as ultrasound imaging devices or radiography devices. Furthermore, trained models may be generated using equipment with functions different from those of the medical image processing device 11.

[0053] In each embodiment, the hardware structure of the processing unit that executes various processes, such as the central control unit 20, image processing unit 21, display control unit 22, input receiving unit 23, storage memory 24, and learning model management unit 30, is a variety of processor as shown below. These various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units; a Programmable Logic Device (PLD), which is a processor whose circuit configuration can be changed after manufacturing, such as an FPGA (Field Programmable Gate Array); and a dedicated electrical circuit, which is a processor with a circuit configuration specifically designed to execute various processes.

[0054] A single processing unit may be composed of one of these various processors, or it may be composed of a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor. Examples of composing multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software are combined to form a single processor, and this processor functions as multiple processing units, as is typical of computers such as client and server systems. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as is typical of System-on-a-Chip (SoC) systems. Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0055] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit (Circuitry) in the form of a combination of circuit elements such as semiconductor elements. The hardware structure of the memory unit is a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). [Explanation of Symbols]

[0056] 10 Medical Image Processing Systems 11 Medical imaging processing equipment 12 Databases 13 Endoscopic Systems 13a Endoscope 14 displays 15 User Interface 20 Central Control Unit 21 Image Processing Unit 22 Display Control Unit 23 Input receiving unit 24 storage memory 30 Learning Model Management Department 31. First training image group creation unit 32 Input Image Group Creation Unit 33 First Model 34. Second Learning Image Set Creation Unit 35 Second Model 41 Archived Image Group 42 Normal images 43 Abnormal Images 44. First set of training images 45. Images for verification 46 Image group for the first input 47 Unextracted normal images 48 Extracted normal images 49 Image group for second input 50 Extracted abnormal images 51 Non-extracted abnormal images 52. Second set of training images 61 Examination Videos 62 Examination Frame Images 63 Examination Frame Image Group 64 detected images 65 Image information display area 66 Save button 67 Delete button R Area of ​​interest T treatment scar

Claims

1. Anomaly detection is performed by inputting an input image set, which includes at least one medical image different from the first training image set, to a first model generated by first training using a first training image set consisting of normal images that do not have a region of interest. Based on the results of the anomaly detection, the extracted images to be used for learning to prevent misrecognition are selected. A second model is generated for detecting the region of interest in a second learning process using a second set of learning images that includes at least the selected extracted images from the input image set. The selection performed on the input image group consisting of the normal images is the first selection, The first selection is a medical image learning method that selects the medical images that have been determined to be abnormal in the results of the abnormality detection into the extracted images.

2. Anomaly detection is performed by inputting an input image set, which includes at least one medical image different from the first training image set, to a first model generated by first training using a first training image set consisting of normal images that do not have a region of interest. Based on the results of the anomaly detection, the extracted images to be used for learning to prevent misrecognition are selected. A second model is generated for detecting the region of interest in a second learning process using a second set of learning images that includes at least the selected extracted images from the input image set. The selection performed on the input image group consisting of abnormal images is a second selection, The second selection is a medical image learning method that selects the medical images that have been determined to be normal in the results of the abnormality detection into the extracted images.

3. The medical image learning method according to claim 1 or 2, wherein in the first learning, each of the medical images in the input first learning image group is generated or restored.

4. Anomaly detection is performed by inputting an input image group that includes at least the medical images different from the first training image group into a first model generated by first training using a first training image group consisting of normal images which are medical images that do not have a region of interest, Based on the results of the anomaly detection, the extracted images to be used for learning to prevent misrecognition are selected. A second model is generated for detecting the region of interest in a second learning process using a second set of learning images that includes at least the selected extracted images from the input image set. The selection performed on the input image group consisting of the normal images is the first selection, The first selection involves selecting the medical images that have been determined to be abnormal in the results of the abnormality detection into the extracted images. The selection performed on the input image group consisting of abnormal images is a second selection, The second selection process involves selecting the medical images that were determined to be normal in the results of the abnormality detection into the extracted images. The aforementioned group of input images may consist of normal images or abnormal images. The first model is a medical image learning method that switches between the first sorting and the second sorting depending on the type of the input image group.

5. The medical image learning method according to any one of claims 1 to 4, wherein the second group of learning images includes the extracted image and a medical image different from the extracted image.

6. Anomaly detection is performed by inputting an input image group that includes at least the medical images different from the first training image group into a first model generated by first training using a first training image group consisting of normal images which are medical images that do not have a region of interest, Based on the results of the anomaly detection, the extracted images to be used for learning to prevent misrecognition are selected. A second model is generated for detecting the region of interest in a second learning process using a second set of learning images that includes at least the selected extracted images from the input image set. The anomaly detection method evaluates the degree of abnormality of each medical image in the input image group, The second learning method is a medical image learning method that generates the second model using the second set of learning images, each of which is weighted according to the degree of abnormality.

7. In the aforementioned weighting, The extracted images, namely the normal extracted image and the abnormal extracted image, are given small weights. The medical image learning method according to claim 6, wherein a large weight is set for non-extracted normal images and non-extracted abnormal images that are not extracted images.

8. The first model is an autoencoder, as described in any one of claims 1 to 7.

9. In the first training, the first model performs autoencoding training and calculates reference intermediate features of the normal image. Based on each of the medical images in the input image group input to the first model, an autoencoded image and intermediate features are obtained. The medical image learning method according to claim 8, which performs anomaly detection by comparing the image before autocoding with the autocoded image and calculating the restoration error, or by comparing the reference intermediate feature with the intermediate feature, or any other of the two.

10. The first model is a GAN, and the method for learning medical images according to any one of claims 1 to 7, wherein the anomaly detection is performed using at least one of a trained judge or a trained generator.

11. Anomaly detection is performed by inputting an input image group that includes at least the medical images different from the first training image group into a first model generated by first training using a first training image group consisting of normal images which are medical images that do not have a region of interest, Based on the results of the anomaly detection, the extracted images to be used for learning to prevent misrecognition are selected. A second model is generated for detecting the region of interest in a second learning process using a second set of learning images that includes at least the selected extracted images from the input image set. The first model performs image interpolation training in the first training, Each of the medical images constituting the input image group input to the first model is missing data, and the missing data is interpolated using the criteria calculated in the first learning stage. A medical image learning method that performs anomaly detection using a restoration error calculated by comparing the medical image before the data loss with the medical image after interpolation.

12. Equipped with a processor, The aforementioned processor, Anomaly detection is performed by inputting an input image set, which includes at least one medical image different from the first training image set, to a first model generated by first training using a first training image set consisting of normal images that do not have a region of interest. Based on the results of the anomaly detection, the extracted images to be used for learning to prevent misrecognition are selected. A second model is generated for detecting the region of interest in a second learning process using a second set of learning images that includes at least the selected extracted images from the input image set. The selection performed on the input image group consisting of the normal images is the first selection, The first selection is a medical image processing device that selects the medical images determined to be abnormal in the results of the abnormality detection into the extracted images.

Images