Learning device, method and program

Abstract

Provided are a learning device, a method, and a program, wherein a trained neural network can be constructed, the neural network being capable of classifying representation media into a plurality of types of classes, even if a large number of pieces of teaching data for learning the plurality of types of classes all at once cannot be prepared. A processor: acquires teaching data comprising a learning representation medium and a correct answer label for at least one class among a plurality of types of classes included in the learning representation medium; inputs the learning representation medium into a neural network to cause same to output a probability of being each of the plurality of types of classes for the classes included in the learning representation medium; integrates the respective probabilities of the plurality of types of classes, on the basis of the classes classified by correct answer labels in the teaching data; and trains the neural network on the basis of a loss calculated from the integrated probabilities and the correct answer labels in the teaching data.

Description

[Technical Field]

[0001] The present disclosure relates to a learning device, a learning method, and a learning program. [Background technology]

[0002] In recent years, machine learning techniques using deep learning have been attracting attention. In particular, various methods have been proposed for classifying images into desired regions by using a trained neural network constructed by deep learning, which is a type of multilayer neural network in which multiple processing layers are hierarchically connected, and by using the trained neural network (see, for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2019-067299 [Patent Document 2] Special Publication No. 2019-505063 Summary of the Invention [Problem to be solved by the invention]

[0004] On the other hand, when classifying an image into multiple types of regions, a trained neural network can be prepared for each type of region, but it is also possible to classify images into multiple types of regions using a single trained neural network. For example, when classifying an image of the chest and abdomen of a human body into a liver region and a lung region at the same time, a neural network that classifies the liver region and a neural network that classifies the lung region can be combined to construct a trained neural network that classifies the liver region and the lung region at the same time. In order to construct such a trained neural network, it is necessary to prepare correct labels that identify the liver region and the lung region in the training image.

[0005] In addition, in some cases, it may be necessary to build a trained neural network that classifies images containing lungs into five lobes: right upper lobe, right middle lobe, right lower lobe, left upper lobe, and left lower lobe. In this case, to train the neural network, it is necessary to prepare correct labels that identify each of the five lobes in the training images.

[0006] Here, since trained neural networks that classify only liver regions and trained neural networks that classify only lung regions are known, it is possible to prepare a large number of correct labels that identify only liver regions and correct labels that identify only lung regions. However, correct labels that identify both liver regions and lung regions impose a heavy burden on the creator who creates the training data. For this reason, it is currently difficult to prepare enough training data for simultaneously learning the classification of liver and lungs to accurately train a neural network. Furthermore, while it is possible to prepare a large number of correct labels that identify lung regions, correct labels that identify each of the five lung lobes also impose a heavy burden on the creator who creates the training data. For this reason, it is currently difficult to prepare enough training data for simultaneously learning the classification of the five lung lobes to accurately train a neural network. This problem occurs not only when building a trained neural network that classifies not only medical images but also representation media such as photographic images, video images, audio, and text into multiple classes.

[0007] The present disclosure has been made in consideration of the above circumstances, and aims to make it possible to construct a trained neural network that can classify expression media into multiple types of classes, even if it is not possible to prepare a large amount of training data for learning to classify multiple types of classes at once. [Means for solving the problem]

[0008] A learning device according to the present disclosure is a learning device that performs machine learning on a neural network that classifies expression media into three or more classes, at least one processor; The processor Acquire training data consisting of a learning representation medium and a correct label for at least one class among a plurality of types of classes included in the learning representation medium; Inputting the training representation medium into a neural network and outputting the probability of each of a plurality of classes for the classes included in the training representation medium; The probability of each of the multiple classes is integrated based on the class classified by the correct answer label of the training data, The neural network is trained based on the loss derived from the integrated probability and the ground truth labels of the training data.

[0009] An "expression medium" is a medium that can be expressed using a computer, and examples of expression media include still images, moving images, audio, and text.

[0010] In the learning device according to the present disclosure, the expression medium is an image, The multiple classes are multiple regions in the image, including the background, The processor may integrate the probabilities of each of the multiple types of classes by adding the probability of classes other than the class classified by the correct label for the learning expression medium and the probability of the background.

[0011] In addition, in the learning device according to the present disclosure, the classes classified by the correct labels include two or more classes among the multiple types of classes, The processor may integrate the probabilities of each of the multiple classes by adding up the probabilities of two or more classes classified by the correct label among the probabilities of the multiple classes.

[0012] In addition, in the learning device according to the present disclosure, the processor may learn the neural network using a plurality of training data sets with different correct answer labels.

[0013] The learning method according to the present disclosure is a learning method for machine learning a neural network that classifies expression media into three or more classes, Acquire training data consisting of a learning representation medium and a correct label for at least one class among a plurality of types of classes included in the learning representation medium; Inputting the training representation medium into a neural network and outputting the probability of each of a plurality of classes for the classes included in the training representation medium; The probability of each of the multiple classes is integrated based on the class classified by the correct answer label of the training data, The neural network is trained based on the loss derived from the integrated probability and the ground truth labels of the training data.

[0014] Another learning device according to the present disclosure is a learning device that performs machine learning on a neural network that classifies regions in an image into three or more classes, at least one processor; The processor acquires training data including a training image and a correct label for at least one region among a plurality of types of regions included in the training image; Inputting training images into a neural network and outputting probabilities of regions in the training images belonging to each of multiple classes; The probability of each of the multiple classes is integrated based on the class classified by the correct answer label of the training data, The neural network is trained based on the loss derived from the integrated probability and the ground truth labels of the training data.

[0015] Another learning method according to the present disclosure is a learning method for machine learning a neural network that classifies regions in an image into three or more classes, the method comprising: acquiring training data consisting of a training image and a correct label for at least one region among a plurality of types of regions included in the training image; Inputting training images into a neural network and outputting probabilities of regions in the training images belonging to each of multiple classes; The probability of each of the multiple classes is integrated based on the class classified by the correct answer label of the training data, The neural network is trained based on the loss derived from the integrated probability and the ground truth labels of the training data.

[0016] The learning method according to the present disclosure and other learning methods may be provided as a program for causing a computer to execute the method. [Effects of the Invention]

[0017] According to the present disclosure, even if it is not possible to prepare a large amount of training data for learning to classify multiple types of classes at once, it is possible to construct a trained neural network that can classify expression media into multiple types of classes. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a medical information system to which a learning device according to a first embodiment of the present disclosure is applied. [Figure 2] FIG. 1 is a diagram showing a schematic configuration of a learning device according to a first embodiment; [Figure 3] Functional configuration diagram of a learning device according to a first embodiment [Figure 4] A diagram showing training data for learning liver region classification. [Figure 5] A diagram showing training data for learning lung region classification. [Figure 6] FIG. 1 is a diagram illustrating a neural network learning process according to a first embodiment; [Figure 7] FIG. 1 is a diagram illustrating a neural network learning process according to a first embodiment; [Figure 8] FIG. 1 is a diagram illustrating a neural network learning process according to a first embodiment; [Figure 9] 1 is a flowchart showing a learning process performed in the first embodiment. [Figure 10] FIG. 10 is a diagram showing training data used for learning in the second embodiment. [Figure 11] FIG. 10 is a diagram illustrating the learning of a neural network according to the second embodiment; [Figure 12] FIG. 10 is a diagram illustrating the learning of a neural network according to the second embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. First, the configuration of a medical information system to which a learning device according to a first embodiment is applied will be described. FIG. 1 is a diagram showing a schematic configuration of a medical information system. In the medical information system shown in FIG. 1, a computer 1 incorporating a learning device according to this embodiment, an imaging device 2, and an image storage server 3 are connected in a communicable state via a network 4.

[0020] The computer 1 includes a learning device according to this embodiment and has the learning program according to the first embodiment installed. The computer 1 may be a workstation or personal computer operated directly by a doctor who makes a diagnosis, or it may be a server computer connected to either of these via a network. The learning program is stored in a storage device of the server computer connected to the network or in network storage in an externally accessible state, and is downloaded and installed on the computer 1 used by the doctor upon request. Alternatively, the learning program may be recorded on a recording medium such as a DVD (Digital Versatile Disc) or CD-ROM (Compact Disc Read Only Memory) and distributed, and then installed on the computer 1 from the recording medium.

[0021] The imaging device 2 is a device that captures an image of a diagnostic target region of a subject to generate a three-dimensional image representing the region, and specifically includes a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, etc. The three-dimensional image composed of a plurality of slice images generated by the imaging device 2 is transmitted to and stored in the image storage server 3. In this embodiment, the imaging device 2 is a CT device, and generates a CT image of, for example, the patient's chest and abdomen.

[0022] The image storage server 3 is a computer that stores and manages various data and is equipped with a large-capacity external storage device and database management software. The image storage server 3 communicates with other devices via a wired or wireless network 4, sending and receiving image data and the like. Specifically, it acquires various data, including image data of 3D images generated by the imaging device 2, via the network and stores and manages the data on a recording medium such as a large-capacity external storage device. The storage format of the image data and communication between devices via the network 4 are based on protocols such as DICOM (Digital Imaging and Communication in Medicine). The image storage server 3 also stores training data, which will be described later.

[0023] Next, a learning device according to the first embodiment will be described. Fig. 2 describes the hardware configuration of the learning device according to the first embodiment. As shown in Fig. 2, learning device 20 includes a CPU (Central Processing Unit) 11, non-volatile storage 13, and memory 16 as a temporary storage area. Learning device 20 also includes a display 14 such as an LCD display, input devices 15 such as a keyboard and mouse, and a network I / F (Interface) 17 connected to network 4. CPU 11, storage 13, display 14, input devices 15, memory 16, and network I / F 17 are connected to bus 18. CPU 11 is an example of a processor in the present disclosure.

[0024] The storage 13 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage 13 as a storage medium stores a learning program 12. The CPU 11 reads the learning program 12 from the storage 13, expands it into the memory 16, and executes the expanded learning program 12.

[0025] Next, the functional configuration of the learning device according to the first embodiment will be described. Fig. 3 is a diagram showing the functional configuration of the learning device according to the first embodiment. As shown in Fig. 3, the learning device 20 includes an information acquisition unit 21 and a learning unit 22. When the CPU 11 executes the learning program 12, the CPU 11 functions as the information acquisition unit 21 and the learning unit 22.

[0026] Here, the learning device 20 according to the first embodiment constructs a trained network for classifying lung regions and liver regions contained in a CT image. To this end, the learning unit 22 trains the neural network using training data. Note that the CT image is an example of a representation medium, and the lung regions, liver regions, and background are examples of multiple types of classes in the present disclosure.

[0027] The information acquisition unit 21 acquires teacher data from the image storage server 3 in response to an instruction from an operator via the input device 15. When multiple teacher data have been acquired from the image storage server 3 and stored in the storage 13, the information acquisition unit 21 acquires the teacher data from the storage 13.

[0028] FIG. 4 is a diagram showing training data for learning the classification of liver regions. As shown in FIG. 4, training data 30 includes training images 30A and correct labels 30B. Training images 30A are one slice image among multiple slice images constituting a CT image. Training images 30A include regions such as the liver and lungs. In correct labels 30B, labels 30C are assigned to the liver regions included in training images 30A. Note that in FIG. 4, the assignment of labels is indicated by diagonal lines. Training images are an example of training representation media.

[0029] FIG. 5 is a diagram showing training data for learning the classification of lung regions. As shown in FIG. 5, training data 31 includes training images 31A and correct labels 31B. Training images 31A are the same cross-sectional images as training images 30A and include regions such as the liver and lungs. In correct labels 31B, labels are assigned to the lung regions included in training images 31A. Specifically, labels 31C and 31D are assigned to the right lung region and the left lung region, respectively.

[0030] The learning unit 22 trains the neural network using training data. FIG. 6 is a diagram schematically illustrating neural network training in the first embodiment. As shown in FIG. 6, the neural network 40 to be trained is, for example, a convolutional neural network, and is composed of an input layer 41, multiple intermediate layers 42, and an output layer 43. The intermediate layer 42 has convolutional layers and pooling layers (neither of which is shown) arranged alternately. A training image is input to the neural network 40, and for each pixel included in the training image, Logits, a value representing the likelihood that the pixel is the background, liver, right lung, or left lung, is output from the neural network 40. Logits is the output of the neural network 40 itself, and the larger the value, the more likely the pixel is the target region. For example, Logits values ​​such as (1.0, 5.0, 2.0, 1.5) are output for the background, liver, right lung, and left lung.

[0031] The learning unit 22 applies a softmax activation function (Soft Max) to the Logits output from the neural network 40 to convert the Logits into probabilities p0 to p3. For example, the probability values ​​(p0, p1, p2, p3) = (0.1, 0.8, 0, 0.1) are obtained for the background, liver, right lung, and left lung. Since (p0, p1, p2, p3) are probabilities, p0 + p1 + p2 + p3 = 1.

[0032] Here, the correct label 30B of the training data 30 only includes label 30C assigned to the liver region. Furthermore, the correct label 31B of the training data 31 only includes labels 31C and 31D assigned to the left and right lungs, respectively. Therefore, the learning unit 22 integrates the derived probabilities based on the classes classified by the correct labels of the training data. For example, when the training image 30A shown in FIG. 4 is input to the neural network 40, the training image 30A is included in the training data 30 used to learn the classification of the liver region. Therefore, the learning unit 22 integrates the probabilities of the background other than the liver, the right lung, and the left lung from the derived probabilities to derive the integrated probability pt0. In this case, pt0 = p0 + p2 + p3. Therefore, when the probabilities (p0, p1, p2, p3) = (0.1, 0.8, 0, 0.1), the integrated probability (pt0, p1) = (0.2, 0.8).

[0033] The learning unit 22 uses the integrated probability distribution and the correct label to derive the cross entropy error (Cross Entropy) as the loss L0. The cross entropy error corresponds to the distance between the probability distribution and the vector represented by the correct label. Here, if the liver region is labeled in the correct label of the input learning image, the correct label vector for deriving the loss L0 with the integrated probability (pt0, p1) is (0, 1).

[0034] On the other hand, as shown in FIG. 7, when the training image 31A shown in FIG. 4 is input to the neural network 40, the training image 31A is included in the teacher data 31 used to learn the classification of right and left lung regions. Therefore, the learning unit 22 integrates the probabilities of elements other than the lungs, i.e., the background and the liver, among the derived probabilities to derive the integrated probability pt1. In this case, pt1 = p0 + p1. Therefore, when the probabilities (p0, p1, p2, p3) = (0.1, 0.8, 0, 0.1), the integrated probability (pt1, p2, p3) = (0.9, 0, 0.1).

[0035] Also, if the correct labels of the input training images are assigned to the left and right lung regions, the correct label vector for deriving the loss L0 with the integrated probability distribution is (0,1,0) for the right lung and (0,0,1) for the left lung.

[0036] The learning unit 22 performs learning of the neural network 40 until the loss L0 satisfies a termination condition. Specifically, the learning unit 22 performs machine learning of the neural network 40 by deriving parameters such as the number of convolutional layers constituting the intermediate layer 42 included in the neural network 40, the number of pooling layers, the kernel coefficients, and the kernel size. The termination condition may be that the loss L0 is equal to or less than a predetermined threshold, or may be that learning has been performed a predetermined number of times.

[0037] Note that when the liver, right lung, and left lung are labeled in the correct labels of the learning images input to the neural network 40, the learning unit 22 derives the loss L0 with respect to the correct labels and trains the neural network 40 without integrating the probabilities (p0, p1, p2, p3) as shown in Fig. 8. In this case, the correct label vectors are (1,0,0,0) for the background, (0,1,0,0) for the liver, (0,0,1,0) for the right lung, and (0,0,0,1) for the left lung.

[0038] When a CT image is input, the trained neural network constructed by such machine learning outputs the probability that each pixel in the CT image is the liver region, the right lung region, the left lung region, or the background. Therefore, by using the trained neural network constructed by the learning device according to the first embodiment, it is possible to classify the CT image into the region with the highest probability for each pixel.

[0039] Next, the processing performed in the first embodiment will be described. Fig. 9 is a flowchart showing the processing performed in the first embodiment. It is assumed that a plurality of training data sets are acquired from the image storage server 3 and stored in the storage 13. It is also assumed that the condition for ending learning is that the loss L0 becomes equal to or less than a threshold value.

[0040] First, the information acquisition unit 21 acquires training data stored in the storage 13 (step ST1). Then, the learning unit 22 inputs training images included in the training data into the neural network 40 (step ST2), and outputs the probability that a region in the training image belongs to each of multiple classes (step ST3). Furthermore, the learning unit 22 integrates the probabilities of each of the multiple classes based on the classes classified by the correct labels of the training data (step ST4). Then, the learning unit 22 determines whether the loss L0 is equal to or less than a threshold (step ST5). If the result of step ST5 is negative, the neural network is trained based on the integrated probability and the loss L0 derived from the correct labels of the training data (step ST6).

[0041] Furthermore, the information acquiring unit 21 acquires new teacher data (step ST7), and the process returns to step ST2, and the processes of steps ST2 to ST5 are repeated. If step ST5 is judged to be positive, the process ends.

[0042] In this way, in the first embodiment, the probabilities of regions in training images output from the neural network being in each of multiple classes are integrated based on the classes classified by the correct labels in the training data, and the neural network is trained based on the integrated probabilities and the losses derived from the correct labels in the training data. Therefore, even if the correct labels of the training images are not classified into each of the multiple classes, it is possible to build a trained neural network that classifies regions in an image into multiple classes using such training data.

[0043] For example, using training data containing correct labels assigned only to liver regions or correct labels assigned only to lung regions, it is possible to build a trained neural network that classifies regions in an image into three or more classes, such as liver, lung, and background. This eliminates the need to create a large number of correct labels that include all the labels of multiple classes, thereby reducing the burden on the creator of training data. Furthermore, even if training data only contains correct labels that assign labels to regions of one of the multiple classes, a trained neural network that classifies regions in an image into multiple classes can be built if training data containing correct labels that assign labels to regions of a different class is available.

[0044] Next, a second embodiment of the present disclosure will be described. Note that the configuration of the learning device according to the second embodiment is the same as the configuration of the learning device according to the first embodiment, so a detailed description of the device will be omitted here. In the second embodiment, the process of integrating probabilities differs from that of the first embodiment.

[0045] The trained neural network constructed in the second embodiment classifies lung regions included in an input image into five lobes, including the right upper lobe, the right middle lobe, the right lower lobe, the left upper lobe, and the left lower lobe. For this purpose, in the second embodiment, training data shown in FIG. 10 is prepared. As shown in FIG. 10, training data 32 used in the second embodiment includes training images 32A and correct labels 32B. The correct labels 32B are assigned different labels 32C, 32D, 32E, 32F, and 32G to the right upper lobe, the right middle lobe, the right lower lobe, the left upper lobe, and the left lower lobe. In the second embodiment, training data 31 including correct labels 31B, which label the right and left lungs, as shown in FIG. 5, is also prepared.

[0046] Here, since the correct labels in which only the right and left lungs are labeled as shown in Figure 5 can be easily created, it is possible to prepare a large amount of training data 31. On the other hand, since the training data shown in Figure 10 places a heavy burden on the creator to create the correct labels, it is not possible to prepare a large amount of training data 32. In the second embodiment, a trained neural network is constructed by training a neural network so that the lungs can be classified into five lobe regions even in such a situation.

[0047] Fig. 11 is a diagram showing a model of neural network learning in the second embodiment. Note that the neural network 50 shown in Fig. 11 is a convolutional neural network similar to the neural network 40 in the first embodiment, and is composed of an input layer 51, multiple intermediate layers 52, and an output layer 53.

[0048] When the training image 31A is input to the neural network 50, for each pixel included in the training image 31A, Logits, which is a value representing the likelihood that the pixel is the background, the right upper lobe, the right middle lobe, the right lower lobe, the left upper lobe, or the left lower lobe, is output from the neural network 50. For example, for the background, the right upper lobe, the right middle lobe, the right lower lobe, the left upper lobe, or the left lower lobe, Logits values ​​such as (1.0, 3.0, 2.0, 1.5, 3.1, 5.0) are output.

[0049] The learning unit 22 applies a softmax activation function (Soft Max) to the Logits output from the neural network 50 to convert the Logits into probabilities p10 to p15. For example, for the background, right upper lobe, right middle lobe, right lower lobe, left upper lobe, and left lower lobe, probabilities such as (p10, p11, p12, p13, p14, p15) = (0.1, 0.1, 0.1, 0.1, 0.1, 0.5) are obtained. Note that p10 + p11 + p21 + p31 + p14 + p15 = 1.

[0050] Here, the correct label 31B corresponding to the training image 31A only labels the left and right lungs. Therefore, in the second embodiment, the learning unit 22 integrates the derived probabilities based on the classes classified by the correct labels of the training data. For example, when the training image 31A is input to the neural network 50, the probabilities p11, p12, and p13 of the right upper lobe, middle lobe, and lower lobe of the right lung are integrated into the probability pt11 of the right lung, and the probabilities p14 and p15 of the left upper lobe and lower lobe of the left lung are integrated into the probability pt12 of the left lung. In this case, pt11 = p11 + p12 + p13, and pt12 = p14 + p15. Therefore, if the probabilities (p10, p11, p12, p13, p14, p15) = (0.1, 0.1, 0.1, 0.1, 0.5), then the combined probability (p10, pt11, pt12) = (0.1, 0.3, 0.6).

[0051] The learning unit 22 uses the integrated probability distribution and the ground truth label to derive the cross-entropy error as a loss L0. The cross-entropy error corresponds to the distance between the probability distribution and the vector represented by the ground truth label. Here, since the ground truth label 31B of the input learning image 31A has labels assigned to the left and right lung regions, the ground truth label vector for deriving the loss with the integrated probability is (0,1,0) for the right lung and (0,0,1) for the left lung.

[0052] The learning unit 22 continues learning the neural network 50 until the loss L0 satisfies the termination condition, which is the same as in the first embodiment.

[0053] 10 is input to the neural network 50, the correct answer label 32B of the training image 32A includes labels for the right upper lobe, right middle lobe, right lower lobe, left upper lobe, and left lower lobe. In this case, the learning unit 22 derives the loss L0 between the probability and the correct label and trains the neural network 50 without integrating the probability distributions (p10, p11, p12, p13, p14, p15) as shown in FIG. In this case, the correct label vectors are (1,0,0,0,0,0) for the background, (0,1,0,0,0,0) for the right upper lobe, (0,0,1,0,0,0) for the right middle lobe, (0,0,1,0,0,0) for the right lower lobe, (0,0,0,1,0,0) for the left upper lobe, (0,0,0,0,1,0) and (0,0,0,0,0,1) for the left lower lobe.

[0054] When a CT image is input, the trained neural network constructed by such machine learning outputs the probability that each pixel in the CT image is the right upper lobe, right middle lobe, right lower lobe, left upper lobe, left lower lobe, or background. Therefore, by using the trained neural network constructed by the learning device according to the second embodiment, it is possible to classify the CT image into the area with the highest probability for each pixel.

[0055] In the second embodiment, the lungs are classified into five lobes, but the classification is not limited to this. For example, the learning device according to the second embodiment can be applied to constructing a trained neural network that classifies the liver into eight liver segments S1 to S8. In this case, segments S1 to S3 are integrated into the left lobe of the liver, and segments S4 to S8 are integrated into the right lobe of the liver, thereby enabling training of the neural network in the same manner as in the second embodiment. Furthermore, when training a neural network that classifies bones into the skull, spine, ribs, scapulae, pelvis, arms, and legs, the skull, spine, ribs, scapulae, and arms can be integrated into the upper body skeleton, and the pelvis and legs can be integrated into the lower body skeleton, thereby enabling training of the neural network in the same manner as in the second embodiment.

[0056] In addition, in the above-described embodiments, the liver and lung regions included in the image are classified, but this is not limiting. The technology of the first embodiment can also be applied to classifying any other part of the human body included in the image, such as the heart, brain, kidneys, bones, and limbs, in addition to the liver and lungs.

[0057] In the above embodiment, CT images are used as images for classifying, but the present invention is not limited to this. By using any images as training images, such as 3D images such as MRI images, or radiographic images obtained by simple radiography, a trained neural network can be constructed that classifies regions in any image into multiple classes.

[0058] In the above embodiment, a trained neural network is constructed to classify regions in a medical image into multiple classes, but the present invention is not limited to this. The technology of this embodiment can also be applied to classifying representation media other than medical images, such as photographic images, video images, audio, and text, into multiple classes.

[0059] Furthermore, in the above embodiment, the following various processors can be used as the hardware structure of a processing unit that executes various processes, such as the information acquisition unit 21 and the learning unit 22. The above various processors include, as described above, a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as programmable logic devices (PLDs), which are processors whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and dedicated electrical circuits, such as an ASIC (Application Specific Integrated Circuit), which are processors with a circuit configuration designed specifically for executing specific processes.

[0060] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0061] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by computers such as client and server. Second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0062] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. [Explanation of symbols]

[0063] 1. Computer 2 Imaging device 3. Image storage server 4 Network 11 CPU 12 Study Program 13. Storage 14 Display 15 Input Devices 20 Learning Device 21 Information Acquisition Department 30~32 Training data 30A~32A Learning images 30B~32B Correct Label 30C, 31C, 31D, 32C, 32D, 32E, 32F, 32G Label 40,50 Neural Networks 41,51 Input layer 42,52 Middle class 43,53 Output layer L0 loss

Claims

1. A learning device that performs machine learning on a neural network that classifies expression media into three or more classes, at least one processor; The processor: acquiring training data consisting of a learning representation medium and a correct label for at least one class among a plurality of types of classes included in the learning representation medium; inputting the training representation medium into the neural network and outputting the probability that the classes included in the training representation medium are each of the plurality of classes; Integrating the probabilities of each of the plurality of classes based on the classes classified by the correct labels of the training data; A learning device that trains the neural network based on the integrated probability and a loss derived from the correct label of the training data.

2. the expression medium is an image, the plurality of classes are a plurality of regions including a background in the image, The learning device of claim 1, wherein the processor integrates the probabilities of each of the multiple types of classes by adding the probability of classes other than the class classified by the correct label for the learning expression medium and the probability of the background.

3. the classes classified by the correct labels include two or more classes among the plurality of types of classes, 2. The learning device according to claim 1, wherein the processor integrates the probabilities of each of the plurality of classes by adding up the probabilities of the two or more classes classified by the correct label among the probabilities of the plurality of classes.

4. The learning device according to claim 1 , wherein the processor learns the neural network using a plurality of training data sets having different correct answer labels.

5. A learning method for machine learning a neural network that classifies expression media into three or more classes, comprising: acquiring training data consisting of a learning representation medium and a correct label for at least one class among a plurality of types of classes included in the learning representation medium; inputting the training representation medium into the neural network and outputting the probability that the classes included in the training representation medium are each of the plurality of classes; Integrating the probabilities of each of the plurality of classes based on the classes classified by the correct labels of the training data; A learning method for training the neural network based on the integrated probability and a loss derived from the correct answer label of the training data.

6. A learning program that causes a computer to execute a learning method for machine learning a neural network that classifies expression media into three or more classes, A step of acquiring training data consisting of a learning representation medium and a correct label for at least one class among a plurality of types of classes included in the learning representation medium; a step of inputting the training representation medium into the neural network and outputting a probability that each of the classes included in the training representation medium is a class of the plurality of types; a step of integrating the probabilities of each of the plurality of types of classes based on the classes classified by the correct answer labels of the training data; and a procedure for training the neural network based on the integrated probability and a loss derived from the correct label of the training data.

Images