Learning method, classification method, learning apparatus, classification method, learning program, and classification program

Abstract

To provide a learning apparatus, and the like, capable of training a model while suppressing reduction in classification performance due to training data for which regions to be extracted and regions not to be extracted are imbalanced.SOLUTION: A learning apparatus for classifying input images, using a model, as to whether the image is to be extracted or not for each of pixels includes: an application unit which inputs a training image to the model; a calculation unit which calculates a loss for each pixel, using an output of the model and a ground truth corresponding to the training image; a setting unit which sets weights for two regions so that the weight may be smaller when the number of pixels included in each region is larger and that the weight may be larger when the number of pixels is smaller, the two regions including a region which is classified to be extracted in the output of the model and which is not to be extracted in the ground truth, and a region which is classified not to be extracted in the output of the model and which is to be extracted in the ground truth; and a training unit which trains the model while applying weights to the loss of each of the pixels using the weights.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The disclosed technology relates to a learning method, a classification method, a learning device, a classification method, a learning program, and a classification program. [Background technology]

[0002] There is a known method of learning using a loss called Dice loss, which is effective for imbalanced data (Non-Patent Document 1). In this method, the Dice coefficient is calculated by dividing the overlap between the correct answer region and the predicted region by the sum of the correct answer region and the predicted region, and the loss is calculated using this.

[0003] There is also a known learning method that uses a loss called focal loss, which is effective for imbalanced data (Non-Patent Document 2). In this method, the loss is calculated by weighting using the predicted probability of each class.

[0004] In addition, medical images are classified into normal and abnormal states. An image diagnostic device is known that assigns individual weights to the predicted probability distribution of normal and abnormal states and calculates and learns the weighted loss for false positives or false negatives (Patent Document 1). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Milletari, et al., "V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. ", International Conference on 3D Vision, 2016. [Non-Patent Document 2] Lin, et al., "Focal loss for dense object detection. ", ICCV, 2017. [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2022-549051 Summary of the Invention [Problem to be solved by the invention]

[0007] Here, let us consider image segmentation. In this case, if the area of ​​the image that is the target of extraction is small, the ratio of the target to other areas will be unbalanced. If a general segmentation loss calculation method is applied to such data, the loss calculated for each pixel is averaged over the entire image, so the effect of the small target of extraction on the learning model will be small, resulting in an incorrect classification that the entire image is not the target of extraction.

[0008] In one aspect, the present invention aims to provide a learning method, a learning device, and a learning program that can learn a model while suppressing a decrease in classification performance caused by learning data that is unbalanced between areas that are not to be extracted and areas that are to be extracted.

[0009] Another object of the present invention is to provide a classification method, a classification device, and a classification program that can suppress a decrease in classification performance caused by learning data that is unbalanced between areas that are not subject to extraction and areas that are subject to extraction. [Means for solving the problem]

[0010] A first aspect of the present disclosure is a learning method for classifying, for each pixel in an input image, whether or not it is an extraction target using a model, wherein a computer executes processes including the steps of: inputting a learning image into the model; calculating a loss for each pixel using an output from the model and a correct answer corresponding to the learning image; setting weights for each region in two types of regions, i.e., a region classified as the extraction target in the output of the model and where the correct answer is a region that is not the extraction target, and a region classified as not the extraction target in the output of the model and where the correct answer is a region that is the extraction target, such that the weights become smaller the greater the number of pixels contained in each region and the larger the number of pixels, and the weights become larger the smaller the number of pixels; and learning the model by weighting the loss of each pixel using the weights.

[0011] A second aspect of the present disclosure is a classification method in which a computer executes a process including a step of comparing a probability that an input image is an extraction target represented by an output of a trained model trained by the learning method of the first aspect with a threshold, and classifying each pixel as to whether it is an extraction target or not.

[0012] A third aspect of the present disclosure is a learning device for classifying, for each pixel in an input image, whether or not it is an extraction target using a model, the learning device including: an application unit that inputs a learning image to the model; a calculation unit that calculates a loss for each pixel using an output from the model and a correct answer corresponding to the learning image; a setting unit that sets weights for each region in two types of regions, i.e., a region classified as the extraction target in the output of the model and where the correct answer is not the extraction target, and a region classified as not the extraction target in the output of the model and where the correct answer is the extraction target, so that the weights become smaller the greater the number of pixels contained in each region and the larger the number of pixels, and a learning unit that weights the loss of each pixel using the weights to train the model.

[0013] A fourth aspect of the present disclosure is a classification device that includes a classification unit that compares the probability that an input image is an extraction target represented by the output of a trained model trained by the learning device of the third aspect with a threshold value, and classifies each pixel as to whether it is an extraction target or not.

[0014] A fifth aspect of the present disclosure is a learning program for classifying, using a model, each pixel in an input image as to whether it is an extraction target or not, the program inputting a learning image into the model, calculating a loss for each pixel using an output from the model and a correct answer corresponding to the learning image, and setting weights for each region such that the greater the number of pixels contained in each region, the smaller the weight is, and the smaller the number of pixels, for two types of regions, i.e., a region that is classified as the extraction target in the output of the model and the correct answer is a region that is not the extraction target, and a region that is classified as not the extraction target in the output of the model and the correct answer is a region that is the extraction target, and setting weights for each region such that the weights become smaller the greater the number of pixels contained in each region, and the weights become larger the smaller the number of pixels, and the program causes a computer to execute a process of weighting the loss of each pixel using the weights to learn the model.

[0015] A sixth aspect of the present disclosure is a classification program that causes a computer to execute a process of comparing the probability that an input image is an extraction target represented by the output of a trained model trained by the learning program of the fifth aspect with a threshold value, and classifying each pixel as to whether it is an extraction target or not. Effect of the Invention

[0016] According to one aspect, a model can be trained while suppressing degradation of classification performance caused by training data that is imbalanced between regions that are not the target of extraction and regions that are the target of extraction.

[0017] In addition, it is possible to suppress a decrease in classification performance caused by learning data in which the areas that are not the extraction target and the areas that are the extraction target are unbalanced. [Brief description of the drawings]

[0018] [Figure 1]FIG. 1 is an explanatory diagram illustrating an example of the configuration of a classification system. [Diagram 2] FIG. 1 is a schematic block diagram of an example of a computer that functions as a learning device and a classification device of the present embodiment. [Diagram 3] 1 is a block diagram showing a configuration of a learning device according to an embodiment of the present invention; [Figure 4] FIG. 13 is a diagram showing an example of a region to be extracted. [Diagram 5] FIG. 13 is a diagram showing an example of a region to be extracted. [Figure 6] FIG. 13 is a diagram illustrating an example of weighting losses. [Figure 7] FIG. 13 is a diagram illustrating an example of weighting a gradient. [Figure 8] 1 is a block diagram showing a configuration of a classification device according to an embodiment of the present invention. [Figure 9] 4 is a flowchart showing a learning process routine in the learning device of the present embodiment. [Figure 10] 4 is a flowchart showing a classification processing routine in the classification device of the present embodiment. [Figure 11] FIG. 1 is a diagram illustrating an example of a medical image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals. In addition, the dimensional ratios of the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0020] <System configuration> 1 is an explanatory diagram showing an example of the configuration of a classification system 2. In this embodiment, a classification system 2 that performs segmentation of a region of a lesion part targeted at a medical image will be described. The classification system 2 includes a learning device 10 and a classification device 20. The learning device 10 and the classification device 20 are communicatively connected via a network N.

[0021] In this embodiment, the brain is taken as an example of a body part to be classified, but other body parts may be used.

[0022] The learning device 10 is, for example, a server computer capable of various information processing and information transmission and reception. Note that the device corresponding to the learning device 10 is not limited to a server computer, and may be, for example, a personal computer. In this embodiment, the learning device 10 performs machine learning such as CNN (Convolution Neural Network) using previously prepared learning data, and when a captured medical image is input, functions as a learning device that learns a model that is a classifier that classifies whether or not each pixel is an extraction target (for example, a lesion part of a target site). Data of the trained model trained by the learning device 10 is installed in the classification device 20, and the classification device 20 uses the trained model to perform segmentation of the area to be extracted from the medical image.

[0023] The classification device 20 is a general-purpose computer such as a personal computer. In this embodiment, the classification device 20 uses a trained model to classify each pixel in a medical image as to whether or not it is a region to be extracted, and performs segmentation of the region to be extracted.

[0024] In this embodiment, the learning device 10 generates (learns) the trained model, but the local classification device 20 may generate the trained model. In this embodiment, the classification device 20 classifies whether or not the area is an extraction target based on the trained model, but the learning device 10 on the cloud may classify whether or not the area is an extraction target using the trained model. In other words, the distinction between the two is for convenience, and a single computer may perform a series of processes.

[0025] <Configuration of the learning device according to this embodiment> FIG. 2 is a block diagram showing the hardware configuration of the learning device 10 of this embodiment.

[0026] 2, the learning device 10 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0027] The CPU 11 is a central processing unit, and executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage 14, and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various arithmetic processing according to the program stored in the ROM 12 or the storage 14. In this embodiment, a learning program for executing a learning process is stored in the ROM 12 or the storage 14. The learning program may be one program, or may be a group of programs composed of multiple programs or modules.

[0028] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including an operating system and various data.

[0029] The input unit 15 includes a pointing device such as a mouse, and a keyboard, and is used to perform various input operations.

[0030] The input unit 15 receives, as learning data, a plurality of medical images to which regions to be extracted have been assigned in advance.

[0031] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may function as the input unit 15 by adopting a touch panel system.

[0032] The communication interface 17 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0033] Next, a description will be given of the functional configuration of the learning device 10. FIG 3 is a block diagram showing an example of the functional configuration of the learning device 10.

[0034] As shown in FIG. 3, the learning device 10 is functionally configured to include a learning data storage unit 101, an acquisition unit 102, an application unit 103, a calculation unit 104, a setting unit 105, and a learning unit 106.

[0035] A plurality of input learning data are stored in the learning data storage unit 101. Specifically, the learning data storage unit 101 stores a plurality of learning medical images to which regions made up of pixels to be extracted are previously assigned.

[0036] The acquisition unit 102 acquires medical images for learning from the learning data storage unit 101 .

[0037] The application unit 103 inputs a medical image for learning to the model. The model outputs the probability that each pixel is an extraction target. A certain probability is used as a threshold to determine whether or not the pixel is an extraction target.

[0038] The calculation unit 104 calculates the loss for each pixel of the training medical images stored in the training data storage unit 101 using the output from the model and the correct answer corresponding to the training medical image.

[0039] Specifically, for each pixel in the medical image used for training, the difference between the probability that it is the output from the model and the binary value indicating whether or not it is an extraction target assigned as the correct answer is calculated.

[0040] Setting unit 105 sets the weights of two types of regions: regions that are classified as extraction targets in the model output and for which the correct answer is not an extraction target (mesh regions in Figure 4), and regions that are classified as not extraction targets in the model output and for which the correct answer is an extraction target (dot regions in Figure 4). The setting unit 105 sets the weights of each region so that the weights become smaller as the number of pixels contained in the region increases and larger as the number of pixels decreases.

[0041] Note that weights may be set for four types of regions. Specifically, the setting unit 105 sets weights for each of the following four types of regions based on the number of pixels included in each region: a region classified as an extraction target in the model output and not an extraction target for the correct answer (mesh region in FIG. 5), a region classified as not an extraction target in the model output and not an extraction target for the correct answer (dot region in FIG. 5), a region classified as an extraction target in the model output and not an extraction target for the correct answer (diagonal mesh region in FIG. 5), and a region classified as not an extraction target in the model output and not an extraction target for the correct answer (gray region in FIG. 5).

[0042] The learning unit 106 uses the weights to weight the loss of each pixel and learns the model.

[0043] Specifically, first, a loss function is calculated by reflecting the weight of each region based on the number of pixels, using any one of the following first to fourth methods, and a model is trained.

[0044] In the first method, the setting unit 105 sets the inverse of the number of pixels contained in each region as the weight for each region, and the loss of each pixel is weighted by the weight of the region containing the pixel to train the model.

[0045] In the second method, for each type of region, the average loss of the pixels in the region is calculated as the loss of the region, and the losses for each type of region are weighted according to the type of region and added together to calculate a single loss. In this way, the effect of the number of pixels in each region can be reduced by adding up the losses for each region.

[0046] In the third method, for each type of region, the average loss over the pixels in the region is calculated as the loss for the region, and then the losses for each type of region are averaged to calculate a single loss.

[0047] In the fourth method, for each type of region, the average loss of the pixels in the region is calculated and this is used as the loss for that region. The losses for each type of region are then summed, with a weight of 0 for each type except for at least one, to calculate a single loss.

[0048] In order to optimize the value of the loss function calculated by any one of the above first to fourth methods, the gradient of the error backpropagation is calculated for each pixel, and the parameters of the model are updated based on the gradient for each pixel.

[0049] Alternatively, the model may be trained using the gradient of backpropagation calculated based on the number of pixels in each type of region (see Fig. 6). Fig. 6 shows an example in which the loss calculated for each pixel is weighted by a weight set based on the number of pixels in each type of region, the gradient of backpropagation is calculated, and the model parameters are updated.

[0050] Also, the model may be trained using the gradient of backpropagation calculated by weighting with the inverse of the number of pixels in each type of region (see Fig. 7). Fig. 7 shows an example in which the gradient of backpropagation is calculated for the loss calculated for each pixel, and weighted with a weight set based on the number of pixels in each type of region, to update the model parameters.

[0051] <Configuration of the Classification Device According to the Present Embodiment>

[0052] FIG. 2 is a block diagram showing the hardware configuration of the classification device 20 of this embodiment.

[0053] 2, the classification device 20, like the learning device 10, has a CPU 11, a ROM 12, a RAM 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other so as to be able to communicate with each other via a bus 19. Note that the classification device 20 preferably includes a processor with high computational processing power such as a GPU in order to perform image processing.

[0054] A classification program for executing classification processing is stored in the ROM 12 or the storage 14. The classification program may be one program, or may be a group of programs configured with multiple programs or modules.

[0055] The input unit 15 receives medical images to be classified.

[0056] Next, a description will be given of the functional configuration of the classification device 20. FIG.

[0057] Functionally, the classification device 20 includes an input acquisition unit 201, a classification unit 202, and an output unit 203, as shown in FIG.

[0058] The input acquisition unit 201 acquires medical images to be classified.

[0059] The classification unit 202 includes a trained model for classifying whether or not an object is an extraction target, which is trained by the learning device 10. In this embodiment, the trained model outputs a probability indicating the degree to which each pixel of the input medical image is an extraction target. The trained model is a neural network. It may also be a model based on other learning algorithms, such as SVM, decision tree, or random forest. A higher probability indicates a higher possibility of being an extraction target.

[0060] Specifically, for each pixel in the medical image, the classification unit 202 uses the trained model to compare the probability that the pixel is an extraction target represented by the output of the trained model with a threshold value, classifies each pixel as to whether it is an extraction target or not, and segments the area consisting of pixels that are extraction targets.

[0061] The output unit 203 outputs the segmentation result by the classification unit 202. For example, for a medical image, the display unit 16 displays a region made up of pixels that are the extraction target.

[0062] <Function of the learning device according to this embodiment> Next, the operation of the learning device 10 according to this embodiment will be described.

[0063] 9 is performed by CPU 11 reading out a learning program from ROM 12 or storage 14, expanding it into RAM 13, and executing it. In addition, it is assumed that a plurality of medical images, each having a region made up of pixels to be extracted in advance, are input to learning device 10 as learning images, and the input plurality of learning medical images are stored in learning data storage unit 101. CPU 11, as acquisition unit 102, acquires a plurality of learning medical images from learning data storage unit 101.

[0064] First, in step S100, the CPU 11 as the application unit 103 inputs each of a plurality of medical images for learning into the model.

[0065] In step S102, the CPU 11, functioning as the calculation unit 104, calculates the loss for each pixel for a plurality of training medical images using the output from the model and the correct answer corresponding to the training medical image.

[0066] In step S104, the CPU 11, as the setting unit 105, sets the weighting of each type of region, including regions that are classified as an extraction target in the model output and for which the correct answer is not an extraction target, and regions that are classified as not an extraction target in the model output and for which the correct answer is an extraction target, so that the weighting becomes smaller the greater the number of pixels contained in each region and the greater the weighting becomes for each smaller number of pixels.

[0067] In step S106, the CPU 11 functions as the learning unit 106 and performs model learning by weighting the loss of each pixel using the weight.

[0068] In step S108, the CPU 11 judges whether or not to end the iteration, and judges that the iteration is to be ended when the number of iterations reaches the upper limit, for example, and ends the learning process. On the other hand, when the number of iterations does not reach the upper limit, the iteration is not ended and the process returns to the above step S100.

[0069] <Function of the classification device according to this embodiment> Next, the operation of the classification device 20 according to this embodiment will be described.

[0070] The CPU 11 of the classification device 20 reads out the classification program from the ROM 12 or the storage 14, loads it into the RAM 13, and executes it, thereby performing the classification process shown in FIG.

[0071] First, in step S120, the CPU 11 functions as the input acquisition unit 201 to acquire a medical image to be classified.

[0072] In step S122, the CPU 11 functions as the classification unit 202 and classifies each pixel of the medical image as to whether or not it is an extraction target, using the trained model.

[0073] In step S124, the CPU 11, functioning as the classification unit 202, segments an area made up of pixels to be extracted.

[0074] In step S126, the CPU 11 outputs the segmentation result by the classification unit 202 as the output unit 203. For example, for the medical image, a region consisting of pixels to be extracted is displayed on the display unit 16, and the classification process is terminated. Then, for example, a doctor diagnoses the subject while looking at the segmentation result of the region to be extracted for the medical image displayed on the display unit 16.

[0075] As described above, the learning device according to this embodiment sets weights for each type of region, including regions that are classified as regions to be extracted in the model output and for which the correct answer is not a region to be extracted, and regions that are classified as regions to be not a region to be extracted in the model output and for which the correct answer is a region to be extracted, so that the weights become smaller as the number of pixels contained in each region increases and become larger as the number of pixels decreases, and the model is trained by weighting the loss of each pixel using the weights. This makes it possible to train the model while suppressing a decrease in classification performance due to training data that is unbalanced between regions that are not to be extracted and regions that are to be extracted.

[0076] For example, even in the case of a medical image for training in which the regions that are not the extraction target and the regions that are the extraction target are unbalanced as shown in FIG. 11, it is possible to train a model while suppressing a decrease in classification performance.

[0077] In addition, the classification device classifies an input image as to whether it is an extraction target or not using a trained model trained by a learning device, thereby making it possible to suppress a decrease in classification performance for imbalanced data.

[0078] <Modification> The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the spirit and scope of the present invention.

[0079] For example, although the example has been described in which the model is learned in the learning device 10, the model may be learned in the classification device 20. In this case, the classification device 20 may further include a learning data storage unit, an acquisition unit, an application unit, a setting unit, and a learning unit.

[0080] In addition, although the case where medical images are the target has been described as an example, the present invention is not limited to this. Images other than medical images may be classified as targets. For example, equipment images of equipment to be inspected may be the target. In this case, the extraction target is a damaged part of the equipment to be inspected.

[0081] In addition, various processes executed by the CPU by reading the software (program) in each of the above embodiments may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. Furthermore, the learning process and the classification process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). Moreover, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0082] In addition, in each of the above embodiments, the learning program and the classification program are described as being pre-stored (installed) in the storage 14, but the present invention is not limited to this. The programs may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The programs may also be downloaded from an external device via a network.

[0083] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims.

[0084] The following supplementary notes are further disclosed regarding the above embodiment.

[0085] (Appendix 1) A learning method for classifying, for each pixel in an input image, whether or not it is an extraction target by using a model, comprising the steps of: inputting training images into the model; Calculating a loss for each pixel using an output from the model and a ground truth corresponding to the training image; a step of setting weights for two types of regions, i.e., a region that is classified as the extraction target in the output of the model and the correct answer is a region that is not the extraction target, and a region that is classified as not the extraction target in the output of the model and the correct answer is a region that is the extraction target, so that the weights become smaller as the number of pixels contained in each region increases and larger as the number of pixels decreases; weighting the loss of each pixel using the weights to train the model; A learning method in which a computer executes a process including the steps of:

[0086] (Appendix 2) 2. The learning method according to claim 1, wherein the image is a medical image of a target area of ​​a subject, and the extraction target is a diseased part of the target area.

[0087] (Appendix 3) The learning method according to claim 1, wherein the image is an equipment image of the equipment to be inspected, and the extraction target is a damaged portion of the equipment to be inspected.

[0088] (Appendix 4) In the setting step, 4. The learning method according to any one of Supplementary Note 1 to Supplementary Note 3, wherein a weight for each of four types of regions is set based on the number of pixels contained in the region: a region classified as the extraction target in the output of the model and where the correct answer is a region that is not the extraction target; a region classified as not the extraction target in the output of the model and where the correct answer is the extraction target; a region classified as the extraction target in the output of the model and where the correct answer is the extraction target; and a region classified as not the extraction target in the output of the model and where the correct answer is the extraction target.

[0089] (Appendix 5) In the setting step, the reciprocal of the number of pixels contained in each region is set as a weight for each region; 5. The learning method according to claim 1, wherein in the learning step, the model is learned by weighting the loss of each pixel with a weight of a region including the pixel.

[0090] (Appendix 6) The learning method according to any one of Supplementary Note 1 to Supplementary Note 4, wherein in the learning step, for each type of region, an average of losses in pixels within the region is calculated to be set as a loss for the region, and the losses for each type of region are weighted according to the type of the region and added together to calculate a single loss.

[0091] (Appendix 7) The learning method according to any one of Supplementary Note 1 to Supplementary Note 4, wherein in the learning step, for each type of region, an average of losses in pixels within the region is calculated as a loss for the region, and the losses for each type of region are averaged to calculate a single loss.

[0092] (Appendix 8) The learning method according to any one of Supplementary Note 1 to Supplementary Note 4, wherein in the learning step, for each type of region, an average of losses in pixels within the region is calculated to be the loss for the region, and the losses for each type of region are added together with 0 as a weight for each type except for at least one type to calculate a single loss.

[0093] (Appendix 9) 5. The learning method according to claim 1, wherein in the learning step, the model is learned using a gradient of backpropagation of an error calculated based on the number of pixels in each type of region.

[0094] (Appendix 10) The learning method according to any one of Supplementary Note 1 to Supplementary Note 4, wherein in the learning step, the model is learned using a gradient of backpropagation of an error calculated by weighting with the inverse of the number of pixels in each type of region.

[0095] (Appendix 11) A classification method in which a computer executes a process including a step of comparing a probability that an input image is an extraction target represented by an output of a trained model trained by the learning method according to any one of Supplementary Note 1 to Supplementary Note 10 with a threshold, and classifying each pixel as to whether it is an extraction target or not.

[0096] (Appendix 12) A learning device for classifying, for each pixel in an input image, whether or not it is an extraction target by using a model, comprising: an application unit for inputting training images into the model; A calculation unit that calculates a loss for each pixel using an output from the model and a correct answer corresponding to the learning image; a setting unit that sets weights for two types of regions, i.e., a region that is classified as the extraction target in the output of the model and the correct answer is a region that is not the extraction target, and a region that is classified as not the extraction target in the output of the model and the correct answer is a region that is the extraction target, so that the weights for each region are smaller as the number of pixels included in the region increases and larger as the number of pixels decreases; a learning unit that uses the weights to weight the loss of each pixel and learns the model; A learning device comprising:

[0097] (Appendix 13) A classification unit that compares the probability that an input image is an extraction target represented by an output of a trained model trained by the learning device according to claim 12 with a threshold value and classifies each pixel as to whether it is an extraction target or not. A classification device comprising:

[0098] (Appendix 14) A learning program for classifying, for each pixel in an input image, whether or not it is an extraction target by using a model, Inputting training images into the model; Calculate a loss for each pixel using the output from the model and a ground truth corresponding to the training image; a weighting for each of two types of regions, i.e., a region that is classified as the extraction target in the output of the model and the correct answer is not the extraction target, and a region that is classified as not the extraction target in the output of the model and the correct answer is the extraction target, the weighting for each region is set so that the weighting for each region becomes smaller as the number of pixels contained in the region increases and the weighting for each region becomes larger as the number of pixels contained in the region decreases; The weights are used to weight the loss of each pixel and train the model. A learning program that causes a computer to carry out processing.

[0099] (Appendix 15) In the input image, the probability that the input image is an extraction target represented by the output of the trained model trained by the learning program described in Supplementary Note 14 is compared with a threshold value to classify each pixel as an extraction target or not. A classification program that causes a computer to carry out the processing. [Explanation of symbols]

[0100] 2. Classification System 10 Learning Device 11 CPU 14. Storage 20 Classifier 101 Learning data storage unit 102 Acquisition Department 103 Application section 104 Calculation section 105 Setting section 106 Learning Department 201 Input acquisition unit 202 Classification Department 203 Output section

Claims

1. A learning method for classifying, for each pixel in an input image, whether or not it is an extraction target by using a model, comprising the steps of: inputting training images into the model; Calculating a loss for each pixel using an output from the model and a ground truth corresponding to the training image; a step of setting weights for two types of regions, i.e., a region that is classified as the extraction target in the output of the model and the correct answer is a region that is not the extraction target, and a region that is classified as not the extraction target in the output of the model and the correct answer is a region that is the extraction target, so that the weights become smaller as the number of pixels included in each region increases and larger as the number of pixels decreases; weighting the loss of each pixel using the weights to train the model; A learning method in which a computer executes a process including the steps of:

2. 2. The learning method according to claim 1, wherein the image is a medical image of a target part of a subject, and the extraction target is a diseased part of the target part.

3. The learning method according to claim 1 , wherein the image is an image of an equipment to be inspected, and the extraction target is a damaged portion of the equipment to be inspected.

4. In the setting step, 2. The learning method according to claim 1, wherein a weight is set for each of four types of regions based on the number of pixels contained in the region: a region classified as the extraction target in the output of the model and which is not the extraction target, a region classified as not the extraction target in the output of the model and which is the correct answer, a region classified as the extraction target in the output of the model and which is the correct answer, and a region classified as not the extraction target in the output of the model and which is the correct answer.

5. In the setting step, the reciprocal of the number of pixels contained in each region is set as a weight for each region; 2. The learning method according to claim 1, wherein in said learning step, said model is learned by weighting the loss of each pixel with the weight of the region including said pixel.

6. 2. The learning method according to claim 1, wherein in the learning step, for each type of region, an average of losses in pixels within the region is calculated as a loss for the region, and the losses for each type of region are weighted according to the type of the region and added together to calculate a single loss.

7. 2. The learning method according to claim 1, wherein in the learning step, for each type of region, an average of the losses in the pixels within the region is calculated as the loss for the region, and the losses for each type of region are averaged to calculate a single loss.

8. 2. The learning method according to claim 1, wherein in the learning step, for each type of region, an average of the losses in the pixels within the region is calculated as the loss for the region, and the losses for each type of region are added together with a weighting of 0 for each type except for at least one type to calculate a single loss.

9. 2. The method according to claim 1, wherein in the learning step, the model is trained using gradients of backpropagation of errors calculated based on the number of pixels in each type of region.

10. 2. The learning method according to claim 1, wherein in said learning step, said model is learned using a gradient of backpropagation of an error calculated by weighting with the inverse of the number of pixels in each type of region.

11. A classification method in which a computer executes a process including a step of comparing the probability that an input image is an extraction target represented by the output of a trained model trained by the learning method described in any one of claims 1 to 10 with a threshold, and classifying each pixel as to whether it is an extraction target or not.

12. A learning device for classifying, for each pixel in an input image, whether or not it is an extraction target by using a model, comprising: an application unit for inputting training images into the model; A calculation unit that calculates a loss for each pixel using an output from the model and a correct answer corresponding to the learning image; a setting unit that sets weights for two types of regions, i.e., a region that is classified as the extraction target in the output of the model and the correct answer is a region that is not the extraction target, and a region that is classified as not the extraction target in the output of the model and the correct answer is a region that is the extraction target, so that the weights for each region are smaller as the number of pixels included in the region increases and larger as the number of pixels decreases; a learning unit that uses the weights to weight the loss of each pixel and learns the model; A learning device comprising:

13. a classification unit that compares the probability of an input image being an extraction target represented by an output of a trained model trained by the learning device according to claim 12 with a threshold value and classifies each pixel as to whether it is an extraction target or not. A classification device comprising:

14. A learning program for classifying, for each pixel in an input image, whether or not it is an extraction target by using a model, Inputting training images into the model; Calculate a loss for each pixel using the output from the model and a ground truth corresponding to the training image; a weighting for each of two types of regions, i.e., a region that is classified as the extraction target in the output of the model and the correct answer is not the extraction target, and a region that is classified as not the extraction target in the output of the model and the correct answer is the extraction target, is set so that the weighting becomes smaller as the number of pixels included in each region increases and becomes larger as the number of pixels decreases; The weights are used to weight the loss of each pixel and train the model. A learning program that causes a computer to carry out processing.

15. In an input image, the probability that the image is an extraction target represented by the output of a trained model trained by the training program according to claim 14 is compared with a threshold value to classify each pixel as an extraction target or not. A classification program that causes a computer to carry out the processing.

Images