Learning device, prediction device, learning method, and program
By dividing images into partial segments and iteratively selecting reliable segments for training, the method improves prediction accuracy and identifies crucial image regions, addressing the challenge of unclear labeling in deep learning techniques.
Patent Information
- Application Number
- JP2023515983
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-04-22
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-04-22
AI Technical Summary
Existing deep learning techniques struggle to accurately predict image characteristics when labels are given only to the entire image, without clear guidance on which parts of the image are crucial for prediction.
A method involving partial image generation, selection, and reliability evaluation is employed, where the input image is divided into smaller parts, and a prediction model is trained using selected reliable partial images, with their reliability determined by dispersion of predicted values, iteratively improving the model's accuracy.
This approach enhances prediction accuracy by focusing on reliable partial images, allowing for precise identification of important image regions, even when overall image labeling is unclear.
Smart Images

Figure 0007589805000001 
Figure 0007589805000002 
Figure 0007589805000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a technique for predicting a feature portion included in an image. [Background technology]
[0002] There is known a technique for classifying and predicting images by deep learning using a neural network. In so-called supervised learning, a model is trained using learning data in which labels are assigned to input images. However, there are cases in which a label is only given for the entire image, and it is unclear which part of the image should be used for learning. Non-Patent Document 1 discloses a method for the above case in which predictions are made for each partial image obtained by dividing an image into a plurality of parts, and learning is performed using only the partial image with the highest prediction value. [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] Dietterich et al. "Solving the multiple instance problem with axis-parallel rectangles". Artificial intelligence, Volume 89, Issues 1-2, January 1997, Pages 31-71 Summary of the Invention [Problem to be solved by the invention]
[0004] One goal of this disclosure is to learn a model that makes predictions with high accuracy when labels are given only for the entire image. [Means for solving the problem]
[0005] In one aspect of the present disclosure, a learning device includes: a partial image generating means for generating, from an input image, a partial image smaller than the input image; a partial image selection means for selecting a plurality of learning partial images to be used as learning data from all the generated partial images; A learning means for learning a prediction model that predicts the probability that a predetermined feature is included in a selected learning partial image; A prediction means for making predictions for all the partial images using a trained prediction model; a label reliability evaluation means for determining the reliability of each partial image based on the variance of the predicted value of each partial image obtained by multiple predictions in the second or subsequent learning, and outputting the partial image having the reliability equal to or greater than a predetermined reference value to the partial image selection means; Equipped with The partial image selection means selects a plurality of learning partial images to be used in the next learning, based on the predicted values for all the partial images.
[0006] In another aspect of the present disclosure, a computer implemented learning method includes: A partial image smaller than the input image is generated from the input image; Selecting a plurality of learning partial images to be used as learning data from all the generated partial images; training a predictive model that predicts the probability that a predetermined feature is included in the selected training partial image; Using the trained prediction model, predictions are made for all of the partial images; Based on the predicted values for all the partial images, a plurality of partial images for learning to be used in the next learning are selected. death, In the second or subsequent learning, a reliability of each partial image is determined based on a variance in a predicted value of each partial image obtained by multiple predictions; The plurality of learning partial images to be used in the next learning are selected from the partial images whose reliability is equal to or greater than a predetermined reference value. .
[0007] In yet another aspect of the invention, a program comprising: A partial image smaller than the input image is generated from the input image; Selecting a plurality of learning partial images to be used as learning data from all the generated partial images; training a predictive model that predicts the probability that a predetermined feature is included in the selected training partial image; Using the trained prediction model, predictions are made for all of the partial images; Based on the predicted values for all the partial images, a plurality of partial images for learning to be used in the next learning are selected. death, In the second or subsequent learning, a reliability of each partial image is determined based on a variance in a predicted value of each partial image obtained by multiple predictions; The plurality of learning partial images to be used in the next learning are selected from the partial images whose reliability is equal to or greater than a predetermined reference value. The processing is executed by a computer. [Brief description of the drawings]
[0008] [Figure 1] 1 shows a prediction device according to a first embodiment. [Diagram 2] 1 is a block diagram showing a hardware configuration of a prediction device according to a first embodiment. FIG. [Diagram 3] FIG. 2 is a block diagram showing a functional configuration of the prediction device during learning. [Figure 4] An example of a training dataset is shown below. [Diagram 5] 4 shows an example of a method for dividing an input image by an image dividing unit. [Figure 6] 13 shows an example of partial image selection in the initial state. [Figure 7] 11A and 11B are diagrams illustrating an example of a process performed by a label reliability evaluation unit. [Figure 8] The distribution of predicted values for partial images obtained in the first and fifth learning loops is shown. [Figure 9] Examples of partial images of positive examples selected in the first and fifth learning loops are shown below. [Figure 10] An example of the display of the prediction result is shown. [Figure 11] 13 is a flowchart of a learning process of the prediction device. [Figure 12] FIG. 2 is a block diagram showing a functional configuration of the prediction device during inference. [Figure 13] 13 is a flowchart of an inference process of the prediction device. [Figure 14] FIG. 11 is a block diagram showing the functional configuration of a learning device according to a second embodiment. [Figure 15] 10 is a flowchart of a process performed by a learning device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings. <Basic principle> The present disclosure relates to deep learning in the case where a label is given only to the entire image and it is unclear which part of the image is the determining factor for the prediction. Specifically, during learning, the prediction device divides an input image into a plurality of partial images and learns a prediction model that makes predictions for each partial image. Here, in the first stage of learning, the prediction device learns using learning data in which the label given to the entire input image is used as the label for each partial image. On the other hand, when learning is performed and a trained model is obtained, the prediction device predicts each partial image included in the input image using the trained model, and selects a partial image to be used in the next learning based on the prediction result to generate learning data. In this way, a highly accurate prediction model is generated by repeating learning of the prediction model while updating the learning data.
[0010] On the other hand, when making an inference using a trained prediction model, the prediction device divides an input image into a plurality of partial images, performs predictions for each of the divided partial images using the prediction model, and obtains a prediction result for the input image by integrating the prediction results for each partial image. In addition, the prediction device presents parts of the input image that were important for the prediction based on the prediction results for each partial image.
[0011] First Embodiment [Outline of the prediction device] 1 shows a prediction device according to a first embodiment. The prediction device 100 performs prediction on input image data (hereinafter also referred to as an "input image"). In this embodiment, the input image is an image of a national flag, and the prediction device 100 predicts the probability that a green portion is included as a feature of the input image.
[0012] [Hardware configuration] 2 is a block diagram showing a hardware configuration of a prediction device 100 according to the first embodiment. As shown in the figure, the prediction device 100 includes an interface (IF) 12, a processor 13, a memory 14, a recording medium 15, a database (DB) 16, and a display device 17.
[0013] The IF 12 inputs image data used for learning and inference of the prediction device 100. The processor 13 is a computer such as a CPU (Central Processing Unit), and controls the entire prediction device 100 by executing a program prepared in advance. The processor 13 may be a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array). Specifically, the processor 13 executes learning processing and inference processing, which will be described later.
[0014] The memory 14 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 14 stores various programs executed by the processor 13. The memory 14 is also used as a working memory while the processor 13 is executing various processes.
[0015] The recording medium 15 is a non-volatile and non-transient recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the prediction device 100. The recording medium 15 records various programs executed by the processor 13. When the prediction device 100 executes various processes, the programs recorded in the recording medium 15 are loaded into the memory 14 and executed by the processor 13.
[0016] DB16 stores image data input through IF12. Specifically, DB16 stores image data used for learning of prediction device 100. Display device 17 is, for example, a liquid crystal display device or a projector, and displays a prediction result by prediction device 100. In addition to the above, prediction device 100 may include input devices such as a keyboard and a mouse for a user to give instructions or input.
[0017] [Function configuration during learning] 3 is a block diagram showing a functional configuration of the prediction device during learning. The prediction device 100a during learning functionally comprises an image division unit 21, a partial image selection unit 22, a learning unit 23, a prediction unit 24, a label reliability evaluation unit 25, and an integration unit 26. The output of the integration unit 26 is supplied to the display device 17.
[0018] At the time of learning, a learning dataset is prepared. In the following description, one input image and its corresponding label are referred to as learning data, and a collection of learning data for a plurality of input images is referred to as a learning dataset. FIG. 4 shows an example of the learning dataset. In this embodiment, the prediction device 100a detects national flags that include green parts, so that an input image that includes green (positive example input image) and an input image that does not include green (negative example input image) are prepared. Note that in FIG. 4, the green parts are shown by hatching for convenience of illustration. In the learning dataset, a label indicating whether or not the input image includes green is assigned to the entire input image.
[0019] The image division unit 21 divides an input image included in the learning data set into partial images smaller than the input image. As described above, the learning data set includes input images of positive examples and input images of negative examples, so the image division unit 21 divides each of the input images of positive examples and the input images of negative examples into partial images. FIG. 5 shows an example of a method for dividing an input image by the image division unit 21. When there is no particular background knowledge, the image division unit 21 simply divides the input image into a grid with equal intervals, as exemplified in FIG. 5. When there is background knowledge, the image division unit 21 may cut out partial images from the input image with a specific location as the center in accordance with the background knowledge. The image division unit 21 outputs each of the obtained partial images to the partial image selection unit 22.
[0020] The partial image selection unit 22, the learning unit 23, the prediction unit 24, and the label reliability evaluation unit 25 repeat a loop process (hereinafter, referred to as a "learning loop") for generating learning data and learning a prediction model a predetermined number of times.
[0021] Specifically, the partial image selection unit 22 selects partial images to be used for learning a prediction model from the multiple partial images input from the image division unit 21. FIG. 6 shows an example of partial image selection in an initial state. In the initial state, that is, in the first learning loop, the partial image selection unit 22 randomly selects a certain percentage (e.g., 20%) of partial images from the multiple partial images obtained by dividing the positive example input image, as shown in FIG. 6(A), and sets the partial images as learning data (also called "positive data") to which a "positive" label is assigned. Note that, as shown in FIG. 6(A), the partial images included in the positive example input image include partial images of colors other than green, but the partial image selection unit 22 sets all of the labels of the multiple partial images selected from the positive example input image as positive data.
[0022] Moreover, the partial image selection unit 22 selects all of the partial images obtained by dividing the negative example input image as shown in FIG. 6(B) as learning data (also called "negative data") to which a "negative" label has been assigned. That is, since the negative example input image does not contain any green parts, the partial image selection unit 22 sets all of the labels of the partial images selected from the negative example input image as negative data. The partial image selection unit 22 outputs the partial images selected in this way to the learning unit 23 as learning data. The partial images selected by the partial image selection unit 22 are examples of partial images for learning.
[0023] The learning unit 23 learns a prediction model using the partial images (labeled positive / negative) input from the partial image selection unit 22 as learning data. The prediction model is a model that predicts the probability that the input image, which is a national flag, contains the color green. Specifically, a deep learning model such as a CNN (Convolutional Neural Network) can be used as the prediction model. The learning unit 23 outputs the prediction model learned by the first learning to the prediction unit 24.
[0024] The prediction unit 24 uses the trained prediction model to perform prediction for all partial images constituting the input image and calculates predicted values. That is, the prediction unit 24 calculates predicted values for all partial images constituting the input image for all input images included in the training dataset (i.e., including input images of positive examples and input images of negative examples), and outputs the predicted values to the label reliability evaluation unit 25 and the integration unit 26.
[0025] The label reliability evaluation unit 25 judges the reliability of the label assigned to each partial image by the prediction unit 24 using a trained prediction model. Specifically, for each partial image, when the fluctuation or variation of the predicted value calculated by the prediction unit 24 for that partial image in multiple past learning loops is large, the label reliability evaluation unit 25 judges that the reliability of the label is low. For example, when an index such as the standard deviation of the predicted value calculated by the prediction unit 24 in multiple learning loops for a certain partial image is larger than a predetermined value, the label reliability evaluation unit 25 judges that the reliability of the label of that partial image is low. Then, the label reliability evaluation unit 25 excludes partial images whose label reliability is judged to be low, and outputs the remaining partial images to the partial image selection unit 22. Note that in the first learning loop, multiple predicted values have not yet been obtained for each partial image, so the label reliability evaluation unit 25 outputs all partial images output by the prediction unit 24 to the partial image selection unit 22.
[0026] FIG. 7 is a diagram for explaining another example of the process by the label reliability evaluation unit 25. Now, assume that the learning loop is repeated four times for the partial images 71 to 73 included in the input image shown in FIG. 7. Since the labels of the partial image 71 obtained by comparing the predicted value with a predetermined threshold in the four learning loops are all stable as "negative", the label reliability evaluation unit 25 determines that the reliability of the label of the partial image 71 is high. Similarly, the labels of the partial image 72 obtained by comparing the predicted value with a threshold in the four learning loops are all stable as "positive", so the label reliability evaluation unit 25 determines that the reliability of the label of the partial image 72 is high. On the other hand, the labels of the partial image 73 obtained by comparing the predicted value with a threshold in the four learning loops are unstable as "negative → positive → negative → positive", so the label reliability evaluation unit 25 determines that the reliability of the label of the partial image 73 is low, and excludes the partial image 73.
[0027] In this way, at the end of the first learning loop, predicted values are obtained for all partial images included in each input image in the learning data set by the prediction model that has been learned in the first time. Therefore, the partial image selection unit 22 selects partial images to be used as learning data in the next learning loop based on the predicted values of each partial image. Specifically, for partial images constituting positive example input images, the partial image selection unit 22 selects partial images whose predicted values are greater than a predetermined threshold value and sets them as positive data. Note that the threshold value in this case is determined, for example, by using Ohtsu's binarization process, a linear discriminant method, or the like. On the other hand, for partial images constituting negative example input images, the partial image selection unit 22 selects all partial images and sets them as negative data. In this way, the learning data to be used in the next learning loop is selected using the prediction model obtained in the previous learning loop. If the number of negative example partial images selected in this way is much greater than the number of positive example partial images, there is a risk that the model will learn to make predictions biased toward negative examples. To prevent this, the balance between the number of positive example and negative example partial images may be adjusted by using only some of the negative example partial images. In this case, a given percentage may be selected from the top partial images with the largest predicted values, or the partial images may be selected randomly based on a given percentage.
[0028] Figure 8 shows the distribution of predicted values of partial images obtained in the first and fifth learning loops. As can be seen by comparing Figure 8(A) and Figure 8(B), by repeating the learning loop, the predicted values become clearly separated into "0" and "1".
[0029] FIG. 9 shows examples of partial images of positive examples selected in the first and fifth learning loops. As shown in FIG. 9(A), in the first learning loop, the partial image selection unit 22 randomly selects multiple partial images from the positive example input image and sets them as positive data. Thereafter, the learning loop is repeated, and the partial image selection unit 22 selects partial images based on a predicted value calculated using the prediction model. The accuracy of the prediction model improves with each repetition of the learning loop, and in the fifth learning loop, the partial image selection unit 22 selects partial images from the green part of the input image.
[0030] The integrating unit 26 integrates the predicted values for each partial image output by the prediction unit 24 to generate a prediction result for the entire input image. For example, for each input image, the integrating unit 26 sets the average value of the predicted values of all partial images constituting the input image as the predicted value of the entire input image. Then, the integrating unit 26 displays the predicted value of the entire input image together with the input image on the display device 17. Fig. 10(A) shows an example of display on the display device 17. The predicted value for the entire input image is displayed together with the input image.
[0031] Furthermore, the integration unit 26 displays a region of the partial image where the prediction value is equal to or greater than a predetermined threshold as a region important in prediction. In the display example of FIG. 10(A), the region A1 belonging to the green part (hatched part) in the input image is displayed as a region important in prediction, that is, a region that has a large influence on the prediction result. Furthermore, the integration unit 26 may display a region of the partial image where the label reliability evaluation unit 25 has determined that the reliability of the label is low. In the display example of FIG. 10(A), the regions A2 and A3 corresponding to the color boundary in the input image are displayed as regions where the reliability of the label is low.
[0032] The integration unit 26 may display the input image and predicted values as shown in FIG. 10(A) when a predetermined number of learning loops are completed, or may continuously display the input image and predicted values as shown in FIG. 10(A) during repetition of the learning loop.
[0033] In the above configuration, the image division unit 21 is an example of a partial image generation means, the partial image selection unit 22 is an example of a partial image selection means, the learning unit 23 is an example of a learning means, the prediction unit 24 is an example of a prediction means, and the integration unit 26 is an example of an output means.
[0034] [Learning process] 11 is a flowchart of the learning process by the prediction device 100a. This process is realized by the processor 13 shown in FIG. 2 executing a program prepared in advance and operating as each element shown in FIG.
[0035] First, the image division unit 21 receives a learning dataset and divides each input image included in the learning dataset into partial images (step S11). Next, the partial image selection unit 22 selects a partial image for each input image to generate learning data (step S12). Note that in the first learning loop, the partial image selection unit 22 randomly selects a predetermined number of partial images for positive example input images and assigns positive labels, and selects all partial images for negative example input images and assigns negative labels. Alternatively, in order to balance the number of positive and negative partial images, only a given proportion of the partial images may be selected instead of all of them.
[0036] Next, the learning unit 23 learns a prediction model using the learning data generated in step S12, and generates a learned prediction model (step S13). Next, the prediction unit 24 determines whether the learning unit 23 has learned a predetermined number of times (step S14). If the learning unit 23 has not learned a predetermined number of times (step S14: No), the prediction unit 24 uses the learned prediction model generated in step S13 to make predictions for partial images included in all input images in the learning dataset (step S15).
[0037] Next, the label reliability evaluation unit 25 evaluates the reliability of the label of each partial image based on the predicted value by the prediction unit 24, and excludes partial images having low reliability labels (step S16). Note that in the first learning loop, since there is no predicted value in the past learning loop, the label reliability evaluation unit 25 does not exclude partial images. Next, the partial image selection unit 22 selects partial images having predicted values equal to or greater than a predetermined threshold as learning data from the multiple partial images remaining after excluding partial images having low reliability labels, and updates the learning data (step S17). Then, the process returns to step S13.
[0038] In this way, the learning loop of steps S13 to S17 is repeated until the learning unit 23 performs learning a predetermined number of times. Note that in the second and subsequent learning loops, the label reliability evaluation unit 25 excludes partial images having low reliability labels using predicted values obtained in past learning loops, so that learning of the prediction model is repeated based on more appropriate learning data. Then, when the learning unit 23 performs learning a predetermined number of times (step S14: Yes), the learning process ends.
[0039] [Functional configuration during inference] 12 is a block diagram showing a functional configuration of the prediction device at the time of inference. A prediction device 100b at the time of inference functionally comprises an image division unit 21, a prediction unit 24, and an integration unit 26. An output of the integration unit 26 is supplied to a display device 17.
[0040] During inference, image data to be predicted is prepared. In this embodiment, an image of a national flag is prepared as image data and input to the image division unit 21 as an input image. The image division unit 21 divides the input image into a plurality of partial images in the same manner as during learning. For example, if the input image is divided into grids at equal intervals as shown in FIG. 5 during learning, the image division unit 21 also divides the input image into grids at the same intervals during inference. The image division unit 21 outputs the partial images obtained by division to the prediction unit 24.
[0041] The prediction unit 24 performs prediction for the input image using the trained prediction model obtained by the above learning process. Specifically, the prediction unit 24 performs prediction for each partial image obtained by the image division unit 21 using the trained prediction model, and outputs the predicted value to the integration unit 26.
[0042] The integration unit 26 integrates the predicted values calculated by the prediction unit 24 for each partial image to calculate a predicted value for the entire input image, and outputs the predicted value to the display device 17. The integration unit 26 may output the calculated predicted value to an external device. This allows the probability that the input image, which is a national flag, contains green to be obtained. Furthermore, the integration unit 26 extracts, in the prediction for the input image, an area of the partial image whose predicted value is equal to or greater than a predetermined reference value as an important area for the prediction, and outputs the extracted area to the display device 17. The display device 17 displays the predicted value and important area A1 on the input image, as exemplified in FIG. 10(B). In this manner, the prediction result for the input image is output.
[0043] The prediction model used by the prediction unit 24 is basically the prediction model at the time when a predetermined number of learning loops are completed, that is, the final model. However, since the accuracy of the final model is not necessarily the highest, the prediction unit 24 may use the prediction model obtained in each learning loop that has the smallest prediction error. Specifically, the prediction result by the prediction model obtained at the end of each learning loop may be compared with the learning data set to calculate the prediction error, and the prediction model with the smallest prediction error may be adopted.
[0044] In the above example, one of the multiple prediction models obtained by multiple learning loops is used for prediction in the prediction unit 24, but instead, some of the multiple prediction models obtained may be used in combination. Specifically, the prediction unit 24 may perform prediction for an input image using multiple prediction models obtained by multiple learning loops, and may output a final prediction result by weighting and adding the multiple prediction results obtained. In this case, a larger weight may be given to the output of a prediction model with high accuracy.
[0045] [Inference processing] 13 is a flowchart of an inference process performed by the prediction device 100b. This process is realized by the processor 13 shown in FIG. 2 executing a program prepared in advance and operating as each element shown in FIG.
[0046] First, the image division unit 21 divides the input image into partial images (step S21). Next, the prediction unit 24 performs prediction for each partial image using a trained prediction model obtained by the learning process, and outputs a predicted value (step S22). Next, the integration unit 26 integrates the predicted values for each partial image, and calculates a prediction result for the entire input image (step S23). Then, the display device 17 displays the prediction result, for example, as illustrated in FIG. 10(B) (step S24). Then, the inference process ends.
[0047] [Example of application] In the above example, the prediction device of this embodiment is used to detect green areas in national flags, but as another example, an application of the prediction device of this embodiment to predicting the effects of medication (drug efficacy or success, hereinafter referred to as "drug efficacy") will be described. In the medical field, a pathological tissue image is input, and drug efficacy is predicted based on the image. For example, there is a method of predicting drug efficacy based on the staining rate of an immune image, based on an image in which cells including pathological tissue are stained. In this case, there are problems such as variation in the judgment of the staining rate depending on the examiner, and the fact that it is not clear which part of the image is the characteristic part reflecting the drug efficacy.
[0048] Therefore, using the prediction device of this embodiment, a pathological tissue image is input, and a prediction model obtained by learning is used to predict drug efficacy, output a prediction score for drug efficacy, and display areas that are important for determining drug efficacy.
[0049] Specifically, during learning, learning data is prepared in which a label indicating the presence or absence of a drug effect is added to the pathological tissue image, and the above-mentioned learning process is executed to learn a prediction model of drug efficacy. In this case, in the method of the present embodiment, since it is sufficient to label the entire pathological tissue image, labeling is possible even if it is not clear which part of the pathological tissue image affects the drug efficacy. Then, during inference, the pathological tissue image to be predicted is input, and the drug efficacy score can be predicted using the trained prediction model. Furthermore, the area of the partial image that showed a high score in the inference process can be extracted as an important area that has a large effect on the drug efficacy, and displayed on a display device.
[0050] When this embodiment is applied to predicting drug efficacy, the image segmentation unit 21 during learning and inference can utilize background knowledge, such as that information around a cell nucleus is particularly important for diagnosis, and generate a partial image centered on the position of the cell nucleus in the input image.
[0051] <Second embodiment> 14 is a block diagram showing the functional configuration of a learning device of the second embodiment. The learning device 50 includes a partial image generating means 51, a partial image selecting means 52, a learning means 53, and a predicting means .
[0052] 15 is a flowchart of processing by the learning device 50 of the second embodiment. First, the partial image generating means 51 generates a partial image smaller than the input image from the input image (step S31). Next, the partial image selecting means 52 selects a plurality of learning partial images to be used as learning data from all the generated partial images (step S32).
[0053] Next, the learning means 53 learns a prediction model that predicts the probability that a predetermined feature is included in the selected learning partial image (step S33). Next, the prediction means 54 uses the learned prediction model to make predictions for all partial images (step S34). Next, the partial image selection means 52 selects learning partial images to be used in the next learning based on the predicted values for all partial images (step S35). In this way, the learning means 53 repeats learning of the prediction model while updating the learning partial images.
[0054] According to the learning device 50 of the second embodiment, it is possible to perform highly accurate predictions even when a label is assigned only to the entire image and it is unclear which part of the image is decisive for the prediction.
[0055] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.
[0056] (Appendix 1) a partial image generating means for generating, from an input image, a partial image smaller than the input image; a partial image selection means for selecting a plurality of learning partial images to be used as learning data from all the generated partial images; A learning means for learning a prediction model that predicts the probability that a predetermined feature is included in a selected learning partial image; A prediction means for making predictions for all the partial images using a trained prediction model; Equipped with The partial image selection means is a learning device that selects a plurality of learning partial images to be used in the next learning based on the predicted values for all of the partial images.
[0057] (Appendix 2) The learning device according to claim 1, wherein the partial image selection means selects, in second and subsequent learning rounds, partial images whose predicted values are equal to or greater than a predetermined threshold as the learning partial images.
[0058] (Appendix 3) The learning device described in Appendix 2, wherein the partial image selection means determines the reliability of the predicted value of each partial image based on predicted values obtained by multiple predictions, and excludes partial images whose reliability is lower than a predetermined reference value from the learning partial images.
[0059] (Appendix 4) The learning device according to claim 3, wherein the partial image selection means determines the reliability of each of the partial images based on the variance in the predicted value of each of the partial images.
[0060] (Appendix 5) The learning device according to any one of Appendices 2 to 4, wherein, in a first learning round, the partial image selection means randomly selects the learning partial image from all of the partial images for a positive example input image that includes the specified feature, and selects all of the partial images as the learning partial image for a negative example input image that does not include the specified feature.
[0061] (Appendix 6) The learning device according to claim 5, wherein the learning means uses, in the first learning, a label that has been previously assigned to the entire input image as a label for each learning partial image included in the input image.
[0062] (Appendix 7) a partial image generating means for generating, from an input image, a partial image smaller than the input image; A prediction means for predicting a probability that a predetermined feature is included in each of all generated partial images by using a prediction model trained by the learning device according to any one of Supplementary Notes 1 to 6; an output means for integrating the prediction results for all the partial images and outputting a prediction score indicating the probability that the input image contains the predetermined feature; A prediction device comprising:
[0063] (Appendix 8) The prediction device according to claim 7, wherein the output means determines an area in the input image that includes the predetermined feature based on the prediction results for all of the partial images, and displays the area on a display device.
[0064] (Appendix 9) A partial image smaller than the input image is generated from the input image; Selecting a plurality of learning partial images to be used as learning data from all the generated partial images; training a predictive model that predicts the probability that a predetermined feature is included in the selected training partial image; Using the trained prediction model, predictions are made for all of the partial images; A learning method for selecting a plurality of learning partial images to be used in the next learning based on the predicted values for all of the partial images.
[0065] (Appendix 10) A partial image smaller than the input image is generated from the input image; Selecting a plurality of learning partial images to be used as learning data from all the generated partial images; training a predictive model that predicts the probability that a predetermined feature is included in the selected training partial image; Using the trained prediction model, predictions are made for all of the partial images; A recording medium having recorded thereon a program for causing a computer to execute a process of selecting a plurality of partial images for learning to be used in the next learning process based on the predicted values for all of the partial images.
[0066] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-mentioned embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. [Explanation of symbols]
[0067] 13 Processors 17 Display device 21 Image division section 22 Partial Image Selection 23 Learning Department 24 Prediction Department 25 Label Reliability Evaluation Department 26 Integration Department 100, 100a, 100b Prediction device
Claims
1. a partial image generating means for generating, from an input image, a partial image smaller than the input image; a partial image selection means for selecting a plurality of learning partial images to be used as learning data from all the generated partial images; A learning means for learning a prediction model that predicts the probability that a predetermined feature is included in a selected learning partial image; A prediction means for making predictions for all the partial images using a trained prediction model; a label reliability evaluation means for determining the reliability of each partial image based on the variance of the predicted value of each partial image obtained by multiple predictions in the second or subsequent learning, and outputting the partial image having the reliability equal to or greater than a predetermined reference value to the partial image selection means; Equipped with The partial image selection means is a learning device that selects a plurality of learning partial images to be used in the next learning based on the predicted values for all of the partial images.
2. 2. The learning device according to claim 1, wherein the partial image selection means selects, in second and subsequent learning rounds, partial images whose predicted values are equal to or greater than a predetermined threshold as the learning partial images.
3. The learning device according to claim 1 or 2, wherein the partial image selection means, in a first learning round, randomly selects the learning partial image from all of the partial images for a positive example input image that includes the specified feature, and selects all of the partial images as the learning partial image for a negative example input image that does not include the specified feature.
4. 4. The learning device according to claim 3, wherein the learning means uses, in the first learning, a label previously assigned to the entire input image as a label for each partial learning image included in the input image.
5. a partial image generating means for generating, from an input image, a partial image smaller than the input image; A prediction means for predicting a probability that a predetermined feature is included in each of all generated partial images by using a prediction model trained by the learning device according to any one of claims 1 to 4; an output means for integrating the prediction results for all the partial images and outputting a prediction score indicating the probability that the input image contains the predetermined feature; A prediction device comprising:
6. 6. The prediction device according to claim 5, wherein the output means determines an area in the input image that includes the predetermined feature based on the prediction results for all of the partial images, and displays the area on a display device.
7. 1. A computer-implemented method of learning, comprising: A partial image smaller than the input image is generated from the input image; Selecting a plurality of learning partial images to be used as learning data from all the generated partial images; training a predictive model that predicts the probability that a predetermined feature is included in the selected training partial image; Using the trained prediction model, predictions are made for all of the partial images; selecting a plurality of learning partial images to be used in the next learning based on the predicted values for all the partial images; In the second or subsequent learning, a reliability of each partial image is determined based on a variance in a predicted value of each partial image obtained by multiple predictions; A learning method in which the plurality of learning partial images to be used in the next learning are selected from the partial images whose reliability is equal to or greater than a predetermined reference value.
8. A partial image smaller than the input image is generated from the input image; Selecting a plurality of learning partial images to be used as learning data from all the generated partial images; training a predictive model that predicts the probability that a predetermined feature is included in the selected training partial image; Using the trained prediction model, predictions are made for all of the partial images; selecting a plurality of learning partial images to be used in the next learning based on the predicted values for all the partial images; In the second or subsequent learning, a reliability of each partial image is determined based on a variance in a predicted value of each partial image obtained by multiple predictions; A program that causes a computer to execute a process in which the multiple learning partial images to be used in the next learning are selected from the partial images whose reliability is equal to or greater than a predetermined reference value.
Citation Information
Patent Citations
Neural network learning method using auto encoder and multiple instance learning and computing system performing the same
KR102163519B1
