Learning support device

Abstract

The objective of this invention is to learn separable feature representations while focusing on small detection regions, and to support the creation of image classification models with limited teacher information. The learning support device of this invention creates a set of images of detection regions to check whether an object has anomalies, and updates the feature extraction model in a way that increases the similarity between the same aforementioned detection regions, while simultaneously updating the aforementioned feature extraction model in a way that decreases the similarity between two image regions containing dissimilar previously described detection regions (see Figure 1).

Description

Technical Field

[0001] This invention relates to a technique that supports learning a classification model through machine learning, the classification model being used to classify anomalies of an object based on an image of the object. Prior Technology

[0002] In machine learning, it is necessary to assign information (labels or annotations) to the data being learned. For example, to create an image classification model using deep learning (DL), a large amount of teacher information is needed, and annotation costs are high.

[0003] Patent document 1 describes a tagging technique. The same document describes "using self-supervised learning performance to train a standard DL-based object detection system with a small amount of labeled data" (see paragraph 0009 of the same document). [Previous Technical Documents] [Patent Literature]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-126130 Summary of the Invention

[0005] [The problem the invention aims to solve] In prior art, such as Patent Document 1, there is no assumption that the object detection system is trained on tiny objects. Patent Document 1, in its self-supervised learning, applies data augmentation involving random cropping (randomly cutting regions from the original image). However, this method learns the representation of the entire image due to the random cropping, making it difficult to focus on tiny objects occupying small areas in the image; instead, it focuses on the dominant objects in the image. Therefore, the learning of the representation of tiny objects fails, and annotation costs increase.

[0006] The present invention is proposed in view of the above-mentioned problems, and aims to learn separable feature representations while focusing on small detection regions, and to support the creation of image classification models with a small amount of teacher information. [Methods for solving problems]

[0007] The learning support device of the present invention creates a set of images of detection regions for checking whether an object has anomalies, and updates the feature extraction model in a way that increases the similarity between the same detection regions, and updates the feature extraction model in a way that decreases the similarity between two image regions containing different detection regions. [Invention Effects]

[0008] The learning support device of the present invention can learn separable feature representations while focusing on small detection areas, and create an image classification model with a small amount of teacher information. Other issues, configurations, advantages, etc., beyond those described above will be explained through the following description of embodiments. Simple Explanation of the Diagram

[0009] [Figure 1] Figure 1 is a structural diagram of the learning support device 100. [Figure 2] Figure 2 is a diagram of the internal structure of the image generation unit 103 in the detection area. [Figure 3] Figure 3 shows a specific example of the detection region image set 1037 and the similar normal region image set 1038. [Figure 4] Figure 4 is a diagram of the internal structure of the image similarity calculation unit 104. [Figure 5] Figure 5 is a diagram of the internal structure of the image distribution display unit 105. [Figure 6] Figure 6 shows an example of the user interface used by the similarity determination unit 1051 to prompt the user with the scatter plot. [Figure 7] Figure 7 shows an example of a user interface in which the annotation candidate prompting unit 1052 prompts the user for annotation candidates. [Figure 8] Figure 8 is a flowchart illustrating the process by which the detection region image generation unit 103 generates a detection region image set 1037 and a similar normal region image set 1038 by inputting learning image data 101 and detection region coordinate data 102. [Figure 9] Figure 9 is a flowchart illustrating the process of image similarity calculation unit 104 determining image similarity by inputting detection region image set 1037 and similar normal region image set 1038, and image distribution display unit 105 saving annotation image set 1053. Implementation

[0010] The embodiments of the present invention described below are based on 2D data (e.g., image data), but the embodiments of the present invention can also be applied to 3D data. That is, the word "image" includes both 2D images (e.g., represented by pixels) and 3D scenes (e.g., represented by voxels or point clouds).

[0011] Figure 1 is a structural diagram of the learning support device 100 according to an embodiment of the present invention. In Embodiment 1, the case of inspecting semiconductors using images taken by a scanning electron microscope or the like will be described. During the inspection process, minute defects are classified using anomaly candidates detected by anomaly detection.

[0012] The learning support device 100 receives the following as inputs: (a) an inspection image of the product, i.e., learning image data 101; and (b) detection area coordinate data 102, which represents the location information of the detection area detected as an anomaly candidate in the learning image data 101 through anomaly detection. The learning support device 100 further includes: a detection area image generation unit 103, which generates partial images from the learning image data 101 based on the detection areas; an image similarity calculation unit 104, which calculates the similarity between the vectors of the partial images; and an image distribution display unit 105, which displays the similarity results.

[0013] Figure 2 is an internal configuration diagram of the detection region image generation unit 103. The detection region image generation unit 103 includes: a detection region image generation unit 1031, which generates partial images from the learning image data 101 and the detection region coordinate data 102; a detection region specification parameter 1032, which describes the conditions when the detection region image generation unit 1031 generates partial images; a similar normal region image generation unit 1033, which explores normal regions similar to the detection region from the learning image data 101 and the detection region coordinate data 102, and generates partial images thereon; a similar normal region specification parameter 1034, which describes the conditions when the similar normal region image generation unit 1033 generates partial images thereon; a preprocessing unit 1035, which generates a detection region image set 1037 and a similar normal region image set 1038 from partial images of the detection region by performing preprocessing such as adding noise to the generated partial images; and an image enhancement parameter 1036, which describes the parameters of the preprocessing.

[0014] The detection region image generation unit 1031 selects a detection region based on the detection region coordinate data 102, and obtains a plurality of partial images from the learning image data 101 by cropping in a manner that includes the selected detection region. The method by which the detection region image generation unit 1031 obtains partial images is not limited to this.

[0015] The detection region specification parameter 1032 is a parameter used by the detection region image generation unit 1031 to specify the cropping region when generating a partial image. For example, parameters such as the image size of the partial image, the maximum area and minimum area of ​​the region shared by multiple partial images may be described. Other parameters used to specify the partial image may also be described.

[0016] The similar normal region image generation unit 1033 explores and detects normal regions in the learning image data 101 that are similar in features to the partial images generated by the region image generation unit 1031 and for which no abnormalities are detected. Multiple partial images are obtained by cropping from the normal regions obtained through exploration. Template matching is suitable as an exploration method. Other exploration methods can also be used. For example, if there is a reference image such as the design data, it can also be used as the similar normal region image.

[0017] The parameter 1034 for specifying the similar normal region is a parameter that specifies the cropping region when the similar normal region image generation unit 1033 generates a partial image. For example, parameters such as the image size of the partial image and the similarity threshold between the detection region and the normal region may be described. Other parameters used to specify the partial image may also be described.

[0018] The preprocessing unit 1305 performs preprocessing on the generated partial image, such as adding noise. Examples of preprocessing include color dithering, grayscale conversion, Gaussian blur, horizontal flipping, vertical flipping, random cropping, and resizing. Other preprocessing can also be performed.

[0019] Image enhancement parameters 1036 control the probability of each preprocessing step performed by the preprocessing unit 1305 by specifying the application probability. For example, by setting the application probability of Gaussian blur to 50% and the application probability of vertical flip to 50%, either Gaussian blur or vertical flip can be applied with the same probability for some images. Alternatively, the application probability can be set to 0% for all steps, thereby not performing any preprocessing.

[0020] The detection region image set 1037 is a collection of images processed by the preprocessing unit 1305 on a portion of the images generated by the detection region image generation unit 1031.

[0021] The similar normal region image set 1038 is a collection of images preprocessed by the preprocessing unit 1305 on some images created by the similar normal region image creation unit 1033.

[0022] Figure 3 illustrates a specific example of the detection region image set 1037 and the similar normal region image set 1038. Detection region 301 is the detection region obtained from the detection region coordinate data 102. Partial image 302 is an example of a detection region image created by the detection region image generation unit 1031 by selecting detection region 301. Partial image 303 is another detection region image that includes detection region 301. Partial image 304 is an example of a similar normal region image created by the similar normal region image generation unit 1033 by selecting partial image 302 as the detection region image. Partial image 305 is an example of a similar normal region image created by the similar normal region image generation unit 1033 by selecting partial image 303 as the detection region image.

[0023] Figure 4 shows the internal structure of the image similarity calculation unit 104. The image similarity calculation unit 104 includes the following: a feature extraction unit 1041, which extracts features from the image data of the detection region image set 1037 and the similar normal region image set 1038 to create a detection region feature vector set 1042 and a similar normal region feature vector set 1043; a vector comparison unit 1044, which calculates the similarity between the vectors of the detection region feature vector set 1042 and the similar normal region feature vector set 1043; and a model update unit 1045, which calculates model update parameters 1046 based on the vector similarity.

[0024] The feature extraction unit 1041 extracts features (feature vectors) from the images of the detection region image set 1037 and the similar normal region image set 1038, respectively. As a method for feature extraction, deep learning methods such as convolutional neural networks and multilayer perceptrons, as well as other machine learning methods, are applicable. Other methods may also be used.

[0025] The feature vector set 1042 of the detection region is the feature vector set extracted by the feature extraction unit 1041 from the image set 1037 of the detection region.

[0026] The feature vector set 1043 of similar normal regions is the feature vector set extracted by the feature extraction unit 1041 from the image set 1038 of similar normal regions.

[0027] The vector comparison unit 1044 calculates the following similarities: a first similarity, i.e., the similarity between vector 1 derived from detection region 1 and vector 2 derived from detection region 1 (detection region 1 is the same as the detection region containing feature vectors in the feature vector set 1042 for extracted detection regions); a second similarity, i.e., the similarity between vector 1 or vector 2 and vectors derived from detection region 2 (a detection region different from detection region 1); a third similarity, i.e., the similarity between vector 1 and vectors from similar normal regions of detection region 1 among the feature vectors contained in the feature vector set 1043 for similar normal regions; and a fourth similarity, i.e., the similarity between vectors derived from similar normal regions of detection region 1 and vectors derived from similar normal regions of detection region 2. Methods for calculating similarity include, for example, cosine similarity and Euclidean distance. Other methods may also be used.

[0028] The model update unit 1045 calculates the update amount of the model parameters, namely the model update parameters 1046, based on the higher first and fourth similarity scores calculated by the vector comparison unit 1044 and the lower second and third similarity scores. The model update parameters 1046 are the parameters of the feature extraction model (such as the convolutional neural network in the example above) used by the feature extraction unit 1041 when extracting image features. Backpropagation is a suitable method for updating the model parameters, but it is not limited to this method.

[0029] By updating the model parameters based on the increased similarity (first similarity) between vectors from images containing the same detection region, the model can extract features focused on the detection region. By updating the model parameters based on the decreased similarity (second similarity) between vectors from images containing different detection regions, the model can extract features focused on the differences within the detection regions. By updating the model parameters based on the decreased similarity (third similarity) between an image containing a detection region and an image containing similar normal regions, the model can focus on the detection region rather than surrounding features. By updating the model parameters based on the increased similarity (fourth similarity) between images containing similar normal regions from different detection regions, the model can be prevented from focusing on surrounding features.

[0030] Figure 5 is an internal configuration diagram of the image distribution display unit 105. The image distribution display unit 105 includes: a similarity determination unit 1051, which determines the learning status of the model by calculating the similarity using the vector comparison unit 1044; an annotation candidate suggestion unit 1052, which suggests annotation candidates to the user based on the feature values ​​determined by the similarity determination unit 1051; and an annotation image set 1053, which manages the data that has been annotated.

[0031] The similarity determination unit 1051 receives vectors from the detection region and similar normal regions, respectively, and first, second, third, and fourth similarities from the vector comparison unit 1044, and visualizes them as a scatter plot. In the scatter plot, the model's learning quality can be determined by confirming whether the feature points are distributed in a manner where the first and fourth similarities are higher, and the second and third similarities are lower. If the determination is negative, the first, second, third, and fourth similarities are passed to the model update unit 1045, and the model is updated. Methods for visualizing feature quantities as scatter plots include, for example, using algorithms such as UMAP (Uniform Manifold Approximation and Projection) and t-SNE (t-random neighbor embedding) to reduce the feature vectors to two dimensions and plot each feature quantity. Other methods can also be used.

[0032] If the similarity assessment unit 1051 determines that the model's learning is good, the annotation candidate suggestion unit 1052 selects annotation candidates from the vectors derived from the detection region and suggests them to the user. For example, a selection method could be to suggest data that are far from normal regions in order. Other methods can also be used for selection.

[0033] The annotation image set 1053 is a dataset of annotation images that is managed when the user annotates the annotation candidates suggested by the annotation candidate prompting section 1052.

[0034] Figure 6 shows an example of the user interface of the similarity determination unit 1051 for providing a scatter plot to the user. When button 601 is pressed, the similarity determination unit 1051 retrieves the image to be displayed. When button 602 is pressed, the similarity determination unit 1051 changes the image. The image display unit 603 displays a portion of the images from the detection region image set 1037 and the similar normal region image set 1038. The scatter plot 604 inputs the images from the image display unit 603 to the feature extraction unit 1041 and displays a scatter plot of the similarity calculated by the vector comparison unit 1044. When button 605 is pressed, the similarity determination unit 1051 determines that the learning status of the currently displayed image is good. When button 606 is pressed, the similarity determination unit 1051 determines that the learning status of the currently displayed image is bad.

[0035] Examples of the images displayed by the image display unit 603 are as follows. Images (1) and (2) are two detection region images containing the same detection region. Image (3) is a similar normal region image corresponding to image (1). Image (5) is a detection region image containing a different detection region than images (1) and (2). Image (4) is a similar normal region image corresponding to image (5).

[0036] The arrow marks shown in scatter plot 604 correspond to the following similarities. The arrow marks between images (1) and (2) represent the first similarity. The arrow marks between images (2) and (5) represent the second similarity. The arrow marks between images (1) and (3) represent the third similarity. The arrow marks between images (3) and (4) represent the fourth similarity. If the first and fourth similarities are high (i.e., the distance between the features represented by the black circles is relatively close), and the second and third similarities are low (i.e., the distance between the features represented by the black circles is relatively far), the user can judge the learning result as good.

[0037] Figure 7 illustrates an example of the user interface of the annotation candidate prompting unit 1052, which prompts the user for annotation candidates. When button 701 is pressed, the annotation candidate prompting unit 1052 changes the selection criterion for the annotation candidate. When button 702 is pressed, the annotation candidate prompting unit 1052 changes the annotation candidate to the next candidate. The image display unit 703 displays the image of the detection region image set 1037. The scatter plot 704 displays a scatter plot that distributes the feature vectors of a portion of the image on a two-dimensional graph. When text is entered or a label is selected for the drop-down list 705, the annotation candidate prompting unit 1052 determines the label of the image displayed by the image display unit 703. When button 706 is pressed, the annotation candidate prompting unit 1052 saves the image of the image display unit 703 and the label of the drop-down list 705 in the annotation image set 1053.

[0038] The scatter plot 704 is a graph that maps the feature vectors of a portion of the image onto a two-dimensional space. Depending on the type of defect, the feature vectors cluster in a certain region. Therefore, when annotating a specific type of defect, it is sufficient to select any portion of the image within that region (the black circle in the scatter plot 704). The selected portion of the image is displayed in the image display unit 703.

[0039] Figure 8 is a flowchart illustrating the process by which the detection region image generation unit 103 generates a detection region image set 1037 and a similar normal region image set 1038 by inputting learning image data 101 and detection region coordinate data 102.

[0040] Step S801: The user takes an image of the object to be detected as abnormal and saves it as image data 101 for learning purposes.

[0041] Step S802: The user specifically records the coordinate data 102 of the detection area where an anomaly was detected on the sample.

[0042] Step S803: The detection region image generation unit 103 selects a detection region from the detection region coordinate data 102 and crops a plurality of images from the learning image data 101 in a manner that includes the selected detection region. Detection regions are selected sequentially, and a plurality of cropped images derived from different detection regions are generated. As illustrated using FIG2, the cropped region is determined by the detection region specification parameter 1032. The detection region specification parameter 1032 is a parameter that determines the image size, the size of the area shared by different images obtained from the same detection region, etc.

[0043] Step S804: The detection region image generation unit 103 identifies a normal region whose surrounding features are similar to those of the image generated in step S803 and no abnormalities are detected, and crops the same number of images. The normal region does not need to be explored from the learning image data 101. As illustrated using Figure 2, the cropped region is determined by the similar normal region specification parameter 1034. The similar normal region specification parameter 1034 determines parameters such as image size and a threshold for the similarity of surrounding features between the detection region and the normal region.

[0044] Step S805: The detection region image generation unit 103 generates a detection region image set 1037 and a similar normal region image set 1038 by adding preprocessing such as color dithering, grayscale conversion, and Gaussian blur to the images generated in steps S803 and 804. As illustrated in FIG2, the preprocessing is determined by the image enhancement parameter 1036. The image enhancement parameter 1036 is a parameter that determines the probability of applying each image enhancement.

[0045] Figure 9 is a flowchart illustrating the process of image similarity calculation unit 104 determining image similarity and image distribution display unit 105 saving annotation image set 1053 by inputting detection region image set 1037 and similar normal region image set 1038.

[0046] Step S901: By inputting the detection region image set 1037 and the similar normal region image set 1038, the feature extraction unit 1041 outputs the detection region feature vector set 1042 and the similar normal region feature vector set 1043. The feature extraction unit 1041 does not necessarily need to be a machine learning model.

[0047] Step S902: The vector comparison unit 1044 extracts vectors derived from the same detection region and vectors derived from different detection regions from the detection region feature vector set 1042. Furthermore, it extracts vectors of similar normal regions derived from the extracted detection regions from the similar normal region feature vector set 1043. Using these vectors, the vector comparison unit 1044 calculates the first, second, third, and fourth similarities illustrated in Figure 3.

[0048] In step S903, it is confirmed whether the number of parameter updates of the feature extraction unit 1041 has reached the number preset by the user. The number of parameter updates represents the learning progress. If it can be regarded as an indicator of learning progress, a value other than the number of parameter updates of the feature extraction unit 1041 can be used. If the number of parameter updates reaches the predetermined number, proceed to S904; otherwise, skip to S906.

[0049] Step S904: The image distribution display unit 105 distributes the vectors used by the vector comparison unit 1044 as a scatter plot. The user uses this scatter plot to confirm the learning status. If the method allows visualization of the similarity calculated by the vector comparison unit 1044, a scatter plot is not required.

[0050] Step S905: The user confirms the learning status and whether it is good. If it is not good, proceed to S906; if it is good, skip to S908.

[0051] Step S906: The model update unit 1045 calculates the gradient by increasing the first similarity and the fourth similarity, and decreasing the second similarity and the third similarity, and calculates the model update parameters 1046.

[0052] Step S907: The model update unit 1045 updates the feature extraction model using the model update parameters 1046 calculated in S906. Return to S901.

[0053] Step S908: The annotation candidate suggestion unit 1052 calculates an anomaly score from the similarity between vectors. Based on the anomaly score, the annotation candidate suggestion unit 1052 suggests annotation candidate regions to the user. The anomaly score is a parameter representing the severity of the defect. The anomaly score can be obtained from the defect inspection results, or the third similarity itself can be considered as the severity of the defect and used as the anomaly score.

[0054] Step S909: The user annotates the candidate annotation regions. The annotation candidate prompting unit 1052 saves the results to the annotation image set 1053.

[0055] <Regarding variations of the present invention> This invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above have been detailed to illustrate the invention in an easily understandable manner, and are not necessarily limited to having all the described configurations. Furthermore, a portion of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of one embodiment can also be added to the configuration of another embodiment. Moreover, for a portion of the configuration of each embodiment, other configurations can be added, removed, or replaced.

[0056] All of the aforementioned components, functions, processing units, and processing methods, some or all of which can be implemented in hardware using, for example, integrated circuit design. Furthermore, the aforementioned components and functions can also be implemented in software by having a processor translate and execute the programs that implement their respective functions. The programs, tables, files, and other information that implement each function can be stored in memory, hard drives, SSDs (Solid State Drives), or recording media such as IC cards, SD cards, and DVDs.

[0057] In the above embodiments, the image distribution display unit 105 can determine whether the learning status is good based on user input on the user interface (S905), or it can determine whether it is good or bad based on the similarity between the feature vectors of the detection region image and the similar normal region image (regardless of user input). Furthermore, the user can correct the learning status determined by the image distribution display unit 105 using buttons 605 or 606.

[0058] In the above embodiments, an example of using both the detection region image and the similar normal region image has been described, but it is also possible to use only the detection region image. In this case, the similar normal region image generation unit 1033 is not required.

[0059] In the above embodiments, the detection area image generation unit 103, the image similarity calculation unit 104, and the image distribution display unit 105 (and their respective functional units) can be constructed by hardware such as circuit devices that implement these functions, or by executing software that implements these functions using a computing device such as a CPU (Central Processing Unit).

[0060] 100: Learning Support Device 101: Learning to use image data 102: Detection area coordinate data 103: Image Creation Unit for Detection Region 104: Image Similarity Calculation Unit 105: Image Distribution Display Unit

Claims

1. A learning support device that supports learning a classification model using machine learning, the classification model being used to classify anomalies of an object based on an image of the object, the learning support device comprising: a detection region image generation unit that generates a set of detection region images containing detection regions for checking the presence or absence of the anomalies from learning image data used for the machine learning; an image similarity calculation unit that calculates the similarity between the detection region images contained in the set of detection region images; and an image distribution display unit that displays the distribution of the similarity and suggests candidate annotation regions as input to the detection regions for annotation in the machine learning; the image similarity calculation unit updates the feature extraction model used to calculate the similarity in such a way that the similarity between the same detection region increases, and updates the feature extraction model in such a way that the similarity between two image regions containing different detection regions decreases.

2. The learning support device as described in claim 1, wherein, The aforementioned detection region image generation unit obtains detection region coordinate data that records the coordinates of the aforementioned detection region in the aforementioned learning image data; the aforementioned detection region image generation unit records the aforementioned coordinates based on the aforementioned detection region coordinate data and generates the aforementioned detection region image set.

3. The learning support device as described in claim 1, wherein, The aforementioned detection area image creation unit obtains detection area specification parameters that define the size of the aforementioned detection area, and creates the aforementioned detection area image set according to the aforementioned detection area specification parameters; the aforementioned detection area specification parameters define the image size of the aforementioned detection area, and define the size of the area shared by different aforementioned detection area images containing the same aforementioned detection area.

4. The learning support device as described in claim 1, wherein, The aforementioned detection region image generation unit specifically identifies normal regions in the aforementioned learning image data whose surrounding features are similar to the aforementioned detection region and which do not detect the aforementioned anomalies; the aforementioned detection region image generation unit generates a set of similar normal region images by extracting a portion of the images containing the aforementioned normal regions from the aforementioned learning image data; the aforementioned image similarity calculation unit calculates the similarity between the aforementioned detection region images in the aforementioned detection region image set and the similar normal region images in the aforementioned similar normal region image set.

5. The learning support device as described in claim 4, wherein, The aforementioned detection area image generation unit obtains a similar normal area specification parameter that specifies the size of the aforementioned normal area, and generates the aforementioned similar normal area image set according to the aforementioned similar normal area specification parameter; the aforementioned similar normal area specification parameter specifies the size of the aforementioned normal area, and specifies a threshold for determining whether its surrounding features are similar to the aforementioned detection area.

6. The learning support device as described in claim 4, wherein, The aforementioned detection region image generation unit generates the aforementioned detection region image set and the aforementioned similar normal region image set by applying at least one of the following preprocessing methods to the aforementioned detection region image and the aforementioned similar normal region image: color dithering, grayscale conversion, Gaussian blur, horizontal flip, vertical flip, random cropping, and resizing.

7. The learning support device as described in claim 6, wherein, The aforementioned detection area image generation unit obtains image enhancement parameters that specify the probability of applying the aforementioned preprocessing; the aforementioned detection area image generation unit applies any of the aforementioned preprocessing steps according to the probability specified by the aforementioned image enhancement parameters.

8. The learning support device as described in claim 4, wherein, The aforementioned image similarity calculation unit extracts feature vectors for the aforementioned detection region image and the aforementioned similar normal region image respectively; the aforementioned image similarity calculation unit calculates the similarity between the aforementioned detection region image and the aforementioned similar normal region image by calculating the similarity between the aforementioned feature vectors.

9. The learning support device as described in claim 8, wherein, The aforementioned image similarity calculation unit selects any one of the aforementioned detection regions from the extracted feature vectors as the first detection region. The aforementioned image similarity calculation unit extracts two of the aforementioned feature vectors from the image containing the aforementioned first detection region as the first feature vector and the second feature vector. The aforementioned image similarity calculation unit calculates the first similarity between the aforementioned first feature vector and the aforementioned second feature vector. The aforementioned image similarity calculation unit updates the aforementioned feature extraction model in such a way that the aforementioned first similarity increases.

10. The learning support device as described in claim 8, wherein, The aforementioned image similarity calculation unit extracts any one of the aforementioned detection regions from the aforementioned feature vector as the first detection region, and extracts the aforementioned feature vector from the image containing the aforementioned first detection region as the third feature vector; the aforementioned image similarity calculation unit extracts any one of the aforementioned detection regions that is different from the aforementioned first detection region from the aforementioned feature vector as the second detection region, and extracts the aforementioned feature vector from the image containing the aforementioned second detection region as the fourth feature vector; the aforementioned image similarity calculation unit calculates the second similarity between the aforementioned third feature vector and the aforementioned fourth feature vector; the aforementioned image similarity calculation unit updates the aforementioned feature extraction model in such a way that the aforementioned second similarity decreases.

11. The learning support device as described in claim 8, wherein, The aforementioned image similarity calculation unit selects any one of the aforementioned detection regions from the aforementioned feature vectors as the first detection region, and extracts the aforementioned feature vectors from the image containing the aforementioned first detection region as the fifth feature vector; the aforementioned image similarity calculation unit extracts the aforementioned feature vectors from the image containing the aforementioned normal region corresponding to the aforementioned first detection region as the sixth feature vector; the aforementioned image similarity calculation unit calculates the third similarity between the aforementioned fifth feature vector and the aforementioned sixth feature vector; the aforementioned image similarity calculation unit updates the aforementioned feature extraction model in such a way that the aforementioned third similarity decreases.

12. The learning support device as described in claim 8, wherein, The aforementioned image similarity calculation unit selects any one of the aforementioned detection regions from which the aforementioned feature vectors have been extracted as the first detection region, and selects any one of the regions that is different from the aforementioned first detection region as the second detection region; the aforementioned image similarity calculation unit extracts the aforementioned feature vectors from the image that contain the aforementioned normal region corresponding to the aforementioned first detection region as the seventh feature vector; the aforementioned image similarity calculation unit extracts the aforementioned feature vectors from the image that contain the aforementioned normal region corresponding to the aforementioned second detection region as the eighth feature vector; the aforementioned image similarity calculation unit calculates the fourth similarity between the aforementioned seventh feature vector and the aforementioned eighth feature vector; the aforementioned image similarity calculation unit updates the aforementioned feature extraction model in such a way that the aforementioned fourth similarity increases.

13. The learning support device as described in claim 1, wherein, The aforementioned image distribution display unit provides a user interface for displaying the aforementioned distribution; when a user inputs the idea that the aforementioned distribution indicates that the aforementioned machine learning has been implemented well on the aforementioned user interface, the aforementioned image distribution display unit will display the aforementioned detection region image corresponding to the aforementioned distribution as the aforementioned annotation candidate region on the aforementioned user interface.

14. The learning support device as described in claim 13, wherein, The aforementioned image distribution display unit will display images of the aforementioned detection regions, such as those with low similarity to the aforementioned detection regions, as the aforementioned annotation candidate regions on the aforementioned user interface.

15. The learning support device as described in claim 13, wherein, The aforementioned user interface displays a distribution map, which displays the feature values ​​of the aforementioned learning image data for each of the aforementioned partial images; the aforementioned image distribution display unit displays the feature values ​​selected on the aforementioned user interface as the aforementioned annotation candidate areas on the aforementioned user interface.

Images