Information processing apparatus, and method and program for controlling information processing apparatus

Abstract

To improve the precision of a neural network model by efficiently selecting learning data and improving uniformity in learning efficiency by the learning data.SOLUTION: A learning data set is clustered into one of a plurality of clusters in advance. A determination is made whether dislike image data is included which has a difference equal to or larger than a threshold between a correct answer value of the learning data set and an inference result of learning data by a neural network model. When the dislike image data is present, the dislike image data is clustered by the same classifier to specify the cluster that the dislike image data belongs to. Then the similarity between data in the cluster and the dislike image data is calculated to add data to the learning data in order of higher similarity.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention particularly relates to an information processing apparatus, a control method for an information processing apparatus, and a program, which are suitable for use in selecting learning data.

Background Art

[0002] In recent years, various services utilizing AI have been provided, and as a method for constructing a model for realizing an AI that predicts an arbitrary event, a method using machine learning is known. As one of the algorithms of a machine learning model, supervised learning using learning data composed of an input and a correct label is known.

[0003] When constructing a model using supervised learning, overfitting can be suppressed and prediction accuracy can be improved by performing learning using high-quality learning data. Here, high-quality learning data represents learning data having a high effect of improving the prediction accuracy of the model. Further, in order to tune a model adapted to a specific situation or application, it is necessary to perform learning using learning data considering the situation or application. Therefore, in supervised learning, it is important to appropriately select the learning data to be used.

[0004] Therefore, a method for excluding unintended data from learning data has been proposed. Patent Document 1 discloses a technique of performing pre-clustering based on feature amounts and class information of object images in image data, and identifying a cluster including incorrect class information by using the average / variance of the distances between a plurality of centroids in the cluster and each feature amount.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0007] When a machine learning model evaluates a plurality of evaluation metrics, it may be required to obtain an evaluation score of a certain level or higher in any of the plurality of evaluation metrics. When the evaluation score is low in a specific evaluation metric, it is necessary to improve the corresponding evaluation score. For example, when there is a bias in the learning data, learning progresses efficiently for specific data, but does not progress efficiently for other data, and the evaluation score remains low.

[0008] In the method described in Patent Document 1, it is said that learning data can be appropriately selected by excluding unintended data from the learning data, and learning can be performed efficiently. However, it takes a very long time to search for data that has been erroneously clustered from all classes. In addition, since only the information within the cluster can be used, the learning efficiency of the neural network model for specific data cannot be sufficiently improved.

[0009] In view of the above problems, an object of the present invention is to efficiently select learning data, improve the non-uniformity of learning efficiency by the learning data, and improve the accuracy of the neural network model.

Means for Solving the Problems

[0010] The information processing apparatus according to the present invention includes: a classification unit that clusters a learning data set for learning a neural network model and classifies it into one of a plurality of clusters; a correct value of the learning data set; and a determination unit that determines image data in which a difference between an inference result of learning data by the neural network model or verification data different from that during learning and the correct value is equal to or greater than a threshold value; a specifying unit that specifies, from among the plurality of clusters, the cluster to which the image data determined by the determination unit belongs, and a calculating unit that calculates a similarity between the data classified into the specified cluster by the classification unit and the determined image data; and a selection unit that selects, as learning data for learning the neural network model, data among the data classified into the specified cluster for which the similarity calculated by the calculating unit is equal to or greater than a predetermined value.

Effect of the Invention

[0011] According to the present invention, learning data can be efficiently selected, the non-uniformity of learning efficiency by the learning data can be improved, and the accuracy of the neural network model can be improved.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0014] (First Embodiment) In the present embodiment, taking the noise reduction task as an example, the processing flow when training a neural network model (hereinafter referred to as an NN model) will be described. The noise reduction task is a task of estimating a noise-free image (pre-degradation image) before degradation by noise from a noisy image (degraded image) degraded by noise.

[0015] Here, consider the case of evaluating a plurality of evaluation metrics using an NN model. For example, when evaluating an evaluation metric (PSNR (Peak Signal to Noise Ratio)) regarding the amount of image degradation in the noise reduction task, the PSNR for each of a plurality of regions in the image may be evaluated. In addition, other cases of evaluating a plurality of evaluation metrics include cases where the accuracy rate for each class is evaluated in an object detection task. Thus, when a machine learning model evaluates a plurality of evaluation metrics, a certain or higher evaluation score may be required for any of the plurality of evaluation metrics, and when the evaluation score is low for a specific evaluation metric, it is necessary to improve the corresponding evaluation score.

[0016] As one of the reasons for the low evaluation score in a specific evaluation metric, the difference in the learning efficiency of the NN model can be considered. For example, when there is a bias in the training data, learning progresses efficiently for specific data, and the difference from the correct value (hereinafter referred to as GT (Ground Truth)) is small and the accuracy also increases. On the other hand, for other data, learning does not progress efficiently, and the difference from GT remains large and the accuracy remains low.

[0017] In addition, in order to improve the evaluation score in an evaluation metric with a low evaluation score, it is also conceivable to replace the training dataset, etc. However, optimizing the training dataset, etc. to improve the evaluation score of a specific evaluation metric takes a lot of time. Therefore, in this embodiment, the training dataset is pre-clustered and classified into a plurality of clusters, and data is added to the training data according to the similarity from the cluster to which the data with a large difference from GT belongs, so that the training data can be selected more efficiently. Hereinafter, the detailed processing in this embodiment will be described.

[0018] FIG. 7 is a block diagram showing an example of the hardware configuration of the information processing apparatus 100 according to this embodiment. In FIG. 7, the processor 701 is, for example, a CPU and controls the operation of the entire information processing apparatus 100. The memory 702 is, for example, a RAM and temporarily stores programs and data, etc. The computer-readable storage medium 703 is, for example, a hard disk or a CD-ROM, etc. and stores programs and data, etc. in the long term. In this embodiment, the program for realizing the functions of each part stored in the storage medium 703 is read into the memory 702. Then, when the processor 701 operates according to the program on the memory 702, the functions of each part are realized.

[0019] In FIG. 7, the input interface 704 is an interface for acquiring information from each device. The output interface 705 is an interface for outputting information to an external device. The bus 706 connects each part above and enables the exchange of data.

[0020] Figure 1 is a block diagram showing a functional configuration example of the information processing apparatus 100 according to the present embodiment. The information processing apparatus 100 includes a model storage unit 101, a learning data set 110, a classifier 120, a data set group 130, a learning unit 140, a disliked image determination unit 150, disliked image data 160, a similarity calculation unit 170, and a data selection unit 180.

[0021] The model storage unit 101 stores an NN model for the purpose of noise reduction. Although it is assumed that the NN model has been pre-trained, it is not limited to being pre-trained, and a pre-trained NN model publicly available by a third party may also be used. Further, as the NN model, a CNN model (Convolutional Neural Network) having a convolutional layer may be used, or a Transformer model having an attention mechanism may be used.

[0022] The learning data set 110 is the same data set as the data set used to train the NN model stored in the model storage unit 101. Note that another image data set may be used for the learning data set 110. The classifier 120 is a classifier generated to cluster the learning data set 110 and the disliked image data 160 described later. The data set group 130 represents the result of clustering the learning data set 110 using the classifier 120. The learning unit 140 trains the NN model stored in the model storage unit 101 using the learning data set 110.

[0023] The disliked image determination unit 150 determines the disliked image data based on the difference between the result of the learning data inferred by the learning unit 140 and the GT of the learning data set 110. Specifically, the disliked image determination unit 150 calculates the difference between the result of the learning data inferred by the learning unit 140 and the GT, and defines the learning data as disliked image data when the difference is equal to or greater than a threshold value. The disliked image data 160 is the image data determined to be disliked image data by the disliked image determination unit 150.

[0024] The similarity calculation unit 170 determines the cluster to which the dislike image data 160 belongs, and calculates the similarity with the data in the dataset group 130 existing within the same cluster. The data selection unit 180 selects data for additional learning of the NN model from the data in the dataset group 130 within the same cluster based on the similarity obtained by the similarity calculation unit 170.

[0025] FIG. 2 is a flowchart showing an example of a processing procedure for selecting learning data in the present embodiment. In step S201, the processor 701 pre-clusters the learning dataset 110 using the classifier 120. Hereinafter, the clustering method will be described in detail.

[0026] FIG. 3 is a diagram for explaining a method of generating a classifier using self-supervised learning in the present embodiment. In the present embodiment, a classifier is generated by self-supervised learning called DeepCluster described in Non-Patent Document 1. First, the dataset 301 is input to the classifier 302, and the classifier 302 is learned using the pseudo label 303 and the inference result 304. The learning model used for the classifier 302 is not particularly limited, and for example, AlexNet described in Non-Patent Document 2 can be used. The pseudo label 303 is generated by inputting the dataset 301 to the classifier 302, acquiring the feature amount before inputting to the fully connected layer, and then clustering the feature amount by the k-means method. The inference result 304 uses the output after inputting the feature amount to the fully connected layer. The Loss between the pseudo label 303 and the inference result 304 is calculated, and the classifier 302 is learned by backpropagating the result. By using the classifier generation method by self-supervised learning as in the present embodiment, it is not necessary to label a large amount of datasets, so the preparation load of the learning data can be minimized.

[0027] FIG. 4 is a diagram for explaining the classification method of the learning dataset 401. Although it is originally a multi-dimensional feature space, for simplicity of explanation, it will be explained in a two-dimensional feature space. Also, in the present embodiment, for simplicity of explanation, it is assumed that the number of clusters is 3. Hereinafter, the data belonging to clusters 1 to 3 are referred to as dataset groups 1 to 3, respectively. When the learning dataset 401 is input to the classifier 402, the positions of each data in the feature space are specified, and they are clustered into dataset groups 1 to 3 as shown in the classification result 403.

[0028] Next, in step S202, the learning unit 140 learns the NN model using the learning dataset 110. Subsequently, in step S203, the dislike image determination unit 150 determines whether or not it includes dislike image data based on the difference between the result of the learning data inferred during the learning in S202 and the GT. As a method for calculating the difference between each inference result and the GT, for example, a Loss function such as mean squared error or cross entropy may be used, or it may be calculated as the difference in pixel values between each inference result and the GT. When all the differences calculated from each inference result are less than the threshold value, the dislike image determination unit 150 determines that the dislike image data is not included in the inferred learning data, and ends the process without doing anything.

[0029] On the other hand, when there is data whose difference is equal to or greater than the threshold value, that data is determined as dislike image data. When the dislike image determination unit 150 determines that it includes dislike image data in this way, it proceeds to step S204, and repeatedly executes the processes of steps S204 to S206 and step S202. Note that the range for calculating the difference between each inference result and the GT may be the entire image or a local part obtained by dividing the image. Also, in the present embodiment, the difference from the GT is calculated using the inference result during learning, but the difference from the GT may be calculated using data different from that during learning, for example, verification data.

[0030] In step S204, the processor 701 inputs the aversive image data whose difference from GT is equal to or greater than the threshold to the classifier 120 to perform clustering. Here, a method for clustering the aversive image data will be described.

[0031] FIG. 5(a) is a diagram for explaining a clustering method using a classifier. When clustering is performed by inputting the aversive image data 501 whose difference from GT is equal to or greater than the threshold to the classifier 402, the feature amount 502a of the aversive image data 501 is output from the classifier 402. Then, it is specified to which cluster of which dataset group the output feature amount 502a belongs with respect to the classification result 403 clustered in advance. In the present embodiment, it is assumed that the feature amount 502a of the aversive image data 501 belongs to the dataset group 3.

[0032] In step S205, the similarity calculation unit 170 calculates the similarity between the aversive image data and each data of the dataset group belonging to the same cluster. The similarity may be obtained in the feature space when clustering by the classifier, or may be calculated as the difference in an image feature amount space different from the classifier, for example, the pixel value or luminance value of the image. Further, the feature amount of the image may be converted into a vector, and the similarity may be calculated by the distance between vectors such as the cosine similarity and the Euclidean distance. In the present embodiment, as an example, the similarity is calculated using the cosine similarity. The cosine similarity cos(x,y) is represented by the following formula (1).

[0033]

Equation

[0034] Here, the feature amount of the aversive image data is an n-dimensional feature vector x = (x1, x2, ···, x n ). On the other hand, the feature amount of the data of the dataset group belonging to the same cluster specified in step S204 is an n-dimensional feature vector y = (y1, y2, ··· y n ). In the present embodiment, the vector values of x and y are substituted into the formula (1) to calculate the cosine similarity.

[0035] In step S206, the data selection unit 180 selects data with a high similarity for the similarity calculated in step S205 and adds it to the learning data. Since the cosine similarity approaches 1 as the similarity between two feature vectors increases, data with a cosine similarity close to 1 is preferentially added to the learning data. When adding data to the learning data, it may be simply added to the learning data set used for learning, or only the selected data may have its usage frequency increased during learning. In this embodiment, a method of selecting data to be added to the learning data from the learning data set 110 (data set group 130) has been described, but learning data for addition may be prepared separately and data may be selected therefrom.

[0036] FIG. 5(b) is a diagram for explaining an example of selecting data with a high similarity for the similarity calculated in step S205. In this embodiment, first, the similarity between the feature amount 502b of the difficult image data in the feature amount space of the image and the data in the data set group 3 belonging to the same cluster is calculated. Thereafter, data existing within a circle 504 having a predetermined threshold 503 as the radius centered on the feature amount 502b of the difficult image data is regarded as having a similarity equal to or higher than a predetermined value, and the data existing within the circle 504 is selected and added to the learning data.

[0037] As in this embodiment, by clustering the learning data set in advance, it is possible to omit the calculation of the similarity with data belonging to other clusters, and the time until data to be added to the learning data is selected can be reduced. Also, by clustering in advance, data cleansing such as identifying incorrect clusters by the method described in Patent Document 1 and preventing unintended data from being mixed into the learning data becomes easy, and appropriate data can be selected as the learning data.

[0038] In this embodiment, the noise reduction task has been described as an example, but it can also be applied to other high image quality tasks such as super-resolution tasks. Further, it is not limited to high image quality tasks, and for example, it can also be applied to class classification tasks and BB (Bounding Box) detection tasks. In the case of the class classification task, the selection of learning data is performed so as to improve classes whose correct answer rate for each class is less than a predetermined threshold. Also, in the case of the BB detection task, the AP (Average Precision) for each class is set, and the selection of learning data is performed so as to improve classes whose AP is less than a predetermined threshold. In addition, as long as it is a task having an evaluation index that can be represented by a numerical value, this embodiment can be applied.

[0039] (Second Embodiment) Hereinafter, in this embodiment, the flow of the process of selecting learning data when performing learning of the NN model using the noise reduction task as an example will be described. In the first embodiment, an example of generating a classifier for clustering a dataset by self-supervised learning described in Non-Patent Document 1 has been described. In contrast, in this embodiment, an example of generating a classifier by supervised learning will be described. The configuration of the information processing apparatus in this embodiment is the same as that in FIGS. 1 and 7, and the basic flow of the processing procedure is also the same as that in FIG. 2, so the description will be omitted. The part different from the first embodiment in this embodiment is the method of generating the classifier used in step S201.

[0040] FIG. 6 is a diagram for explaining a method of generating a classifier using supervised learning in the present embodiment. First, a dataset 601 is input to a classifier 602, and an inference result 603 is output. Then, the Loss between the output inference result 603 and teacher data 604 with pre-labeled correct labels is calculated, and the classifier 602 is learned by the error backpropagation method. As the teacher data 604, for example, those labeled for each evaluation index of the noise reduction task are used. Here, as the evaluation index, in addition to the PSNR described above, for example, SNR (Signal to Noise Ratio), SSIM (Structual SIMilarity), or MSE (Mean Squared Error) may be used.

[0041] By using the method of the present embodiment, when the evaluation score is low in a specific evaluation index, data similar to the data with the low evaluation score can be immediately identified, and the learning of the NN model can be efficiently advanced. Note that in the present embodiment, the teacher data 604 uses those labeled for each evaluation index, but is not limited thereto. For example, it may be labeled for each time series to reflect characteristics such as aging changes in the learning data, or labeled for each image characteristic such as luminance, lightness, and chroma to eliminate the bias between hues in the learning data.

[0042] (Other Embodiments) In the above-described embodiment, a classifier is generated using self-supervised learning or supervised learning, but the method of generating the classifier is not limited thereto. For example, unsupervised learning may be used to generate a classifier so as to perform clustering according to the similarity of the feature vectors of the images. Also, in the method of generating the classifier, hierarchical clustering such as the Ward method may be used, or typical non-hierarchical clustering such as the k-means method may be used.

[0043] In the above-described embodiment, as shown in FIG. 5(b), data within a predetermined distance centered on the feature amount of the disliked image data was selected and added to the learning data. However, data to be added to the learning data may be selected by a different method. For example, after clustering the disliked image data to identify the cluster to which the disliked image data belongs, data within a predetermined distance from the center of gravity within the cluster may be added as learning data. By using this method, it is possible to eliminate the need for calculating the similarity, and it is possible to select learning data at a higher speed.

[0044] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0045] The disclosure of this embodiment includes the following configurations, methods, and programs.

[0046] (Configuration 1) Classification means for clustering a learning data set for learning a neural network model and classifying it into any of a plurality of clusters, Determination means for determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from that at the time of learning is equal to or greater than a threshold value, Calculation means for identifying the cluster to which the image data determined by the determination means belongs from among the plurality of clusters, and calculating the similarity between the data classified into the identified cluster by the classification means and the determined image data, Selection means for selecting, as learning data for learning the neural network model, data in which the similarity calculated by the calculation means is equal to or greater than a predetermined value among the data classified into the identified cluster, An information processing apparatus comprising the same.

[0047] (Configuration 2) The classification means classifies using a classifier generated by supervised learning using teacher data labeled for each evaluation index, each time series, or each image characteristic, and is the information processing apparatus according to Configuration 1. (Configuration 3) The evaluation index is SNR (Signal to Noise Ratio), PSNR (Peak Signal to Noise Ratio), SSIM (Structual SIMilarity), or MSE (Mean Squared Error), and is the information processing apparatus according to Configuration 2. (Configuration 4) The image characteristic is luminance, lightness, or chroma, and is the information processing apparatus according to Configuration 2. (Configuration 5) The classification means classifies using a classifier generated by unsupervised learning, and performs classification according to the similarity of the feature vectors of the images, and is the information processing apparatus according to Configuration 1. (Configuration 6) The classification means classifies using a classifier generated by hierarchical clustering or non-hierarchical clustering, and is the information processing apparatus according to Configuration 1.

[0048] (Configuration 7) The determination means divides the learning data into local regions, and determines by calculating the difference from the correct value for each local region, and is the information processing apparatus according to any one of Configurations 1 to 6. (Configuration 8) The calculation means specifies the cluster to which the determined image data belongs from among the plurality of clusters using the same classifier as the classifier used by the classification means, and is the information processing apparatus according to any one of Configurations 1 to 7. (Configuration 9) The calculation means calculates the similarity using the feature amount of the image, and is the information processing apparatus according to any one of Configurations 1 to 8. (Configuration 10) The calculation means converts the feature amount of the image into a vector and calculates the similarity based on the distance between the vectors, according to the information processing apparatus described in Configuration 9. (Configuration 11) The calculation means calculates the similarity based on the difference in pixel values or luminance values of the image, according to the information processing apparatus described in any one of Configurations 1 to 8.

[0049] (Configuration 12) Classification means for clustering a learning data set for learning a neural network model and classifying it into one of a plurality of clusters, Determination means for determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from the learning time is equal to or greater than a threshold value, Specification means for specifying, from among the plurality of clusters, the cluster to which the image data determined by the determination means belongs, Selection means for selecting, as learning data for learning the neural network model, data within a predetermined distance from the center of gravity within the specified cluster among the data classified by the classification means into the cluster specified by the specification means, An information processing apparatus characterized by comprising the above.

[0050] (Method 1) A classification step of clustering a learning data set for learning a neural network model and classifying it into one of a plurality of clusters, A determination step of determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from the learning time is equal to or greater than a threshold value, A calculation step of specifying, from among the plurality of clusters, the cluster to which the image data determined in the determination step belongs, and calculating the similarity between the data classified into the specified cluster in the classification step and the determined image data, A selection step of selecting, as learning data for training the neural network model, data among the data classified into the specified cluster that has a similarity calculated in the calculation step equal to or greater than a predetermined value. A control method for an information processing apparatus, characterized by comprising the above.

[0051] (Method 2) A classification step of clustering a learning data set for training a neural network model and classifying it into one of a plurality of clusters. A determination step of determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from the learning time is equal to or greater than a threshold value. An identification step of identifying, from among the plurality of clusters, the cluster to which the image data determined in the determination step belongs. A selection step of selecting, as learning data for training the neural network model, data among the data classified in the classification step into the cluster identified in the identification step, where the distance from the centroid within the identified cluster is within a predetermined value. A control method for an information processing apparatus, characterized by comprising the above.

[0052] (Program 1) A classification step of clustering a learning data set for training a neural network model and classifying it into one of a plurality of clusters. A determination step of determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from the learning time is equal to or greater than a threshold value. An identification step of identifying, from among the plurality of clusters, the cluster to which the image data determined in the determination step belongs, and a calculation step of calculating the similarity between the data classified into the identified cluster in the classification step and the determined image data. A selection step of selecting, as learning data for training the neural network model, data among the data classified into the specified cluster and having a similarity calculated in the calculation step that is equal to or greater than a predetermined value; A program for causing a computer to execute.

[0053] (Program 2) A classification step of clustering a learning data set for training a neural network model and classifying it into any of a plurality of clusters; A determination step of determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from that during learning is equal to or greater than a threshold value; An identification step of identifying, from among the plurality of clusters, the cluster to which the image data determined in the determination step belongs; A selection step of selecting, as learning data for training the neural network model, data among the data classified in the classification step into the cluster identified in the identification step and having a distance from the centroid within the cluster that is within a predetermined value; A program for causing a computer to execute.

Explanation of Signs

[0054] 120 Classifier, 150 Difficult Image Determination Unit, 170 Similarity Calculation Unit, 180 Data Selection Unit

Claims

1. Classification means for clustering a learning data set for learning a neural network model and classifying it into one of a plurality of clusters; Determination means for determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from the learning time is equal to or greater than a threshold value; Calculation means for specifying the cluster to which the image data determined by the determination means belongs from among the plurality of clusters, and calculating the similarity between the data classified into the specified cluster by the classification means and the determined image data; Selection means for selecting, as learning data for learning the neural network model, data among the data classified into the specified cluster that has a similarity calculated by the calculation means equal to or greater than a predetermined value; An information processing apparatus comprising the above.

2. The information processing apparatus according to claim 1, wherein the classification means classifies using a classifier generated by supervised learning using teacher data labeled for each evaluation index, each time series, or each image characteristic.

3. The information processing apparatus according to claim 2, wherein the evaluation index is SNR (Signal to Noise Ratio), PSNR (Peak Signal to Noise Ratio), SSIM (Structual SIMilarity), or MSE (Mean Squared Error).

4. The information processing apparatus according to claim 2, wherein the image characteristic is luminance, lightness, or chroma.

5. The information processing apparatus according to claim 1, wherein the classification means performs classification according to the similarity of the feature vectors of the images using a classifier generated using unsupervised learning.

6. The information processing apparatus according to claim 1, wherein the classification means classifies using a classifier generated using hierarchical clustering or non-hierarchical clustering.

7. The information processing apparatus according to claim 1, wherein the determination means divides the learning data into local regions, and determines by calculating the difference from the correct value for each local region.

8. The information processing apparatus according to claim 1, wherein the calculation means specifies, from among the plurality of clusters, the cluster to which the image data determined belongs by using the same classifier as the classifier used by the classification means.

9. The information processing apparatus according to claim 1, wherein the calculation means calculates a similarity using feature amounts of an image.

10. The information processing apparatus according to claim 9, wherein the calculation means converts the feature amounts of the image into vectors and calculates a similarity based on distances between the vectors.

11. The information processing apparatus according to claim 1, wherein the calculation means calculates a similarity based on differences in pixel values or luminance values of an image.

12. Classification means for clustering a learning data set for learning a neural network model and classifying it into one of a plurality of clusters; Determination means for determining image data in which a difference between a correct value of the learning data set and an inference result of the learning data by the neural network model or verification data different from that during learning is equal to or greater than a threshold value; Specification means for specifying, from among the plurality of clusters, the cluster to which the image data determined by the determination means belongs; Selection means for selecting, as learning data for learning the neural network model, data within a predetermined value from the center of gravity within the specified cluster among the data classified by the classification means into the cluster specified by the specification means; An information processing apparatus, characterized by comprising:

13. A classification step of clustering a learning data set for learning a neural network model and classifying it into one of a plurality of clusters; A determination step of determining image data in which a difference between a correct value of the learning data set and an inference result of the learning data by the neural network model or verification data different from that during learning is equal to or greater than a threshold value; A calculation step of specifying, from among the plurality of clusters, the cluster to which the image data determined in the determination step belongs, and calculating a similarity between the data classified into the specified cluster in the classification step and the determined image data; A selection step of selecting, as learning data for learning the neural network model, data in which the similarity calculated in the calculation step is equal to or greater than a predetermined value among the data classified into the specified cluster; A control method for an information processing apparatus, characterized by comprising

14. A classification step of clustering a learning data set for learning a neural network model and classifying it into one of a plurality of clusters, A determination step of determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from the learning time is equal to or greater than a threshold value, A specifying step of specifying, from among the plurality of clusters, the cluster to which the image data determined in the determination step belongs, A selection step of selecting, as learning data for learning the neural network model, data within a predetermined value from the center of gravity within the specified cluster among the data classified in the classification step into the cluster specified in the specifying step, A control method for an information processing apparatus, characterized by comprising

15. A classification step of clustering a learning data set for learning a neural network model and classifying it into one of a plurality of clusters, A determination step of determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from the learning time is equal to or greater than a threshold value, A calculation step of specifying the cluster to which the image data determined in the determination step belongs from among the plurality of clusters and calculating the similarity between the data classified into the specified cluster in the classification step and the determined image data, A selection step of selecting, as learning data for learning the neural network model, data having a similarity calculated in the calculation step equal to or greater than a predetermined value among the data classified into the specified cluster, A program for causing a computer to execute

16. A classification step of clustering a learning data set for learning a neural network model and classifying it into one of a plurality of clusters, A determination step of determining image data in which the difference between the correct value of the learning data set and the inference result of the learning data by the neural network model or verification data different from the learning time is equal to or greater than a threshold value, A specifying step of specifying, from among the plurality of clusters, the cluster to which the image data determined in the determination step belongs, A selection step of selecting, as learning data for training the neural network model, data among the data classified in the classification step that is within a predetermined value from the center of gravity within the specified cluster in the specified cluster in the specific process; A program for causing a computer to execute the above.

Images