Learning device, learning method, and program

Abstract

To provide a technology capable of learning a highly accurate and effective feature even in the case of including an erroneous teacher label in a data group for learning.SOLUTION: A learning device according to an embodiment includes: a feature extraction unit for extracting a feature of data of a learning object included in a data group for learning given as a set of data for learning including a pair of data of a learning object and a teacher label showing similarity between the data of a learning object and another data by using a feature extractor generated in advance; and an update unit for acquiring a value of a loss function on the basis of a feature of the data of learning object and the teacher label related to the feature, and updating a parameter of the feature extractor so as to make an evaluation value based on the value of the loss function small.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for learning information through machine learning and a technique for estimating information based on the learned model. [Background technology]

[0002] In recent years, there has been remarkable progress in AI (Artificial Intelligence) technology. In the field of image recognition, image recognition technology has already been developed that exceeds human accuracy. In the field of voice recognition, many technologies that achieve stable voice recognition even in natural noisy environments have been put into practical use and can now be easily used from small devices such as smartphones. Machine translation has also become available online in everyday life, dramatically lowering the barriers to international communication.

[0003] Needless to say, the driving force behind the recent advances in AI technology is the technology for learning deep neural networks, known as deep learning. A notable feature of deep learning is its high feature learning ability, which makes it possible to directly learn feature representations (also called features, feature quantities, feature vectors, etc.) that accurately express the meaning and content of various data, such as images, sounds, and language, from the original data.

[0004] Even before the rise of deep learning, feature representations have played a central role in many information processing applications. For example, image recognition has been achieved by training a classifier based on features extracted from images and classifying images into semantic categories. Image search also judges the similarity of content based on features. Similarly, speech recognition is achieved by training an acoustic model that extracts features from speech waveforms and classifies them into phoneme classes. Before deep learning, such features or feature extractors were designed manually, so one of the key factors that determined performance was how low-capacity and appropriate features could be extracted. Designing these features or feature extractors was often difficult for designers, but deep learning has made it possible to directly learn the relationship between input and output from data, which has brought about a breakthrough in dramatically improving the accuracy of learning.

[0005] However, even when deep learning is used, it is essential to design an effective learning method in order to accurately train features or feature extractors. In particular, in information processing, it is often desirable for the features to be more similar the closer the data contents are to each other, and conversely, for the features to be less similar the closer the data contents are to each other.

[0006] For example, in classification problems such as image recognition and speech recognition, features of data in the same class are placed close to each other in the feature space, and features of different classes are placed farther apart, which makes it easier to classify them. Also, in image search, that is, a process of searching a large number of images for images similar in content to data given as a query image, it is convenient to place features of data that are similar in content close to each other in the feature space, and those that are not close to each other. Up to now, various technologies have been developed to realize learning technologies that enable learning of features that meet such requirements.

[0007] For example, in Non-Patent Document 1, a function called Contrastive Loss is proposed as a loss function for learning features that satisfy the above requirements. Contrastive Loss is a loss function that requests that, for a pair of two data, features whose contents are similar are positioned close to each other, and features whose contents are not similar are positioned farther away. More specifically, Contrastive Loss is a function that assigns a relevance label yij to a pair of two data Ii and Ij (i ≠ j) that indicates whether the contents are similar or not, and if they are similar, yij = 1, and if not, yij = 0. Such Contrastive Loss is defined, for example, as in Equation (1).

[0008]

number

[0009] Here, xi is a feature of Ii. m is a parameter called margin, which is set in advance to a value such as 0.2. d(xi,xj) is a distance function that represents the distance between xi and xj. Any distance function can be used for d(xi,xj), but it can be, for example, the Euclidean distance.

[0010] Non-Patent Document 2 proposes Triplet Loss as a loss function based on an idea similar to Contrastive Loss. The difference between Contrastive Loss and Triplet Loss is that Contrastive Loss compares two sets of data, whereas Triplet Loss compares three sets of data. More specifically, Triplet Loss groups three data for a given data Ia: data Ip that is similar in content to the data Ia, and data In that is distant in content (Ia ≠ Ip ≠ In). In this case, Triplet Loss is defined, for example, as in equation (2).

[0011]

number

[0012] In Triplet Loss, xa / Ia may be called the anchor, xp / Ip may be called the positive, and xn / In may be called the negative.

[0013] Patent Document 1 proposes a learning method that uses Triplet Loss to accurately classify a part of an image even when labels are assigned only to that part of the image.

[0014] Furthermore, Non-Patent Document 3 proposes a technology that uses Triplet Loss to enable learning of an estimator that estimates the pose of an object in an image even when there is no correct answer data for the pose of the object. Specifically, assume that there is an anchor image, which is an image of an object in a reference pose. When an image of an object in an arbitrary pose is input, a geometric transformation (alignment) is performed on the input image using a geometric transformation layer that introduces STN (Spatial Transformer Network) described in Non-Patent Document 4 so that the input image matches the anchor image. The pose parameters used in this geometric transformation are used to estimate the deviation of the input image from the anchor image. [Prior art documents] [Patent documents]

[0015] [Patent Document 1] JP 2020-047055 A [Non-patent literature]

[0016] [Non-Patent Document 1] Sumit Chopra, Raia Hadsell, Yann LeCun, “Learning a Similarity Metric Discriminatively, with Application to Face Verification”, in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp.539-546, 2005. [Non-Patent Document 2] Florian Schroff, Dmitry Kalenichenko, James Philbin. “FaceNet: A Unified Embedding for Face Recognition and Clustering”, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815-823, 2015. [Non-Patent Document 3] Koki Ueno, Go Irie, Masashi Nishiyama, Yoshio Iwai. Weakly Supervised Triplet Learning of Canonical Plane Transformation for Joint Object Recognition and Pose Estimation, Proceedings of 2019 IEEE International Conference on Image Processing (ICIP), 2019. [Non-Patent Document 4] Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial Transformer Networks. Proceedings of Neural Information Processing Systems, 2015. Summary of the Invention [Problem to be solved by the invention]

[0017] From a broad perspective, all of the above conventional technologies are technologies that, given labels indicating the relevance of several pieces of data (whether the content is similar or distant) as training data, learn to make the characteristics of data that are indicated by the training data as being closely related closer to each other, and learn to make the characteristics of data that are indicated as being distant closer to each other.

[0018] Naturally, these technologies assume that the labels given as training data are accurate. However, in reality, it is often difficult to imagine a set of training data that is free of errors. In order to guarantee that there are no errors, it is necessary to pay close attention, such as by checking the accuracy multiple times, which is very costly. Furthermore, the association between data is often a criterion that itself contains ambiguity, and due to the resulting fluctuations in labels, there are many cases in which accurate training labels cannot be obtained even if great care is taken.

[0019] When using such a set of training data containing erroneous labels, all of the above conventional techniques will perform training based on incorrect associations. In other words, the techniques will end up training by moving the features of the data closer / closer than they should, resulting in the problem that it is not possible to learn highly accurate and effective features.

[0020] Given the above background, there is a need for learning technology that can learn highly accurate and effective features even in cases where it is not possible to obtain accurate teacher labels that indicate whether the content of the data is close or distant.

[0021] In view of the above circumstances, the present invention aims to provide a technology that makes it possible to learn effective features with high accuracy even when an erroneous teacher label is included in a training data group. [Means for solving the problem]

[0022] One aspect of the present invention is a learning device that, in relation to a training data group provided as a collection of training data including a pair of training target data and a teacher label indicating similarity between the training target data and other data, comprises: a feature extraction unit that extracts features of the training target data included in the training data group using a pre-generated feature extractor; and an update unit that calculates a value of a loss function based on the features of the training target data and the teacher label related to the features, and updates parameters of the feature extractor so as to reduce an evaluation value based on the value of the loss function.

[0023] One aspect of the present invention is an estimation device including a feature extraction unit that extracts features of target data based on a feature extractor generated by the above-mentioned learning device, and an estimation unit that estimates information about the target data based on the features of the target data extracted by the feature extraction unit.

[0024] One aspect of the present invention is a learning method for a training data group provided as a collection of training data including a pair of training target data and a teacher label indicating similarity between the training target data and other data, the learning method comprising: a feature extraction step of acquiring features of the training target data included in the training data group using a feature extractor generated in advance; and an update step of determining a value of a loss function based on the features of the training target data and the teacher label associated with the features, and updating parameters of the feature extractor so as to reduce an evaluation value based on the value of the loss function.

[0025] One aspect of the present invention is an estimation method having a second feature extraction step of extracting features of target data based on a feature extractor generated by the above-mentioned learning method, and an estimation step of estimating information about the target data based on the features of the target data extracted in the second feature extraction step.

[0026] One aspect of the present invention is a program for causing a computer to function as the above-mentioned learning device.

[0027] One aspect of the present invention is a program for causing a computer to function as the above-described estimation device. Effect of the Invention

[0028] According to the present invention, even if an erroneous teacher label is included in a group of learning data, it is possible to learn effective features with high accuracy. [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 illustrates an example of the configuration of a learning device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a configuration of a feature extractor that is a learning target in the embodiment. [Diagram 3] 11 is a flowchart showing an example of the flow of a learning process executed by the learning device. [Figure 4] 1 is a schematic diagram illustrating a configuration example of an image recognition system according to an embodiment. [Diagram 5] FIG. 1 is a diagram illustrating an example of a configuration of an image recognition device according to an embodiment. [Figure 6] 11 is a flowchart showing an example of the flow of a learning process executed by the learning device. [Figure 7] 1 is a flowchart showing an example of the flow of an image recognition process executed by an image recognition device. [Figure 8] FIG. 2 is a diagram illustrating an example of a hardware configuration of a learning device and an image recognition device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0031] FIG. 1 is a diagram showing an example of the configuration of a learning device according to an embodiment. The learning device 10 is a device that learns a feature extractor as a learned model by machine learning based on a group of learning data. The feature extractor extracts features of target data when data to be processed (hereinafter referred to as "target data") is input, and outputs the features. The learning device 10 includes, for example, a learning data input unit 11, a learning data storage unit 12, a learning unit 13, and a learned model storage unit 14.

[0032] The learning data input unit 11 inputs a learning data group and stores the input learning data group in the learning data storage unit 12. The learning unit 13 generates a feature extractor 20 based on the learning data group stored in the learning data storage unit 12, and stores the generated feature extractor 20 in the learned model storage unit 14 as a learned model. The learning unit 13 also updates the feature extractor 20 and corrects the learning data group based on the feature extractor 20 stored in the learned model storage unit 14 and the learning data group stored in the learning data storage unit 12. Specifically, the learning unit 13 includes, for example, a feature extraction unit 131, an update unit 132, a detection unit 133, and a correction unit 134. Details of each functional unit will be described below, but first, the configuration of the feature extractor 20, which is the learning target, will be described.

[0033] <Feature extractor configuration> The feature extractor 20 is a function that inputs target data and outputs features of the target data. The feature extractor 20 may be configured as any function that can express such input / output relationships, but here, an example of the configuration of the feature extractor 20 will be described assuming that the target data is mainly an image. Note that an image is an example of target data. The target data may be any data that can form a feature extractor that expresses the above-mentioned input / output relationships, and in addition to images, any data such as sound and language may be used as the target data. Hereinafter, an image as the target data will be referred to as a "target image."

[0034] FIG. 2 is a diagram showing an example of the configuration of the feature extractor 20 to be trained in this embodiment. This configuration is based on a convolutional neural network and is configured by combining four types of layers: an input layer, a convolution layer, a pooling layer, a fully connected layer, and a geometric transformation layer. For example, the feature extractor 20 is configured by combining an input layer 21, a first convolution layer 22, a first pooling layer 23, a second convolution layer 24, a second pooling layer 25, a first fully connected layer 26, a second fully connected layer 27, and an output layer 28. The convolution layer, the pooling layer, and the fully connected layer are publicly known, and can be configured based on the technology described in Reference 1, for example.

[0035] [Reference 1] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of Neural Information Processing Systems, 2012.

[0036] The feature extractor 20 configured in this manner can output features as fixed-length vectors from the input target image. Note that the neural network of the feature extractor 20 in this embodiment is not limited to this configuration, and any configuration may be adopted as long as it satisfies the above-mentioned input / output requirements. For example, an L2 normalization layer may be added to the final layer (before the output layer). This is preferable because it makes the features more robust.

[0037] In addition, for example, the configuration shown in FIG. 2 has a structure in which the combination of a convolutional layer and a pooling layer is repeated twice, followed by connecting two fully connected layers. However, instead of these two fully connected layers, a configuration in which a global pooling layer disclosed in Reference 2 or Reference 3 below is connected may be used.

[0038] [Reference 2] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition, Proceedings of Conference on Computer Vision and Pattern Recognition, 2016. [Reference 3] Giorgos Tolias, Ronan Sicre, and Herve Jegou. Particular Object Retrieval with Integral Max-pooling of CNN Activations. ArXiv Preprint: https: / / arxiv.org / abs / 1511.05879, 2015.

[0039] In this way, a configuration without a fully connected layer is preferable because it is possible to always obtain features with the same number of dimensions regardless of the size of the input target image. In the following, unless otherwise specified, an example will be described in which images are assumed as learning data and the feature extractor 20 with the configuration shown in Figure 2 is used.

[0040] <Feature extractor learning process> Next, the learning process of the feature extractor 20 will be described. As described above, the feature extractor 20 that the learning device 10 of this embodiment learns is used in the process of extracting features of a target image. Therefore, the learning process needs to be performed at least once before the process of extracting features of a target image is performed. More specifically, the learning process is a process of determining the weights of the neural network, which are the parameters of the feature extractor 20, based on a group of learning data. To execute the learning process, it is necessary to prepare a group of learning data in advance.

[0041] The training data group can have several configurations depending on how the teacher label that indicates the association between data is given. Here, for example, assume that the training data group includes a set I={I1, I2, . . . , Im} of m pieces of data. In this case, a first configuration example is a configuration in which a label yij that is 1 when certain data Ii and Ij (i ≠ j) are associated with each other and 0 when they are not is given as the teacher label. In this case, the training data group is configured as a set of training data that groups together sets (Ii, Ij, yij).

[0042] As a second configuration example, for two pieces of data Ij and Ik that are different from Ii, the data that is highly related to Ii (positive data) is defined as Ij, and the data that is less related to Ii (negative data) is defined as Ik, and the set of learning data that lumps together (Ii, Ij, Ik) is defined as the learning data group.

[0043] In addition to the above examples, the learning data may be configured in any manner as long as at least one piece of data in the data set I is given an association with at least one piece of data different from that piece of data.

[0044] In addition, as long as the above conditions are satisfied for the training data group, any means may be used to generate the training data group. For example, the training data group may be generated manually, or some or all of the steps may be automatically performed by a computer.

[0045] If there are no errors in the teacher labels of the training data group configured as above, it is possible to learn highly accurate and effective features using, for example, the methods of Non-Patent Documents 1 to 3. However, as described above, in reality, it is difficult to imagine a training data group without errors.

[0046] For example, as in the first configuration example, when the learning data is composed of a set of three data (Ii, Ij, yij), and there is data Ii and Ij that are related to each other, a case in which the teacher label should be assigned as yij=1 but erroneously assigned as yij=0 (or vice versa) can be considered as an example of a case in which the teacher label is incorrect.

[0047] Alternatively, in the case where the training data is composed of pairs (Ii, Ij, Ik) as in the second configuration example, one example would be where the positive data and negative data have been swapped, i.e., Ii and Ik are highly correlated and Ii and Ij are low correlated, yet the training data set contains training data that is (Ii, Ij, Ik).

[0048] In the learning process of this embodiment, when there is a possibility that a learning data group may contain an incorrect teacher label, the accuracy of the learning data group is improved by a detection process for detecting learning data that is likely to have an incorrect teacher label added (hereinafter referred to as "mislabeled data") and a correction process for correcting the teacher label included in the detected mislabeled data, thereby making it possible to learn highly accurate and effective features even when a learning data group containing an incorrect teacher label is used. The detection process and correction process will be described in more detail below.

[0049] <Detection process> Here, a case will be described in which the learning data group is composed of three data sets (Ii, Ij, yij) and an incorrect teacher label is given to Ii and Ij. For example, assume that an incorrect teacher label yij=0 is given to Ii and Ij even though they are highly related data. In this case, if a correctly trained feature extractor 20 is obtained and features xi and xj of Ii and Ij are extracted, the similarity between features xi and xj tends to be large (the distance is small). In other words, in this case, when the value of the loss function of equation (1) is calculated, a large value is obtained.

[0050] In the opposite case, that is, when Ii and Ij are data with low relevance but an incorrect teacher label yij=1 is given, the value of the loss function will also be large.

[0051] According to this property, it is considered that there is a high possibility that the training data with a large loss function value has an error in the teacher label, assuming that a correctly trained feature extractor 20 has been obtained. Therefore, by regarding the training data with a large loss function value as mislabeled data and evaluating the loss function value, it is possible to detect mislabeled data with a high probability.

[0052] However, in reality, the learning of the feature extractor 20 is not completed during the execution of the learning process, and a trained feature extractor 20 is not available, so that mislabeled data cannot be detected by the above method.

[0053] On the other hand, in a training data group, the number of mislabeled data is often very small compared to the number of data with correct teacher labels, and even if training is performed with a training data group including incorrect teacher labels, it is considered that the fact that "mislabeled data tends to have a large value of the loss function" remains. Therefore, in this embodiment, such a property of the loss function is utilized, and the value of the loss function is calculated using the feature extractor 20 during training, and data with a large value is detected as mislabeled data.

[0054] For example, features are extracted from all learning data by applying the feature extractor 20 during learning, and the loss function value is calculated for the extracted features using equation (1). The top g features with the largest loss function values ​​are then detected as mislabeled data. g may be determined, for example, as a certain percentage of the number of data in the learning data group, such as a number equivalent to 10% of the total, or may be changed as learning progresses. Alternatively, a threshold t may be set for the loss function value, and data having a value equal to or greater than t may be detected as mislabeled data. In addition to these, any method may be used to detect mislabeled data as long as it selects data based on the magnitude of the calculated loss function value.

[0055] Although the method of detecting mislabeled data using the loss function of equation (1) has been described above using an example in which the training data is composed of three data sets (Ii, Ij, yij), the training data does not necessarily have to be composed of the sets (Ii, Ij, yij), and the loss function used does not necessarily have to be that of equation (1). For example, even if the training data is composed of three data sets (Ii, Ij, Ik) and the loss function of equation (2) is used, mislabeled data can be detected similarly based on the magnitude of the value of the loss function.

[0056] <Correction process> As mentioned above, mislabeled data tends to have a large loss function value. On the other hand, if the loss function value decreases when the teacher label is changed, it can be said that the changed teacher label is likely to be the correct label. For example, if (Ii,Ij,yij=0) is detected as mislabeled data, the loss function value of equation (1) is calculated for both (Ii,Ij,yij=0) and (Ii,Ij,yij=1) with the teacher label value changed. If the loss function value of (Ii,Ij,yij=1) is smaller, it can be said that the training data has been correctly corrected by changing the teacher label. Therefore, in this case, the training data before the change is replaced with the training data after the change. This operation is also the same when changing yij=1 to yij=0.

[0057] Similarly, when training data composed of three data pairs (Ii, Ij, Ik) is detected as mislabeled data, the loss function value of equation (2) is calculated for (Ii, Ij, Ik) and (Ii, Ik, Ij), and if the loss function value for (Ii, Ik, Ij) is smaller, (Ii, Ij, Ik) is replaced with (Ii, Ik, Ij).

[0058] Note that the purpose of the correction process is to change the teacher label so as to reduce the value of the loss function, and the correction method is not limited to the above example as long as it follows this purpose.

[0059] Although a typical example of the correction process has been described above, the accuracy of the correction process can be improved by implementing further improvements.

[0060] For example, consider a case where the learning data is composed of a set of three data (Ii, Ij, yij). Generally, in the case of a group of data collected naturally and randomly, the number of data sets that are not related to each other (i.e., those for which yij should be 0) is much larger than the number of related data (i.e., those for which yij should be 1). If the above correction process is directly applied to such a learning data group collected in this way, there is a possibility that it will be corrected to overly relevant data, exceeding the originally expected amount.

[0061] Therefore, by adding a regularization term that corrects the value of the above loss function, the frequency of being corrected to yij = 1 is suppressed. Any function can be used for the regularization term as long as this purpose can be achieved. For example, a regularization term defined as in equation (3) can be introduced, and the loss function can be defined as in equation (4).

[0062]

Number

[0063]

Number

[0064] p is a parameter representing the probability that yij = 1, and is a real value satisfying 0 < p < 1. The smaller p is, the less likely yij = 1 is. λ is a parameter for adjusting the balance between the original loss function and the regularization term. λ can be any real value, and for example, it can be set to half of the median of the distance between xi and xj.

[0065] Figure 3 is a flowchart showing an example of the flow of the learning process executed by the learning device 10. Here, regarding the content of each process, a case where the learning data is configured in the form of (Ii, Ij, yij) will be described with appropriate examples as needed. First, the learning unit 13 generates the feature extractor 20 (step S201).

[0066] Step S201 is for generating a feature extractor 20 that is at least necessary when the subsequent step S202 is executed for the first time. The feature extractor 20 generated here is, so to speak, an initial-state feature extractor 20, and the learning process of this embodiment is for obtaining a trained feature extractor 20 by increasing the accuracy of the initial-state feature extractor 20 through subsequent detection and correction processes. Therefore, the method of generating the feature extractor 20 here is not limited to a specific one, and when the initial-state feature extractor 20 has been generated in advance and stored in the trained model storage unit 14, step S201 may be omitted.

[0067] For example, typically, the initial state feature extractor 20 may be generated by randomly determining the weights of the neural network. Also, for example, if there is a neural network that has been trained in advance using some data and method, the initial state feature extractor 20 may be generated by copying the parameters of the trained neural network. In the following, in the process flow of FIG. 3, the feature extractor 20 up to the point where the final trained feature extractor 20 is obtained is referred to as the feature extractor 20 in the middle of training.

[0068] Next, the feature extraction unit 131 applies the feature extractor 20 in the middle of training to all data included in the training data group I={I1, I2, . . . , Im}, thereby generating a set X={x1, x2, ..., xm} of features of all data included in the data group I (step S202).

[0069] Next, the update unit 132 obtains a value of a loss function based on the features xi and xj corresponding to each learning data (Ii, Ij, yij) and the teacher label yij (step S203). Next, the update unit 132 updates the parameters of the feature extractor 20 during learning so as to reduce the value of the loss function (step S204). For example, the update unit 132 can update the parameters in the following manner.

[0070] As is clear from equations (1) and (2), the loss function is piecewise differentiable with respect to the parameters of the feature extractor 20. In view of this, the update unit 132 can update the parameters by the gradient method so that the value of the loss function L becomes smaller. For example, when updating the parameters based on the stochastic gradient descent method, if a certain parameter of the feature extractor 20 is denoted as w, the parameter w can be updated per step based on equation (6).

[0071]

number

[0072] For the constituent layers of a normal neural network, it is possible to calculate the differential value of the loss function L with respect to an arbitrary parameter w by the backpropagation method. With such a configuration, the update unit 132 can calculate the differential value of the loss function L with respect to an arbitrary parameter w by the formula (5). Note that the update unit 132 may introduce a commonly used improvement method for the stochastic gradient descent method, such as using a momentum term or weight decay, or may apply another gradient descent method.

[0073] The update unit 132 may update the parameters so as to reduce the value of the loss function for any one of all the learning image data, or may update the parameters so as to reduce the value of the loss function for multiple learning image data. When updating the parameters so as to reduce the value of the loss function for multiple learning image data, the update unit 132 may update the parameters so as to reduce an evaluation value (e.g., a sum) based on the values ​​of the loss functions of the multiple learning image data.

[0074] Next, the detection unit 133 determines whether or not the execution condition for the correction process is satisfied (step S205). For example, when the magnitude of the change in the parameters before and after the update of the feature extractor 20 updated in step S204 is greater than a threshold, it is considered that the accuracy of the learning data group needs to be improved, and in this case, the detection unit 133 may determine to execute the correction process. Also, for example, when the value of the loss function calculated in step S203 when updating the feature extractor 20 is smaller than a threshold, it is considered that the accuracy of the learning data group is sufficiently high, and in this case, the detection unit 133 may determine not to execute the correction process. In addition, the execution condition for the correction process may be set arbitrarily. By determining such an execution condition for the correction process, the learning device 10 can omit execution of the correction process in a specific situation, shorten the processing time, and reduce the processing load. Note that it is not essential to determine the execution condition for the correction process. The learning device 10 may be configured to execute the correction process each time the process is repeated, without determining the execution condition for the correction process.

[0075] In step S205, if it is determined that the execution condition of the correction process is satisfied (step S205), the detection unit 133 performs a detection process using the feature extractor 20 in the middle of learning that has updated the parameters in step S204, and detects mislabeled data (step S206). Next, the correction unit 134 performs a correction process on the detected mislabeled data, and changes the teacher label so that the value of the loss function becomes smaller (step S207). On the other hand, in step S205, if it is determined that the execution condition of the correction process is not satisfied (step S205-NO), the detection unit 133 skips steps S206 and S207 and proceeds to step S208.

[0076] Next, the learning unit 13 determines whether or not the end condition of the learning process is satisfied (step S208). When the learning unit 13 determines that the end condition of the learning process is satisfied (step S208-YES), the learning unit 13 ends the learning process. When the learning unit 13 determines that the end condition is not satisfied (step S208-NO), the learning unit 13 returns to the process in step S202 and repeatedly executes steps S202 to S207 until the end condition is satisfied. The end condition of the learning process may be set arbitrarily. For example, the end condition may be a condition that "the process ends after a predetermined number of repetitions (e.g., 100 times)," or a condition that "the process ends when the reduction in error falls within a certain range for a certain number of repetitions."

[0077] 3, the frequency of execution of the detection process (step S206) and the correction process (S207) may be adjusted to any frequency. Preferably, the detection process and the correction process are executed once for multiple executions of the parameter update process (steps S202 to S204), and after the detection process and the correction process are executed a certain number of times, the detection process and the correction process are not executed any more. For example, steps S206 and S207 may be executed once every 1000 repetitions of steps S202 to S204, and after steps S206 and S207 are executed three times, steps S206 and S207 are not executed any more.

[0078] The above describes the learning method of the feature extractor 20 in the embodiment. Below, an example of the learning method of the embodiment will be described using an example in which the name of a subject in an image is estimated based on the image.

[0079] FIG. 4 is a schematic diagram showing a configuration example of an image recognition system 100 according to an embodiment. The image recognition system 100 includes, for example, a learning device 10A and an image recognition device 30. The image recognition device 30 is a device that estimates and outputs the name of a subject in an input image data (hereinafter referred to as a "target image") as target data. Specifically, the image recognition device 30 extracts features of the target image using a feature extractor 20, and estimates the name of the subject in the target image by evaluating the similarity with a reference image prepared in advance based on the extracted features. The image recognition device 30 is an example of an "estimation device."

[0080] The learning device 10A is a device that generates a feature extractor 20 used by the image recognition device 30 when extracting features of a target image by using the learning method of the embodiment. The configuration of the learning device 10A is basically the same as that of the learning device 10 described in Figs. 1 to 3, so a detailed description will be omitted here. In the learning device 10A, the learning data input unit 11 inputs a set of data represented by a pair of image data and teacher labels as a learning data group, and the learning unit 13 generates a feature extractor 20 that extracts features related to a subject from an image based on the input learning data group, and stores the feature extractor 20 in the trained model storage unit 14. The feature extractor 20 stored in the trained model storage unit 14 is supplied to the image recognition device 30 prior to image recognition processing of the target image.

[0081] In the following, a case will be described in which the learning data is configured as a set of data (Ii, Ij, Ik) represented by a set of three pieces of data: an image Ii depicting a certain subject, an image Ij depicting the same subject as the image Ii, and an image Ik depicting a subject different from the image Ii. Hereinafter, image data provided as learning data will be referred to as learning image data. The generation of a learning image data group configured in this way may be performed manually, or some or all of the steps may be performed automatically by a computer. For example, if an image set that satisfies the following requirements (a) to (c) is obtained, it is possible to automatically configure the learning image data group by computer processing.

[0082] (a) an image set includes one or more images, and each image in the image set depicts at least one object; (b) In the entire training image data set, there are two or more types of objects depicted in the images, and there are at least two images depicting the same type of object. (c) Images of the same type of object are distinguishable from each other.

[0083] Then, for a set of images that satisfy the above requirements (a) to (c), a learning image data group can be constructed by repeatedly performing the following processes (1) to (3) until the desired number of sets are obtained.

[0084] (1) Randomly select one image from the image set and call it image Ii. (2) Randomly select one image that shows the same object as image Ii, and call this image Ij. (3) Randomly select an image that depicts an object different from image Ii, and call this image Ik.

[0085] FIG. 5 is a diagram showing a configuration example of an image recognition device 30 according to an embodiment. The image recognition device 30 includes, for example, a target image input unit 31, a trained model storage unit 32, a feature extraction unit 33, an image recognition unit 34, a reference image data storage unit 35, and an output unit 36. The target image input unit 31 inputs a target image and outputs it to the feature extraction unit 33. The trained model storage unit 32 stores the feature extractor 20 generated by the learning device 10A as a trained model in advance. The feature extraction unit 33 extracts features from the target image using the feature extractor 20 stored in the trained model storage unit 32. The reference image data storage unit 35 stores a group of reference image data in advance. Here, the reference image is an image that is referred to in order to identify the subject of the target image, and is an image in which an object with a known name is depicted. In addition, the reference image data is data in which the name of a known object depicted in each reference image is assigned to each reference image.

[0086] The image recognition unit 34 recognizes the subject of the target image based on the features of the target image extracted by the feature extraction unit 33 and the reference image data stored in the reference image data storage unit 35. A method of recognizing the subject will be described later. The image recognition unit 34 is an example of an "estimation unit." The output unit 36 ​​outputs the recognition result of the subject by the image recognition unit 34. For example, the output unit 36 ​​may be a display unit that displays the recognition result, a transmission unit that transmits information indicating the recognition result to another device, or a recording unit that records information indicating the recognition result on a recording medium.

[0087] 6 is a flowchart showing an example of the flow of the learning process executed by the learning device 10 A. Here, an example will be described in which the learning image data is composed of the above-mentioned three image sets (Ii, Ij, Ik).

[0088] First, the learning unit 13 generates the feature extractor 20 in an initial state (step S301).

[0089] Next, the feature extraction unit 131 applies the feature extractor 20 in the middle of training to all image data I={I1, I2,..., Im} included in the training image data group to extract features of each image, thereby generating a set of features X={x1, x2,..., xm} of all images included in the training image data group (step S302).

[0090] Next, the update unit 132 obtains a value of the loss function based on the features xi, xj, and xk corresponding to the learning image data (Ii, Ij, Ik) (step S303). The update unit 132 updates the parameters of the feature extractor 20 during learning so as to reduce the value of the loss function (step S304).

[0091] Next, the detection unit 133 determines whether or not the execution condition for the correction process is satisfied (step S305). If it is determined that the execution condition for the correction process is satisfied (step S305), the detection unit 133 performs a detection process using the feature extractor 20 in the middle of learning that has updated the parameters in step S304, and detects mislabeled data (step S306).

[0092] In addition, the teacher labels in the learning image data (Ii, Ij, Ik) are image Ij given as positive data for image Ii, and image Ik given as negative data for image Ii. As described above, in this case, learning image data in which image Ij and image Ik are swapped is detected as mislabeled data.

[0093] The correction unit 134 performs a correction process on the detected mislabeled data and changes the teacher label so that the value of the loss function becomes smaller (step S307). On the other hand, if it is determined in step S305 that the execution condition for the correction process is not satisfied (step S305-NO), the detection unit 133 skips steps S306 and S307 and proceeds to step S308.

[0094] For example, the correction unit 134 updates the learning image data group by replacing the positive data and negative data of the detected learning image data (Ii, Ij, Ik) with a set of data (Ii, Ik, Ij) as new learning image data.

[0095] Next, the learning unit 13 determines whether or not the end condition of the learning process is satisfied (step S308). If the learning unit 13 determines that the end condition of the learning process is satisfied (step S308-YES), the learning unit 13 ends the learning process, and if the learning unit 13 determines that the end condition is not satisfied (step S308-NO), the learning unit 13 returns to the process in step S302 and repeatedly executes steps S302 to S307 until the end condition is satisfied.

[0096] The flow of the learning process according to the embodiment described above is basically the same as that shown in Fig. 3. Therefore, the specific examples and modified examples supplemented in Fig. 3 are also applicable to the flowchart in Fig. 6.

[0097] <Image recognition processing> 7 is a flowchart showing an example of the flow of image recognition processing executed by the image recognition device 30. Here, it is assumed that a reference image data group R={R1, R2, ..., Rn} is pre-stored in the reference image data storage unit 35. Here, a case will be described in which the image recognition device 30 identifies the name of an object appearing in a target image Q using the reference image data group R and a feature extractor 20 generated by the learning device 10A.

[0098] First, the target image input unit 31 inputs, as target data, an image Q in which an object to be identified is captured (hereinafter, referred to as a "target image") (step S401).

[0099] Next, the feature extraction unit 33 extracts features xq of the target image Q using the trained feature extractor 20 pre-stored in the trained model storage unit 32 (step S402). Here, the feature extractor 20 pre-stored in the trained model storage unit 32 is generated in advance by the learning device 10A learning the learning image data group. Similarly, the feature extraction unit 33 extracts features XR={xr1, xr2, . . . , xrn} of all reference images included in the reference image data group R (step S403). Specifically, the feature extraction unit 33 inputs the target image Q and all reference images to the feature extractor 20, and obtains the feature amount of each image as the output.

[0100] Next, the image recognition unit 34 determines, from among all the reference images, a reference image having a feature closest to the feature of the target image (hereinafter referred to as a "similar image") based on each feature amount acquired for the target image and all the reference images (step S404). Specifically, the image recognition unit 34 identifies a feature closest to the feature xq of the target image from the feature group XR acquired for the reference image data group R, and determines the reference image from which the identified feature is extracted as a similar image. Then, based on the reference image data related to the determined similar image Rq, the image recognition unit 34 outputs the name of the object given to the similar image Rq as the name of the object appearing in the target image Q (step S405).

[0101] The feature set XR={xr1, xr2, . . . , xrn} of the reference image may be generated in advance prior to the image recognition process by the image recognition device 30, in which case step S403 may be omitted. In this case, the target image input unit 31 may be configured to input the feature set XR={xr1, xr2, . . . , xrn} of the reference image in addition to the target image.

[0102] <Device configuration> FIG. 8 is a diagram showing an example of the hardware configuration of the learning device 10 and the image recognition device 30 of the embodiment. For example, the learning device 10 and the image recognition device 30 can be realized by the hardware configuration of an information processing device 40 illustrated in FIG. 8. For example, the information processing device 40 includes a CPU (Central Processing Unit) 42, a memory 43, a storage unit 44, an input unit 45, and a display unit 46, which are connected by a bus 41. The information processing device 40 functions as the above-mentioned learning device 10 or image recognition device 30 by executing a program stored in the storage unit 44 by the CPU 42 and the memory 43. Specifically, the learning device 10 configures the learning unit 13 by executing the program, and the image recognition device 30 configures the feature extraction unit 33 and the image recognition unit 34 by executing the program.

[0103] Here, the storage unit 44 is configured using a storage device such as a magnetic hard disk device, a semiconductor storage device, etc. In addition to the above programs, the storage unit 44 stores various information for the information processing device 40 to function as the learning device 10 or the image recognition device 30, such as a learning data group, a reference image data group, and a trained model (the feature extractor 20 in this embodiment).

[0104] The input unit 45 functions as the learning data input unit 11 or the target image input unit 31 by including an input device such as a mouse, a keyboard, a trackball, a switch, a button, a joystick, or a touch panel. The input unit 45 accepts input of various operations by a user of the information processing device 40, converts the input operation into an electrical signal, and outputs it to the CPU 42. The input unit 45 may be configured to include a voice input device such as a microphone and accept input of operations by input voice. In addition, when the input unit 45 includes a touch panel, the input unit 45 may be configured integrally with the display unit 46. In addition, the input unit 45 is not limited to an input device configured as a part of the information processing device 40. For example, the input unit 45 may be configured as an input interface that inputs an electrical signal indicating an operation on the information processing device 40 from an external device configured separately from the information processing device 40.

[0105] The display unit 46 is configured using a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic EL (Electro-Luminescence) display, etc. Alternatively, the display unit 46 may be configured as an interface that connects these display devices to the display unit 46. The display unit 46 functions as the output unit 36.

[0106] All or part of the functions of the information processing device 40 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The program may be transmitted via an electric communication line.

[0107] According to the embodiment of the learning device 10 configured as described above, even if the group of training data contains an incorrect teacher label, it is possible to learn effective features with high accuracy, and by using the features extracted by the feature extractor trained in this way, it is possible to more accurately identify target data.

[0108] <Modification> In the above embodiment, the learning device 10 and the image recognition device 30 are configured as separate devices, but these devices may be configured as one device. Also, some or all of the functions of the learning device 10 or the image recognition device 30 in the embodiment may be distributed and configured in multiple devices that can communicate via a network.

[0109] A part or the whole of the learning device 10 or the image recognition device 30 in the above-mentioned embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. The above-mentioned program may be a program for realizing a part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0110] Although the learning method and estimation method when the learning data group is images have been described above, the application of the technical idea is not limited to image recognition, but can be applied to any machine learning based on teacher data and any estimation process based on the learning result. For example, the learning method and estimation method of the present embodiment can be applied to text analysis, natural language processing, voice recognition, time series data analysis, etc.

[0111] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0112] 10...Learning device, 11...Learning data input unit, 12...Learning data storage unit, 13...Learning unit, 131...Feature extraction unit, 132...Update unit, 133...Detection unit, 134...Correction unit, 14...Learned model storage unit, 20...Feature extractor, 21...Input layer, 22...First convolution layer, 23...First pooling layer, 24...Second convolution layer, 25...Second pooling layer, 26...First Fully connected layer, 27...second fully connected layer, 28...output layer, 30...image recognition device, 31...target image input unit, 32...trained model memory unit, 33...feature extraction unit, 34...image recognition unit, 35...reference image data memory unit, 36...output unit, 40...information processing device, 41...bus, 42...CPU, 43...memory, 44...memory unit, 45...input unit, 46...display unit, 100...image recognition system

Claims

1. a feature extraction unit that extracts features of the learning target data included in the learning data group using a feature extractor that is generated in advance, the learning data group being provided as a collection of learning data including a pair of learning target data and a teacher label that indicates similarity between the learning target data and other data; an update unit that calculates a value of a loss function based on features of the data to be learned and teacher labels related to the features, and updates parameters of the feature extractor so as to reduce an evaluation value based on the value of the loss function; a detection unit that detects a learning data group having an error in the teacher label as mislabeled data from the learning data group based on the feature extractor with updated parameters; A correction unit that determines a corrected teacher label by changing the teacher label paired with the mislabeled data, and corrects the teacher label of the mislabeled data to the corrected teacher label only when the value of the loss function is smaller when the corrected teacher label is used than when the teacher label is used; A learning device comprising:

2. The detection unit detects, as the mislabeled data, a group of learning data in which a value of a loss function based on the features extracted by the feature extractor whose parameters have been updated is equal to or greater than a predetermined threshold. The learning device according to claim 1 .

3. a feature extraction step of acquiring features of the learning target data included in the learning data group using a feature extractor generated in advance, the learning data group being provided as a set of learning data including a pair of learning target data and a teacher label indicating similarity between the learning target data and other data; an updating step of calculating a value of a loss function based on the features of the data to be learned and teacher labels related to the features, and updating parameters of the feature extractor so as to reduce an evaluation value based on the value of the loss function; a detection step of detecting a group of training data having an error in the teacher label as mislabeled data from the group of training data based on the feature extractor with updated parameters; A correction step of determining a corrected teacher label by changing the teacher label paired with the mislabeled data, and correcting the teacher label of the mislabeled data to the corrected teacher label only when the value of the loss function is smaller when the corrected teacher label is used than when the teacher label is used; The computer executes the learning method.

4. A program for causing a computer to function as the learning device according to claim 1 or 2.

Images