Information processing device, information processing method, and program

Abstract

To increase accuracy of learning by learning a relationship with data having low similarity to representative values of other classes, out of data in the other classes.SOLUTION: An information processing apparatus includes: extraction means which extracts feature quantities from data; leaning means which trains the extraction means so that the feature quantities of the data may be classified into classes to which the data belongs; first calculation means which calculates a first loss value on the basis of a first similarity which is a similarity between a representative value of the feature quantities of the classes obtained by the learning and the feature quantities of the target data; and second calculation means which calculates a second loss value on the basis of a second similarity which is a similarity between feature quantities having low similarity to the representative values of the classes, out of the feature quantities of the classes, and the feature quantities of the target data. The learning means is configured to perform the learning on the basis of the first loss value and the second loss value.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a technique for classifying data.

Background Art

[0002] In recent years, a face recognition method has been developed that performs authentication by collating face images to determine that they are of the same person. In face recognition, the similarity between two face images is calculated, and if the similarity is greater than or equal to a threshold value, it is recognized that they are of the same person. In order to calculate the similarity between images, a feature vector extractor for extracting a feature vector from an input image is pre-learned. The feature vector extractor is learned to extract similar feature quantity vectors from face images of the same person and dissimilar feature vectors from face images of different persons. At the time of authentication, two input images are input to the feature vector extractor, feature vectors are extracted from each input image, and it is determined whether they are of the same person based on the magnitude of the similarity between the two feature vectors. Non-Patent Document 1 discloses a face recognition method for learning a feature vector extractor by a machine learning method using a CNN (Convolutional Neural Network).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The method disclosed in Non-Patent Document 1 trains a feature vector extractor as a multi-class classification problem, assigning different class labels to each person and classifying facial images of the same person into the same class. The feature vector extractor is trained based on the similarity between the feature vectors of the training data and the center vectors of each class. Therefore, it is not possible to learn about the relationship with data from other classes that have a low similarity to the center vector, which presents a challenge in that training is difficult.

[0005] This invention was made to solve the above-mentioned problems, and aims to improve the accuracy of learning by learning about the relationship between data from other classes that have a low similarity to the representative value of the class in question. [Means for solving the problem]

[0006] Therefore, the information processing device according to the present invention comprises: an extraction means for extracting features from data; a learning means for training the extraction means so that the features of the data are classified into the class to which the data belongs; a first calculation means for calculating a first loss value based on a first similarity, which is the similarity between the representative value of the features of each class obtained by the learning and the features of the target data; and a second calculation means for calculating a second loss value based on a second similarity, which is the similarity between the features of each class that have a low similarity to the representative value of that class and the features of the target data, wherein the learning means calculates the first loss value and the second loss value, The features of the target data should be similar to the features of the data in the class to which the target data belongs, and different from the features of the data in other classes. It is characterized by performing learning. [Effects of the Invention]

[0007] According to the present invention, the accuracy of learning can be improved by learning about the relationship between data from other classes that have a low similarity to the representative value of the class in question. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram illustrating the feature space. [Figure 2] This figure shows an example of the overall configuration of the information processing device according to Embodiment 1. [Figure 3] This diagram illustrates the structure and operation of a neural network. [Figure 4] This is a flowchart of the learning process. [Figure 5] This is a flowchart showing the process for updating the list of incorrect samples. [Figure 6] This is a flowchart showing the process for selecting outlier samples. [Figure 7] This is a flowchart of the authentication process. [Figure 8] This figure shows an example of the overall configuration of the information processing device according to Embodiment 2. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the attached drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[0010] <Embodiment 1> This embodiment describes a method for training a feature vector extractor to extract feature vectors from a facial image in order to perform authentication processing on a person's facial image. A person's face is just one example of a target object. The target of the authentication process is not limited to images of people's faces, but may also be images of other target objects. Furthermore, the data is not limited to images as long as it represents the target object.

[0011] In this embodiment, a different class is assigned to each person's face image, and a feature vector extractor is trained so that face images of the same person are classified into the same class. This training is performed by defining a loss function so that the feature vectors of the training data approach the center vector of the same class as the training data in the feature space, and move away from the center vector of a different class in the feature space. The distance in the feature space can be expressed as the cosine similarity between the feature vector of the training data and the center vector of each class. As training progresses, the feature space is distorted so that the feature vectors extracted from the training data of each class cluster towards the class center.

[0012] Figures 1(a) and 1(b) are schematic diagrams representing the feature space. The region of Class 1 is represented by ellipse 10, and the region of Class 2 is represented by ellipse 20. The circles with diagonal lines (e.g., circle 12) represent feature vectors extracted from face images belonging to Class 1. The circles with diagonal lines 11, which are larger than circle 12, represent the center vector W1 of Class 1. The circles with dot patterns (e.g., circle 22) represent feature vectors extracted from face images belonging to Class 2. The circles with dot patterns 21, which are larger than circle 22, represent the center vector W2 of Class 2. The arrows 15a,b, 16a,b, 25a,b, and 26a,b indicate the direction of the forces that attempt to distort the feature space through learning.

[0013] Here, due to the influence of attributes (variations in face orientation, presence or absence of accessories such as masks, variations in lighting, etc.), there are cases where the appearance of a face image resembles a different person more than that of the same person. In such cases, the feature vector of the training data tends to be closer to the feature vector of another class than to the center vector of its own class. This is represented by feature vector x1 (circle 13) in class 1, which is far from the center vector W1, and feature vector x2 (circle 23) in class 2, which is far from the center vector W2. Feature vector x1 is close to feature vector x2. Feature vector x1 is subject to attractive forces with the center vector W1 (diagonal arrows 15a, 15b) and repulsive forces with the center vector W2 (diagonal arrows 16a, 16b). Similarly, feature vector x2 is subject to attractive forces with the center vector W2 (dotted arrows 25a, 25b) and repulsive forces with the center vector W1 (dotted arrows 26a, 26b). Because the feature space is continuous, the forces acting between feature vectors x1 and x2 and the central vector W1, and the forces acting between feature vectors x1 and x2 and the central vector W2, are mutually antagonistic, making learning difficult. Therefore, as shown by arrow 30 in Figure 1(b), it is desirable to perform learning in such a way that a force acts to separate the feature vectors x1 and x2.

[0014] Therefore, in this embodiment, a list of training data information that is far from the center vector of its own class (hereinafter referred to as "outlier samples") is maintained for each class, and outlier samples are selected from this list (hereinafter referred to as the "outlier sample list") for each class. Then, training is performed so that the feature vector of the training data is far from the center vector of a class other than the training data in question, as well as far from the feature vector of an outlier sample of a class other than the training data in question. This allows training to proceed smoothly in regions far from the center of each class.

[0015] FIG. 2 is a diagram showing an example of the overall configuration of the information processing apparatus according to the present embodiment. As shown in FIG. 2, the information processing apparatus includes a CPU 101, a first memory 103, a second memory 104, an input device 105, a display device 106, and a communication I / F 107. These components are interconnected via a bus 102. The CPU (Central Processing Unit) 101 controls the overall operation of the information processing apparatus. The first memory 103 and the second memory 104 are storage devices such as a ROM (Read Only Memory) or a hard disk, and store control programs executable by the CPU 101 and various types of data. In the present embodiment, it will be described that the first memory 103 mainly stores control programs, and the second memory 104 mainly stores various types of data.

[0016] The first memory 103 stores program modules of a learning unit 110, a first calculation unit 111, a second calculation unit 112, a third calculation unit 113, an update unit 114, a selection unit 115, and an authentication unit 116. Hereinafter, it will be described that the CPU 101 operates as each functional block by executing each program module stored in the first memory 103. Also, the processing of each step of the flowchart described later will be described as the processing of each functional block. Note that a GPU (Graphics Processing Unit) may operate as each functional block in cooperation with the CPU 101. Note that the processing of each step of the flowchart described later may be configured to be realized by hardware such as an electronic circuit.

[0017] The second memory 104 stores various types of data related to the present embodiment. The second memory 104 stores a neural network 120, a learning dataset 121, a data number 122, a first loss value 123, a second loss value 124, a third loss value 125, an outlier sample array 126, and a mixing coefficient 127. The second memory 104 also stores an authentication threshold 128, a first face image 129, a second face image 130, a first feature vector 131, a second feature vector 132, a similarity 133, and an authentication determination result 134.

[0018] The neural network 120 stores parameters such as weight parameters for constructing the neural network, which is the model to be learned. As learning progresses, these parameters are updated. The details of the neural network will be described later using FIG. 3. Note that the model to be learned is not limited to a neural network as long as it has a part for converting input data into a feature vector and is learnable. The feature vector is a feature quantity represented by a vector that can be input to the model to be learned. The learning dataset 121 stores one or more learning data. The learning data is data to be learned, and in this embodiment, it is a face image of a person. For the data to be learned, a data number for identifying the data and a class label indicating the class (correct class) to which the data belongs are added. In this embodiment, an ID (class ID) representing the person in the face image is used as the class label.

[0019] The input device 105 is a keyboard, touch panel, etc., and inputs the user's operation information to the CPU 101. The display device 106 is a liquid crystal display, etc., and displays the processing result of the information processing device, etc. under the control of the CPU 101. The communication I / F (interface) 107 communicates with an external device and transmits and receives data under the control of the CPU 101.

[0020] The learning unit 110 learns a neural network such as a CNN (Convolutional Neural Network). The learning unit 110 uses the face image of a person included in the learning data as input data and its class ID as teacher data to learn a feature vector extractor by a neural network. The learning unit 110 learns the neural network based on the loss value calculated by a predetermined loss function.

[0021] Subsequently, the structure and operation of the neural network will be described. Figure 3 shows the structure and operation of the neural network according to this embodiment. The input data 201 is a face image with class ID i. The training value 206 represents the class ID attached to the input data 201 in a predetermined format. In this embodiment, the total number of classes is c, and the training value 206 is represented in c-dimensional vector form. Specifically, it is a vector in which only the element at the class ID position of the input data 201 is 1, and the other elements are 0. In the example in Figure 3, since the class ID of the input data 201 is i, the training value 206 is a c-dimensional vector in which the i-th element (index i of the training value vector) is 1 and the other elements are 0.

[0022] The feature vector extractor 202 is constructed using parameters such as weight parameters stored in the neural network 120, and its structure is not limited, but for example, it may be a CNN consisting of multiple convolutional layers. The feature vector extractor 202 performs processing such as convolution operations and outputs a feature vector 203 from the face image of the input data 201. The feature vector 203 is a d-dimensional vector x. A fully connected layer 204 is connected to the feature vector extractor 202. The fully connected layer 204 is a classifier for classifying classes from the feature vector 203. The fully connected layer 204 is a d × c matrix constructed using weight parameters stored in the neural network 120, and in the example in Figure 3, it is represented in the form of c rows of d-dimensional vectors Wj (j ∈ [1, c]). In this embodiment, the neural network includes the feature vector extractor 202 and the fully connected layer 204, and the weight parameters of this neural network are learned by the learning unit 110 and stored in the neural network 120. The fully connected layer 204 could also be in tensor form to process multiple training data simultaneously, but for simplicity of explanation, it will be described as a matrix below.

[0023] The learning unit 110 performs an inner product operation between the feature vector 203 and the vectors Wj, which are the row components of the fully connected layer 204, to calculate the first classification result 205. The first classification result 205 is a c-dimensional vector S. Each element of vector S is represented by Sj. In this inner product calculation, the feature vector 203 and the vectors Wj, which are the row components of the fully connected layer 204, are both L2 normalized before the inner product calculation is performed. Therefore, each element Sj of the first classification result 205 is the cosine similarity between the feature vector 203 and the vectors Wj, which are the row components of the fully connected layer 204.

[0024] The learning unit 110 uses the third loss value 211 to train the element Si of the correct class i in the first classification result 205 so that it approaches 1 (angle of vector x and vector Wi is 0°). As training progresses, vector Wi approaches the average of the feature vectors 203 of all the training data belonging to class i. Therefore, each row component of the fully connected layer 204 represents the center vector of each class. In other words, each element of the first classification result 205 represents the similarity between the feature vectors 203 and the center vector of each class. Each element of the first classification result 205 is an example of the first similarity, which shows the similarity between the features of the target data and the representative values ​​of each class. Furthermore, the learning unit 110 learns to move elements Sj (j≠i) other than the correct class in the first classification result 205 toward 0 (the angle between vector x and vector Wj is 90°). In other words, learning is performed to move away from the center vector of a class other than the training data.

[0025] The first calculation unit 111 calculates a first loss value 207 based on the first classification result 205 and the target value 206. The first calculation unit 111 calculates the first loss value 207 using, for example, a softmax loss function. The above part is the same as the technology described in Non-Patent Literature 1.

[0026] In this embodiment, an outlier matrix 208 is used. The outlier matrix 208 is generated using an update unit 114 that updates the outlier list and a selection unit 115 that selects outlier samples from the outlier list. Details of the processes performed by the update unit 114 and the selection unit 115 will be described later. The outlier matrix 208 is a matrix of the same size as the fully connected layer 204. Although the outlier matrix 208 may be in tensor form to process multiple training data simultaneously, it will be described as a matrix below for simplicity. The vectors of each row component of the outlier matrix 208 are represented by Whj (j∈[1,c]). Among the row components of the outlier matrix 208, the element Whi of the correct class i is set to the center vector of that class, and the element Whj (j≠i) of other classes is set to a feature vector with low similarity to the center vector of that class.

[0027] The learning unit 110 calculates a second classification result 209 by performing an inner product calculation between the feature vector 203 and the outlier sample matrix 208 in the same manner as it did between the feature vector 203 and the fully connected layer 204. The second classification result 209 is a c-dimensional vector Sh. Each element of vector Sh is represented by Shj. In this inner product calculation, the feature vector 203 and the vectors Whj, which are the row components of the outlier sample matrix 208, are both L2 normalized before the inner product calculation is performed. Therefore, each element Shj of the second classification result 209 is the cosine similarity between the feature vector 203 and the vectors Whj, which are the row components of the outlier sample matrix 208. Here, each row component of the outlier sample matrix 208 contains feature vectors that have a low similarity to the center vector in each class, excluding the correct class i. Therefore, each element of the second classification result 209 represents the similarity between the feature vector 203 and the feature vectors of other outlier classes that are far from the center vector. Each element of the second classification result 209 is an example of a second similarity metric, which shows the similarity between the features of each class that have a low similarity to the representative value of that class and the features of the target data. The learning unit 110 uses the third loss value 211 to train the elements Shj (j≠i) of the second classification result 209 that are not of the correct class to approach 0 (the angle between vector x and vector Whj is 90°). In other words, the learning is performed to move away from outlier samples of classes other than the training data.

[0028] The second calculation unit 112 calculates a second loss value 210 based on the second classification result 209 and the training value 206. The second calculation unit 112 calculates the second loss value 210, for example, using a softmax loss function. The third calculation unit 113 calculates a third loss value 211 based on the first loss value 207 and the second loss value 210. Details of the processing of the third calculation unit 113 will be described later. The learning unit 110 updates the weight parameters stored in the neural network 120 based on the third loss value 211 calculated by the third calculation unit 113. The parameter updates are performed using methods such as backpropagation (BP).

[0029] (Learning process) Next, the learning process performed by the information processing device according to this embodiment will be described. Figure 4 is a flowchart of the learning process. In the following description of the flowchart, each step will be preceded by an "S" to omit the notation of the steps.

[0030] First, in S301, the learning unit 110 retrieves training data from the training dataset 121 in the second memory 104. The learning unit 110 stores the data number of the training data in data number 122 in the second memory 104 and sets the face image of the training data as input data 201 in Figure 3. The learning unit 110 also uses the class ID of the training data to generate a c-dimensional vector such that only the element at class ID is 1 and all other elements are 0, and sets the generated vector as the target value 206 in Figure 3. Assume that the class ID of the input data 201 is i (i ∈ [1, c], where c is the number of classes).

[0031] In S302, the feature vector extractor 202 extracts a feature vector 203 from the input data 201 set in S301. The feature vector 203 is an example of a feature. The feature vector extractor 202 uses weight parameters stored in the neural network 120 in the second memory 104. The feature vector extractor 202 is an example of an extraction means for extracting features from data. In S303, the learning unit 110 calculates the dot product of the feature vector 203 extracted in S302 and the fully connected layer 204 to calculate the first classification result 205. The weight parameters stored in the neural network 120 in the second memory 104 are used for the fully connected layer 204. As mentioned above, the center vector of each class can be obtained from the weight parameters of the fully connected layer 204, so the first classification result 205 represents the similarity between the feature vector 203 and the center vector of each class. The center vector (representative vector) of each class is an example of a representative value for each class.

[0032] In S304, the first calculation unit 111 calculates a first loss value 207 based on the first classification result 205 calculated in S303 and the target value 206 set in S301. The first calculation unit 111 is an example of a first calculation means that calculates a first loss value based on a first similarity. The calculated first loss value 207 is stored in the first loss value 123 in the second memory 104. The first loss value 207 is a loss (loss value) calculated between the first classification result 205 and the target value 206 using a predetermined loss function (e.g., a softmax loss function). For calculating the first loss value 207, a softmax loss function with a margin may be used, as described in Non-Patent Literature 1. In this embodiment, margins are not mentioned for the sake of simplicity, but a softmax function with a margin can also be applied in this embodiment. Margins can be provided in various ways, as described in Non-Patent Literature 1. The margin value is a hyperparameter and may be set using validation data (an evaluation dataset for determining the hyperparameter) to improve learning performance.

[0033] In S305, the update unit 114 updates the outlier sample array 126 held in the second memory 104 based on the first class classification result 205 calculated in S303. The outlier sample array 126 holds information about training data (outlier samples) with low similarity to the central vector of each class in list format. The outlier samples contain the feature vectors of the training data. Details of the outlier sample list update process performed in S305 will be described later in Figure 5. The outlier sample array 126 is an example of a storage means for holding features with low similarity to the representative value of each class.

[0034] In S306, the selection unit 115 selects outlier samples and generates an outlier sample matrix 208. Specifically, it selects one outlier sample for each class from the outlier sample array 126 held in the second memory 104. The feature vectors included in the outlier sample matrix 208 are selected from the outlier sample array 126 and are therefore feature vectors with low similarity to the central vector in each class. Details of the outlier sample selection process performed in S306 will be described later in Figure 6.

[0035] In S307, the learning unit 110 performs an inner product operation between the feature vector 203 extracted in S302 and the outlier sample matrix 208 generated in S306 to calculate a second classification result 209. In S308, the second calculation unit 112 calculates a second loss value 210 based on the second classification result 209 calculated in S307 and the target value 206 set in S301. The second calculation unit 112 is an example of a second calculation means that calculates a second loss value based on a second similarity. The calculated second loss value 210 is stored in the second loss value 124 in the second memory 104. The second loss value 210 is a loss (loss value) calculated between the second classification result 209 and the target value 206 using a predetermined loss function (e.g., a softmax loss function).

[0036] In the S308 process, a margined softmax loss function may be used, as explained in the S304 process. Furthermore, since the first loss value 207 is calculated using the center vectors of each class, and the second loss value 210 is calculated using outlier samples of each class, the second calculation unit 112 may set a different margin value than that used in the first calculation unit 111. For example, since outlier samples do not cluster around the center of their own class and tend to be similar to the feature vectors of other classes, the margin value used in the second calculation unit 112 may be set smaller than the margin value used in the first calculation unit 111. This is not limited to this; each margin value may be appropriately explored using validation data, etc. Also, the margined softmax loss function used in the loss function described in Non-Patent Literature 1 includes a scale parameter, which may also be used as a hyperparameter. Similar to the margin value, different values ​​may be used for this scale in the first calculation unit 111 and the second calculation unit 112.

[0037] In S309, the third calculation unit 113 calculates a third loss value 211 based on the first loss value 207 calculated in S304 and the second loss value 210 calculated in S308. Specifically, the third calculation unit 113 weights and adds the first loss value 207 and the second loss value 210 by the mixing coefficient α stored in the mixing coefficient 127 in the second memory 104. The mixing coefficient α is an example of a weighting coefficient. The third loss value 211 is calculated by the following equation (1). Third loss value = (1-α) × first loss value + α × second loss value ... (1) However, the symbols represent the following: α: Mixing coefficient (0≦α≦1) The mixing coefficient α may be a pre-set fixed value, or it may be a setting value that the user can adjust as appropriate. The calculated third loss value 211 is stored in the third loss value 125 in the second memory 104.

[0038] In S310, the learning unit 110 performs backpropagation using the third loss value 211 calculated in S309 to update the weight parameters stored in the neural network 120 in the second memory 104. In this way, the learning unit 110 learns the weight parameters of the neural network, which includes the feature vector extractor 202 and the fully connected layer 204. The backpropagation process is performed from the fully connected layer 204 toward the input layer. The outlier matrix 208 is a parameter that is not learned, and is therefore treated as a constant from the perspective of neural network learning. The learning unit 110 uses the third loss value 211 to learn the elements of the correct class in the first classification result 205 to approach 1, and the elements of the non-correct class to approach 0. That is, the learning unit 110 learns the feature vector extractor 202 so that the feature vector 203 extracted from the input data 201 is classified into the correct class. The learning unit 110 is an example of a learning means that trains an extraction means so that the features of the data are classified into the class to which the data belongs.

[0039] In S311, the learning unit 110 determines when learning is complete. If the learning unit 110 determines that learning is complete, the processing in this flowchart ends. If it determines that learning should continue, the process returns to S301. The learning unit 110 then retrieves the next learning data from the learning dataset 121 and executes the process repeatedly. In this embodiment, the determination of when learning is complete is based on whether the decrease in the third loss value 211 calculated in S309 has converged to a predetermined range, but it is not limited to this.

[0040] (Update process for the list of incorrect samples) Next, we will describe the details of the outlier sample list update process performed in S305. Figure 5 is a flowchart showing the details of the outlier sample list update process.

[0041] First, an example of the structure of the outlier sample array 126 is shown below. The outlier sample array 126 is an array with c elements (total number of classes) that holds the outlier sample list for each class. Wh_list_1=[Wh_11,Wh_12,····,Wh_1m] Wh_list_2=[Wh_21,Wh_22,····,Wh_2m] ... Wh_list_j=[Wh_j1,Wh_j2,····,Wh_jm] ... Wh_list_c=[Wh_c1,Wh_c2,····,Wh_cm] As described above, the outlier sample array 126 consists of the array elements Wh_list_1 to Wh_list_c. Wh_list_j is the outlier sample list of class j, and each element within the brackets [] represents an outlier sample that makes up the outlier sample list of class j. The number of elements in the outlier sample list is m, and the list length is m. Each element of the outlier sample list includes the data number idx of the training data, the similarity s between the center vector and feature vector x of the class to which the training data belongs, and the feature vector x. Each element of the outlier sample list is sorted in ascending order by similarity s, as will be explained in S403 below.

[0042] The outlier list update process updates the outlier list for the class to which the currently being trained input data 201 belongs. As a prerequisite, the similarity between the center vector and feature vector 203 of each class has already been calculated in S303 and is stored as the first class classification result 205. First, in S401, the update unit 114 generates an update candidate element, which is an element of the update candidate in the list of outlier samples. Specifically, it generates an element containing the above-mentioned ternary pair {idx, s, x}. The update unit 114 obtains data number 122 in the second memory 104 as data number idx. The update unit 114 also obtains the element of the correct class (here, element Si) from the first classification result 205 calculated in S303 as similarity s. The update unit 114 also obtains the feature vector 203 calculated in S302 as feature vector x.

[0043] In S402, the update unit 114 retrieves the i-th array element from the outlier sample array 126. The i-th array element is, in other words, the outlier sample list of class i. In S403, the update unit 114 combines each element of the outlier sample list of class i with the update candidate elements generated in S401 and sorts them in ascending order by similarity s. In S404, the update unit 114 updates the outlier sample list of class i with the top m elements sorted in S403. In this way, the update unit 114 adds the information of the input data 201 to the list if the similarity s of the update candidate element is lower than the similarity s of the m-th element in the outlier sample list of class i. The update unit 114 may also prioritize adding update candidate elements to the outlier sample list if their similarity s is lower than a predetermined value. The outlier sample list update process is then completed. Subsequent processing returns to the flowchart in Figure 4. The update unit 114 is an example of an update means that updates the features held in the outlier sample array based on a first similarity.

[0044] With the outlier sample list update process described above, each time training data is acquired, if there is training data with lower similarity to the class's center vector, the feature vector of that training data can be added to the outlier sample list. This allows each class's outlier sample list to hold multiple data points with low similarity to the center vector for that class.

[0045] A variation of the process for updating the list of incorrect samples is described below. As a first variation, the update unit 114 may initialize the outlier sample array 126 at each training epoch. Since the weight parameters of the feature vector extractor 202 themselves change during training, it is conceivable that the feature vector x and similarity s values ​​corresponding to the training data in the outlier sample list will become different from the values ​​calculated at the current stage as training progresses. Therefore, the outlier sample array 126 is initialized at an appropriate timing, such as at each training epoch. The outlier sample list can be initialized by replacing the feature vector x of all elements in the list with the center vector of that class. In this case, the similarity s of all elements in the outlier sample list will be 1, so the elements in the outlier sample list will be updated immediately after the start of training in the next training epoch. Also, since it is the same as the center vector of that class until it is updated, it is less likely to be a factor in calculating loss values ​​that hinder training.

[0046] As a second variation, the update unit 114 may delete elements with the same data number idx in the outlier sample list obtained in S402 if such elements include elements with the same data number idx as the update candidate elements generated in S401. Since elements in the outlier sample list include feature vectors x and similarity s calculated by the feature vector extractor 202 in the past, they differ from the values ​​calculated at the current stage. By deleting elements in the outlier sample list, it is possible to prevent elements containing feature vectors x and similarity s calculated in the past for a given training data from remaining in the outlier sample list.

[0047] As a third variation, in S403, the update unit 114 compares the feature vector x of each element in the outlier sample list with the feature vector x of the update candidate element generated in S401. If there is an element in the outlier sample list with a feature vector x similar to the feature vector x of the update candidate element, the update unit 114 adds the element with the lower similarity s among the elements compared to the outlier sample list. The element with the higher similarity s among the elements compared is not added to the outlier sample list, even if its similarity s is lower than that of the m-th element. In this way, the feature vector x of each element in the outlier sample list is not biased.

[0048] As a fourth variation, the training data may have data attributes pre-assigned, and the update unit 114 may update the list of outlier samples for each class according to the data attributes. The data attributes may include, for example, information on face orientation, information on the presence or absence of accessories such as masks, glasses, sunglasses, and hats, and lighting information. Each element of the list of outlier samples will include the data attribute in addition to the above-mentioned ternary set {idx, s, x}, and the sorting in S403 and the update in S404 will be performed on data with the same data attributes as the input data 201. This makes it possible to maintain a rich variety of attributes for the outlier samples.

[0049] (Exclusion sample selection process) Next, we will describe the details of the outlier sample selection process performed in S306. Figure 6 is a flowchart showing the details of the outlier sample selection process. In S501, the selection unit 115 sets the variable label_id to the class ID of the input data 201 currently being learned. In this embodiment, i is set. In S502, the selection unit 115 initializes the variable j to 1. The variable j is an integer loop variable representing the class ID (j ∈ [1, c]), and is incremented with each loop operation (processing in S503 to S507) to take values ​​from 1 to the total number of classes c. In S503, the selection unit 115 determines whether the variable j is equal to the variable label_id. If they are found to be equal, the process proceeds to S504; otherwise, the process proceeds to S505.

[0050] In S504, the selection unit 115 sets the vector variable Ws to the vector of the j-th row component of the fully connected layer 204. In this way, for the correct class of the training data currently being studied, the center vector of that class is set. As a result, the model is trained to increase the similarity between the feature vector of the training data currently being studied and the center vector of the class to which the training data belongs. In S505, the selection unit 115 selects the first element from the j-th array element of the outlier sample array 126 (the j-th outlier sample list) and sets the feature vector x of the selected element to the vector variable Ws. Since the outlier sample list is sorted in ascending order by similarity s in S403 of Figure 5, the first element is the element with the lowest similarity s in the outlier sample list. The selection unit 115 is an example of a selection means for selecting features for each class from the outlier sample list.

[0051] In S506, the selection unit 115 sets the vector variable Ws, which was set in S504 or S505, into the j-th row component of the outlier sample matrix 208. In S507, the selection unit 115 determines whether the vector variable Ws has been set for all classes. If it is determined that there are classes for which it has not been set, the process moves to S508, increments the variable j, and then moves to S503. If it is determined that the vector variable Ws has been set for all classes, the process returns to the flowchart in Figure 4.

[0052] Through the outlier sample selection process described above, it is possible to select the feature vector furthest from the center vector of each class from the outlier sample list of each class, and generate an outlier sample matrix with the selected feature vector set.

[0053] A variation of the outlier selection process is described below. In the S505 process, the first element of the outlier sample list is set to the vector variable Ws. However, as a first modification, the selection unit 115 does not necessarily have to select the first element; it may randomly select from the first to the mth element. This allows for even learning of various outlier samples within the outlier sample list.

[0054] As a second variation, the selection unit 115 may select an element from the list of outlier samples that has the feature vector x with the highest similarity to the feature vector 203 of the input data 201. This is because data from other classes that are similar to the feature vector 203 of the input data 201 are data for which the accuracy of classification is insufficient and should be prioritized for training.

[0055] As a third variation, the selection unit 115 may select elements from the second element onward. For example, it may select the element with the median similarity s from the outlier sample list for each class, or it may select the nth (1≦n≦m) element. Elements closer to the beginning of the outlier sample list deviate significantly from the class center, and there is a possibility that the true value attached to the data is incorrect. By avoiding the first element of the outlier sample list (the element with the lowest similarity s), it is possible to perform learning while avoiding data that may have incorrect true values, and this is expected to have the effect of stabilizing the learning process.

[0056] (Authentication process) Next, the authentication process performed by the information processing device according to this embodiment will be described. Figure 7 is a flowchart of the authentication process. In step S601, the authentication unit 116 acquires a first face image and stores it in the second memory 104 as the first face image 129. The method for acquiring the first face image may be to input an image specified by the user via the input device 105, or to receive it from an external device via the communication interface 107. In S602, the authentication unit 116 extracts feature vectors from the first face image acquired in S601 using the weight parameters of the neural network 120, which has been trained by the learning unit 110. The extracted feature vectors (hereinafter referred to as the first feature vectors) are then stored in the first feature vector 131 in the second memory 104. The first feature vectors are extracted in the same manner as the feature vector 203 is extracted from the input data 201 using the feature vector extractor 202.

[0057] In cases where a face image of a person that has been registered in advance is to be compared with an unknown image, the processes in S601-602 may be performed in advance before the execution of the process in the next step. In that case, the first face image 129 in the second memory 104 stores the face image of a person that has been registered in advance, and the first feature vector 131 in the second memory 104 stores the feature vector extracted from that face image.

[0058] In S603, the authentication unit 116 acquires a second face image and stores it in the second face image 130 in the second memory 104, similar to the process in S601. In S604, the authentication unit 116 extracts a feature vector from the second face image acquired in S603, similar to the process in S602, and stores the extracted feature vector (hereinafter referred to as the second feature vector) in the second feature vector 132 in the second memory 104. In S605, the authentication unit 116 calculates the similarity between the first feature vector extracted in S602 and the second feature vector extracted in S604, and stores the calculated similarity in the similarity 133 in the second memory 104. The similarity is, for example, the cosine similarity between the feature vectors. In S606, the authentication unit 116 reads the threshold from the authentication threshold 128 in the second memory 104, compares the similarity calculated in S605 with the threshold, and determines whether the similarity is greater than or equal to the threshold. If the authentication unit 116 determines that the similarity is greater than or equal to the threshold, the process moves to S607 and determines that the person is the real person (the person in the first face image is the same person as the person in the second face image). If the authentication unit 116 determines that the similarity is less than the threshold, the process moves to S608 and determines that the person is a different person (the person in the first face image is a different person from the person in the second face image). The authentication unit 116 stores the determination result in the authentication determination result 134 in the second memory 104. After that, the information processing device may output the determination result to the display device 106 and present it to the user, or it may transmit it to an external device via the communication I / F 107. In this way, the series of processes for authenticating a face image is completed.

[0059] According to Embodiment 1 described above, an appropriate loss value can be calculated not only for the class center of other classes but also for data far from the class center. This allows for efficient learning in regions far from the center of each class, thereby improving the accuracy of learning. Furthermore, while the center vector of each class is the average feature vector of that class, and therefore the detailed features of individual data are averaged, outlier samples are the individual data themselves and thus retain detailed features. By using such outlier samples in the learning process, learning with more detailed features can be achieved. In addition, by performing authentication processing using the feature vector extractor learned in this manner, authentication processing can be performed with high accuracy.

[0060] <Embodiment 2> In this embodiment, a method for changing the mixing coefficient α used in the third calculation unit 113 according to the progress of learning will be described. In the following, the parts that are common with Embodiment 1 will be omitted from the explanation, and the focus will be on the differences from Embodiment 2.

[0061] Figure 8 shows an example of the configuration of the information processing apparatus according to this embodiment. The configuration of this embodiment is substantially the same as that of Embodiment 1, but differs in that an adjustment unit 117 for adjusting the mixing coefficient α is added to the first memory 103. The adjustment unit 117 will be described below.

[0062] The adjustment unit 117 changes the mixing coefficient α from a minimum value to a maximum value according to the number of learning epochs. The adjustment unit 117 updates the mixing coefficient α stored in the mixing coefficient 127 each time a learning epoch is completed. The number of learning epochs is represented as Ep. The number of learning epochs Ep is stored in the second memory 104 and increases as learning progresses.

[0063] The adjustment unit 117 gradually changes the mixing coefficient α so that it increases from the minimum value to the maximum value according to the number of learning epochs Ep. Here, the minimum value of the mixing coefficient is denoted as αmin and the maximum value as αmax. αmin and αmax are hyperparameters, and 0 ≤ αmin ≤ αmax ≤ 1. αmin and αmax are set in advance and stored in the second memory 104. The number of learning epochs at the start of the mixing coefficient α update process is denoted as Ep_start, and the number of learning epochs at the stop of the mixing coefficient α update process is denoted as Ep_end. Ep_start and Ep_end are also hyperparameters, set in advance and stored in the second memory 104. The values ​​of these hyperparameters may be determined by searching using validation data, etc.

[0064] The adjustment unit 117 changes the mixing coefficient α according to the following equation (2).

number

[0065] By using equation (2) above, the contribution of outliers to the loss value can be reduced in the early stages of learning (when the number of learning epochs is small), and increased in the later stages of learning (when the number of epochs is large). It is expected that the learning data will consist mostly of data that is easy to learn to resemble the class-centered vector, and few data that is difficult to learn to resemble the class-centered vector. Therefore, in the first half of learning, the learning will be mainly based on the majority data, and in the second half of learning, the influence of minority data that does not resemble the class-centered vector can be gradually increased. The adjustment unit 117 is an example of an adjustment means that adjusts the weighting coefficients used to calculate the third loss value so as to gradually decrease the contribution of the first loss value to the third loss value and gradually increase the contribution of the second loss value to the third loss value.

[0066] According to the above embodiment 2, it becomes possible to adjust the influence of learning from outlier samples so that it gradually increases in accordance with the progress of learning. This makes it possible to advance learning efficiently and improve the accuracy of learning.

[0067] Modified examples of this embodiment are described below. In the above, the progress of learning is determined by the number of learning epochs, but as a first modification, the adjustment unit 117 may calculate the amount of change in the loss value during learning at each learning step and use the calculation result to determine the progress of learning. The amount of change in the loss value is the difference between the loss value in the previous step and the loss value in the current step. The loss value here is, for example, the third loss value 211 calculated by the third calculation unit 113, and a moving average over several steps may be taken. The adjustment unit 117 sets the mixing coefficient α to αmin in the early stages of learning, and gradually increases it up to αmax when the amount of change in the loss value falls below a predetermined value. For example, the value from αmin to αmax is increased in increments of 10. When the mixing coefficient α is changed by one step, the balance of learning changes, so the amount of change in the loss value is large for a while, but when the amount of change falls below the predetermined value again, the mixing coefficient is changed by another step.

[0068] As a second modification, the adjustment unit 117 may calculate the amount of change in the class-centered vector at each learning step and use the calculation result to grasp the progress of learning. Specifically, each time the weight parameters of the fully connected layer 204 are updated by learning, the adjustment unit 117 calculates the amount of change due to the update of the weight parameters, i.e., the difference from the value in the previous step. Similar to the first modification, a moving average over several steps may be taken. The adjustment unit 117 sets the mixing coefficient α to αmin in the early stages of learning, and gradually increases it to αmax when the amount of change due to the update of the weight parameters of the fully connected layer 204 falls below a predetermined value.

[0069] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

Claims

1. An extraction method for extracting features from data, A learning means that trains the extraction means so that the features of the data are classified into the class to which the data belongs, A first calculation means calculates a first loss value based on the representative value of the features of each class obtained through the learning process and a first similarity, which is the similarity between the features of the target data. The system includes a second calculation means for calculating a second loss value based on a second similarity, which is the similarity between the feature of each class that has a low similarity to the representative value of that class and the feature of the target data. The learning means is characterized in that, based on the first loss value and the second loss value, it learns so that the features of the target data approach the features of the data in the class to which the target data belongs, and move away from the features of the data in a class other than the class to which the target data belongs.

2. A storage means for storing the feature quantities that have low similarity to the representative value of the class for each class, Based on the first similarity, an update means updates the feature quantities held by the holding means with the feature quantities of the target data, It further possesses, The information processing apparatus according to claim 1, characterized in that the second calculation means calculates the second loss value using the feature quantities held by the holding means.

3. The storage means maintains a list of features with low similarity to the representative value of the class, for each class. The system further includes a selection means for selecting the feature quantities for each class from the aforementioned list, The information processing apparatus according to claim 2, characterized in that the second calculation means calculates the second loss value using the feature quantity selected by the selection means.

4. The information processing apparatus according to claim 3, wherein the update means adds the feature quantities of the target data to the list when the first similarity for the class to which the target data belongs is lower than the similarity between the feature quantities held in association with the class by the holding means and the representative value of the class.

5. The information processing apparatus according to claim 3, wherein the selection means selects the feature with the lowest similarity to the representative value of the class from among the feature quantities included in the list.

6. The information processing apparatus according to claim 3, wherein the selection means selects from among the features included in the list the feature whose similarity to the representative value of the class is second to last.

7. The information processing apparatus according to claim 3, characterized in that the selection means randomly selects the feature quantities included in the list.

8. The information processing apparatus according to claim 3, wherein the selection means selects a feature from among the feature included in the list that is similar to the feature of the target data.

9. The information processing apparatus according to claim 4, characterized in that the update means does not add any of the similar feature quantities to the list when the feature quantities held by the holding means in association with the class to which the target data belongs are similar to the feature quantities of the target data.

10. The aforementioned data has attributes attached to it. The storage means stores the feature quantities for each attribute, The information processing apparatus according to any one of claims 2 to 9, characterized in that the updating means updates the feature quantities extracted from the data having the same attributes as the target data with the feature quantities of the target data.

11. The information processing apparatus according to any one of claims 1 to 10, characterized in that the learning means performs learning based on a third loss value obtained by weighting and adding the first loss value and the second loss value.

12. The information processing apparatus according to claim 11, further comprising an adjustment means for adjusting the weighting coefficient used to calculate the third loss value, such that as learning progresses, the contribution of the first loss value to the third loss value is gradually reduced and the contribution of the second loss value to the third loss value is gradually increased.

13. The information processing apparatus according to claim 12, characterized in that the adjustment means adjusts the weighting coefficients based on the number of learning epochs.

14. The information processing apparatus according to claim 12, characterized in that the adjustment means adjusts the weighting coefficient based on the amount of change in the third loss value during learning.

15. The information processing apparatus according to claim 12, characterized in that the adjustment means adjusts the weighting coefficient based on the amount of change in the representative value during learning.

16. The information processing apparatus according to any one of claims 1 to 15, characterized in that the feature quantity is a feature vector, the representative value is a representative vector of the class, and the similarity is calculated by the cosine similarity between the vectors.

17. The learning means performs learning using a neural network, The information processing device according to any one of claims 1 to 16, characterized in that it obtains the representative value for each class from the weight parameters of the neural network.

18. The information processing apparatus according to any one of claims 1 to 17, characterized in that the data is an image of the target object.

19. A learning process involves training an extraction means for extracting features from data so that the features of the data are classified into the class to which the data belongs. A first calculation step involves calculating a first loss value based on the representative values ​​of the features of each class obtained through the learning process and a first similarity score which is the similarity between the features of the target data. Among the features of each class, the feature with low similarity to the representative value of that class, The process includes a second calculation step of calculating a second loss value based on a second similarity, which is the similarity between the target data and the aforementioned feature quantities, The information processing method is characterized in that, in the learning process, learning is performed such that the features of the target data approach the features of the data in the class to which the target data belongs, and move away from the features of the data in a class other than the class to which the target data belongs, based on the first loss value and the second loss value.

20. A program for causing a computer to function as one of the means of the information processing apparatus according to any one of claims 1 to 18.

Images