Machine learning device, machine learning method, and machine learning program

Abstract

To provide a machine learning technique configured to facilitate conversion of loss and reduce learning time.SOLUTION: A machine learning apparatus 200 continuously learns new classes smaller in number than basic classes. A basic class feature extraction unit 50 extracts feature vectors of the basic classes. A new class feature extraction unit 52 extracts feature vectors of the new classes. A mixed feature calculation unit 60 mixes the feature vectors of the basic classes and the feature vectors of the new classes to calculate mixed feature vectors of the basic classes and the new classes. A learning unit 80 classifies query samples of a query set on the basis of distances between positions of the mixed feature vectors of the query samples of the query set on a projection space and positions of classification weight vectors of the classes, to learn classification weight vectors of the new classes so as to minimize the loss of the classification.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to machine learning techniques. [Background technology]

[0002] Humans can learn new knowledge through long-term experience and retain previous knowledge. However, the knowledge of a convolutional neural network (CNN) depends on the dataset used for training, and adapting to changes in the data distribution requires retraining the CNN parameters for the entire dataset. As a CNN learns new tasks, its estimation accuracy for previous tasks decreases. Thus, continuous training of a CNN inevitably leads to catastrophic forgetting, in which the learning results of previous tasks are forgotten while learning a new task.

[0003] Incremental learning or continual learning has been proposed as a method to avoid fatal forgetting. Incremental learning is a learning method in which, when a new task or new data arises, the model is improved and learned, rather than learning from scratch.

[0004] On the other hand, since new tasks often require only a small amount of sample data, few-shot learning has been proposed as a method for efficiently learning with a small amount of training data. In few-shot learning, new tasks are learned using a small amount of different parameters, without re-learning previously learned parameters.

[0005] A method called incremental few-shot learning (IFSL) has been proposed, which combines continuous learning, which learns new classes without fatal forgetting of the learning results of base classes, with few-shot learning, which learns new classes that are fewer in number than the base classes (Non-Patent Document 1). In incremental few-shot learning, base classes can be learned from a large dataset, and new classes can be learned from a small amount of sample data. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Yoon, SW, Kim, DY, Seo, J., & Moon, J. (2020, November). XtarNet: Learning to extract task-adaptive representation for incremental few-shot learning. In International Conference on Machine Learning (pp. 10852-10860). PMLR. Summary of the Invention [Problem to be solved by the invention]

[0007] An example of a continuous few-shot learning method is XtarNet, described in Non-Patent Document 1. XtarNet learns to extract task-adaptive representations (TARs) in continuous few-shot learning, but the meta-learning for extraction has the problem that the loss is difficult to converge and learning takes a long time.

[0008] The present invention has been made in view of these circumstances, and its purpose is to provide a machine learning technique that makes it easy for losses to converge and can shorten the learning time. [Means for solving the problem]

[0009] In order to solve the above problem, a machine learning device according to one embodiment of the present invention is a machine learning device that continuously learns a smaller number of new classes than base classes, and includes: a base class feature extraction unit that extracts feature vectors of the base classes; a new class feature extraction unit that extracts feature vectors of the new classes; a mixed feature calculation unit that mixes the feature vectors of the base classes and the feature vectors of the new classes to calculate mixed feature vectors of the base classes and the new classes; and a learning unit that classifies query samples of the query set based on the distance between the position of the mixed feature vector of the query samples of the query set and the position of the classification weight vector of each class in a projection space, and learns the classification weight vector of the new class so as to minimize classification loss.

[0010] Another aspect of the present invention is a machine learning method for continuously learning a smaller number of new classes than base classes, the method including: a base class feature extraction step for extracting feature vectors of the base classes; a new class feature extraction step for extracting feature vectors of the new classes; a mixed feature calculation step for mixing the feature vectors of the base classes and the feature vectors of the new classes to calculate mixed feature vectors of the base classes and the new classes; and a learning step for classifying query samples of the query set based on a distance between a position of the mixed feature vector of the query samples of the query set and a position of a classification weight vector of each class in a projection space, and learning a classification weight vector of the new class so as to minimize a classification loss.

[0011] Any combination of the above components, and any transformation of the present invention into a method, device, system, recording medium, computer program, etc., are also valid aspects of the present invention. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide a machine learning technique that makes it easy for losses to converge and shortens the learning time. [Brief explanation of the drawings]

[0013] [Figure 1A] FIG. 10 is a diagram illustrating the configuration of a pre-training module. [Figure 1B] FIG. 10 is a diagram illustrating the configuration of a continuous few-shot learning module. [Figure 1C] FIG. 1 is a diagram illustrating episode-based training. [Figure 2A] FIG. 10 is a diagram illustrating a configuration for generating a task-specific mixture weight vector for calculating a task-adaptive representation from a support set. [Figure 2B] FIG. 10 is a diagram illustrating a configuration for calculating a task-adaptive representation from a support set and generating a classification weight vector set W based on the task-adaptive representation. [Figure 3] FIG. 10 is a diagram illustrating a configuration for calculating a task-adaptive representation from a query set, classifying a query sample based on the task-adaptive representation and a task-adjusted classification weight vector set, and minimizing classification loss. [Figure 4] FIG. 1 is a conceptual diagram of a projection space. [Figure 5] 5(a) to 5(c) are diagrams illustrating a conventional learning procedure in an episode format. [Figure 6] 1 is a configuration diagram of a machine learning device according to a first embodiment of the present invention. [Figure 7] 7(a) to 7(c) are diagrams illustrating the learning procedure in the episode format according to the first embodiment. [Figure 8] 8(a) to 8(c) are diagrams illustrating a conventional procedure for calculating the loss for a query sample. [Figure 9] 1 is a flowchart illustrating a conventional procedure for calculating a loss for a query sample. [Figure 10] FIG. 10 is a configuration diagram of a machine learning device according to a second embodiment of the present invention. [Figure 11] 11(a) to 11(c) are diagrams illustrating a loss calculation procedure for a query sample according to the second embodiment. [Figure 12]10 is a flowchart showing a procedure for calculating a loss for a query sample according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] First, we provide an overview of continuous few-shot learning with XtarNet. XtarNet learns to extract task-adaptive representations (TAR). First, it uses a backbone network pre-trained on a dataset of base classes to obtain features of the base classes. Second, it uses an additional module meta-trained across episodes of the novel class to obtain features of the novel class. The mixture of features of the base classes and features of the novel class is called a task-adaptive representation (TAR). The base class and novel class classifiers use this TAR to quickly adapt to the given task and perform the classification task.

[0015] The learning procedure of XtarNet will be outlined with reference to FIGS. 1A to 1C.

[0016] 1A is a diagram illustrating the configuration of a pre-training module 20. The pre-training module 20 includes a backbone CNN 22 and basic class classification weights 24.

[0017] The base class dataset 10 includes N samples. An example of a sample is an image, but is not limited to this. The backbone CNN 22 is a convolutional neural network that pre-trains the base class dataset 10. The base class classification weights 24 are the weight vector W of the base class classifier. base and shows the average feature values ​​of the samples in the dataset 10 of the basic class.

[0018] In training stage 1, a backbone CNN 22 is pre-trained with a dataset 10 of base classes.

[0019] FIG. 1B is a diagram illustrating the configuration of the continuous few-shot learning module 100. The continuous few-shot learning module 100 is obtained by adding a meta module group 30 and a new class classification weight 34 to the pre-training module 20 in FIG. 1A. The meta module group 30 includes three multilayer neural networks, which will be described later, and performs post-training on a new class dataset. The number of samples included in the new class dataset is smaller than the number of samples included in the base class dataset. The new class classification weight 34 is a weight vector W of the classifier for the new class. novel and indicates the average feature value of the samples in the dataset of the new class.

[0020] In the learning stage 2, a set of meta-modules 30 is trained episodically based on the pre-trained modules 20.

[0021] FIG. 1C is a diagram illustrating episodic training. Episodic training includes a meta-training stage and a test stage. The meta-training stage is performed for each episode, and the meta-module set 30 and new classification weights 34 are updated. The test stage performs a classification test using the meta-module set 30 and new classification weights 34 updated in the meta-training stage.

[0022] Each episode consists of a support set S and a query set Q. The support set S consists of a new class dataset 12, and the query set Q consists of a base class dataset 14 and a new class dataset 16. In the learning stage 2, in each episode, query samples of both the base class and new class included in the query set Q are classified based on the support samples in the given support set S, and the parameters of the meta-module group 30 and the new class classification weights 34 are updated to minimize the classification loss.

[0023] The configuration for processing the support set S in XtarNet will be described with reference to Figures 2A and 2B, and the configuration and learning process for processing the query set Q in XtarNet will be described with reference to Figure 3.

[0024] In addition to the backbone CNN22, XtarNet uses the following three different meta-learnable modules as a meta-module set30: (1) MetaCNN: A neural network that extracts features from new classes (2) MergeNet: A neural network that combines features of base classes with features of new classes. (3) TconNet: A neural network that adjusts the weights of the classifier.

[0025] Figure 2A shows the task-specific mixture weight vector ω for computing the task-adaptive representation TAR from the support set S. pre and ω meta FIG. 10 is a diagram illustrating a configuration for generating

[0026] The support set S includes the dataset 12 of the new class. Each support sample in the support set S is input to the backbone CNN 22. The backbone CNN 22 processes the support sample to output a feature vector of the base class (referred to as a "base feature vector"), which is supplied to the averaging unit 23. The averaging unit 23 averages the base feature vectors output by the backbone CNN 22 for all support samples to calculate an average base feature vector, which is input to the MergeNet 36.

[0027] The output of the hidden layer of the backbone CNN 22 is input to MetaCNN 32. MetaCNN 32 processes the output of the hidden layer of the backbone CNN 22 to output a feature vector of a new class (referred to as a "new feature vector") and supplies it to the averaging unit 33. The averaging unit 33 averages the new feature vector output by MetaCNN 32 over all support samples to calculate an average new feature vector, which is input to MergeNet 36.

[0028] MergeNet36 processes the average basic feature vector and the average novel feature vector through a neural network to calculate a task-specific mixture weight vector ω to calculate the task-adaptive representation TAR. pre and ω meta Output.

[0029] The backbone CNN22 is a basic feature vector extractor f that extracts basic feature vectors for the input x. θ It operates as follows: for an input x, it generates a basic feature vector f θ (x) is output. The intermediate layer output of the backbone CNN22 for the input x is a θ (x). MetaCNN32 uses the intermediate layer output a θ It acts as a new feature vector extractor g that extracts a new feature vector for (x), and the intermediate layer output a θ (x) to create a new feature vector g(a θ (x)).

[0030] FIG. 2B is a diagram illustrating a configuration for calculating a task-adaptive representation TAR from a support set S and generating a classification weight vector set W based on the task-adaptive representation TAR.

[0031] The vector product calculator 25 calculates the basic feature vector f output from the backbone CNN 22 for each support sample x in the support set S. θ (x) and the mixed weight vector ω output from MergeNet36 pre The product of each element between the vectors is calculated and provided to the vector sum calculator 37.

[0032] The vector multiplication calculator 35 calculates the intermediate layer output a of the backbone CNN 22 for each support sample x of the support set S. θ For (x), the new feature vector g(a θ (x)) and the mixture weight vector ω output from MergeNet36 meta The product of each element between the vectors is calculated and provided to the vector sum calculator 37.

[0033] The vector sum calculator 37 calculates the basic feature vector f θ (x) and the mixture weight vector ω pre and the new feature vector g(a θ (x)) and the mixture weight vector ω meta The vector sum of the product of these is calculated, and output as the task-adaptive representation TAR of each support sample x in the support set S, which is then provided to the TconNet 38 and the projection space construction unit 40. The task-adaptive representation TAR is a mixed feature vector that combines the basic feature vector and the new feature vector.

[0034] The formula for calculating the task adaptation representation (TAR) is as follows, where the product of each vector component is represented by ×. TAR=ω pre ×f θ (x)+ω meta ×g(a θ (x)) The formula for calculating the task adaptive representation TAR is to calculate the sum of the element-wise products between the mixture weight vector and the feature vector. The task adaptive representation TAR is calculated for each support sample in the support set S.

[0035] TconNet38 uses a classification weight vector set W=[W base ,W novel ] and uses the task-adaptive representation TAR of each support sample to generate a task-adjusted classification weight vector set W * Output.

[0036] The projection space construction unit 40 calculates the average {C k} and W after task adjustment * Construct a task-adaptive projection space M such that M is consistent on the projection space M.

[0037] Figure 3 shows the task-adaptive representation TAR calculated from the query set Q, and the task-adaptive representation TAR and the task-adjusted classification weight vector set W * FIG. 10 is a diagram illustrating a configuration for classifying a query sample based on the .times. ...

[0038] The vector product calculator 25 calculates the basic feature vector f output from the backbone CNN 22 for each query sample x in the query set Q. θ (x) and the mixed weight vector ω output from MergeNet36 pre The product of each element between the vectors is calculated and provided to the vector sum calculator 37.

[0039] The vector multiplication unit 35 calculates the intermediate layer output a of the backbone CNN 22 for each query sample x in the query set Q. θ For (x), the new feature vector g(a θ (x)) and the mixture weight vector ω output from MergeNet36 meta The product of each element between the vectors is calculated and provided to the vector sum calculator 37.

[0040] The vector sum calculator 37 calculates the basic feature vector f θ (x) and the mixture weight vector ω pre and the new feature vector g(a θ (x)) and the mixture weight vector ω meta The vector sum of the product of these is calculated, and the result is output as a task-adaptive representation TAR of each query sample x in the query set Q, which is then provided to the projection space query classifier 42.

[0041] The task-adjusted classification weight vector set W output by TconNet38 * is input to the projection space query classifier 42.

[0042] The projection space query classification unit 42 calculates the Euclidean distance between the position of the task-adaptive representation TAR calculated for each query sample in the query set Q and the position of the mean feature vector of the class to be classified in the projection space M, and classifies the query sample into the closest class. Here, the projection space construction unit 40 works to determine whether the mean position of the class to be classified in the projection space M is the same as the task-adjusted classification weight vector set W * Note that this is consistent with

[0043] The loss optimization unit 44 evaluates the loss of classification of the query sample using a cross-entropy function, and proceeds with learning so that the classification result of the query set Q approaches the correct answer and the classification loss is minimized. As a result, the position of the task-adapted representation TAR calculated for the query sample and the position of the average feature vector of the class to be classified, i.e., the classification weight vector set W after task adjustment, are calculated. * The learnable parameters of MetaCNN32, MergeNet36, and TconNet38 and the new classification weights W are used to minimize the distance between the positions of novel will be updated.

[0044] FIG. 4 is a conceptual diagram of the projection space M. The reference positions of the 200 basic classes B1 to B200 (the basic class classification weights W after task adjustment) base * ), the reference positions of the five new classes N1 to N5 (the new class classification weights W novel * ), and the task-adaptive representation TAR of the query samples in the query set Q are projected onto the projection space M, which functions as a joint classification space. For convenience, the base classes B11 to B190 are not shown in the figure.

[0045] The loss optimization unit 44 estimates the probability distribution of each class based on the Euclidean distance between the position of the task-adaptive representation TAR of the query sample and the average feature vector of each of the 205 classes, including the basic classes and new classes, in the projection space M, calculates the loss of class classification using a cross-entropy function, and minimizes the loss.

[0046] Next, problems to be solved and means for solving them will be described for the first embodiment of the present invention.

[0047] Figures 5(a) to 5(c) are diagrams illustrating a conventional episodic learning procedure. As shown in Figure 5(a), in Episode 1, 205 classes are the target classes for classification, which are a combination of 200 basic classes B1 to B200 and five new classes N1 to N5. As shown in Figure 5(b), in Episode 2, 205 classes are the target classes for classification, which are a combination of 200 basic classes B1 to B200 and five new classes N6 to N10. As shown in Figure 5(c), in Episode 3, 205 classes are the target classes for classification, which are a combination of 200 basic classes B1 to B200 and five new classes N11 to N15.

[0048] As such, in conventional learning, the number of classes to be classified for each episode is 205. Because all classes are classified, the loss expressed by the cross-entropy function is difficult to converge, and the time required to calculate the Euclidean distance for all classes and estimate the probability distribution is time-consuming, resulting in an overall long learning time.

[0049] 6 is a configuration diagram of a machine learning device 200 according to the first embodiment of the present invention. Here, the description will focus on the configuration that is added to XtarNet, omitting the description of the configuration that is common to XtarNet as appropriate.

[0050] The machine learning device 200 includes a base class feature extraction unit 50 , a new class feature extraction unit 52 , a mixed feature calculation unit 60 , an adjustment unit 70 , a learning unit 80 , a weight selection unit 90 , and a base class label information storage unit 92 .

[0051] A query set Q consisting of a basic class dataset 14 and a new class dataset 16 is input to a basic class feature extraction unit 50. The basic class feature extraction unit 50 is, for example, a backbone CNN 22. The basic class feature extraction unit 50 extracts and outputs a basic feature vector of each query sample in the query set Q.

[0052] The new class feature extraction unit 52 receives as input the intermediate output of the base class feature extraction unit 50. An example of the new class feature extraction unit 52 is MetaCNN 32. The new class feature extraction unit 52 extracts and outputs a new feature vector for each query sample in the query set Q.

[0053] The mixed feature calculation unit 60 calculates a mixed feature vector as a task adaptive representation TAR by mixing the basic feature vector and the new feature vector of each query sample, and provides the mixed feature vector to the adjustment unit 70 and the learning unit 80. The mixed feature calculation unit 60 is, for example, MergeNet36.

[0054] The adjustment unit 70 calculates a task-adjusted classification weight vector set W using the task-adaptive representation TAR of each query sample. * and provides it to the weight selection unit 90. The adjustment unit 70 is, for example, a TconNet 38.

[0055] In meta-learning, labels are assigned to the base classes of the query set Q. The base class label information storage unit 92 stores label information assigned to the base classes selected for the query set Q of each episode, and provides the label information of the base classes to the weight selection unit 90 for each episode.

[0056] The weight selection unit 90 selects the classification weight vector set W after task adjustment output from the adjustment unit 70 in each episode. * From the,weights of the classifiers of the base classes corresponding to the label information of the base classes selected in the query set Q are selected, and the selected classifier weights are projected onto the projection space M.

[0057] The training unit 80 performs training to classify the query sample based on the distance between the position of the task-adaptive representation TAR of the query sample and the weight of the selected classifier in the projection space M, and minimizes the classification loss. The training unit 80 is, for example, the projection space query classifier 42 and the loss optimizer 44.

[0058] 7(a) to 7(c) are diagrams illustrating the learning procedure of the episode format according to the first embodiment. In meta-learning, labels are assigned to the base classes of the query set Q. Using the label information of these base classes, a predetermined number of base classes selected as the query set Q are sequentially added and processed for each episode.

[0059] As shown in Fig. 7(a), in Episode 1, the five basic classes B1 to B5 and the five new classes N1 to N5 selected in the query set of Episode 1 are projected onto the projection space M. In Episode 1, the 10 classes to be classified are the five basic classes B1 to B5 and the five new classes N1 to N5.

[0060] As shown in Figure 7(b), in Episode 2, in addition to the five basic classes B1 to B5 selected in the query set of Episode 1, five basic classes B6 to B10 and five new classes N6 to N10 newly selected in the query set of Episode 2 are projected onto the projection space M. In Episode 2, the 15 classes to be classified are the 10 basic classes B1 to B10 and the five new classes N6 to N10.

[0061] As shown in Figure 7(c), in Episode 3, in addition to the 10 basic classes B1 to B10 selected in the query sets of Episodes 1 and 2, five basic classes B11 to B15 and five new classes N11 to N15 newly selected in the query set of Episode 3 are projected onto the projection space M. In Episode 3, the 20 classes to be classified are the 15 basic classes B1 to B15 and the five new classes N11 to N15.

[0062] 7(a) to 7(c), for convenience of explanation, the positions of the classes to be classified in the projection space M are shown as if they have not moved at all, but it should be noted that in reality the positions of the classes to be classified change with the learning of each episode. Also, for convenience of explanation, it has been stated that five basic classes selected for the query set are added for each episode, but in reality, new basic classes that have not previously appeared in the query set are added, so it should be noted that five classes are not necessarily always added.

[0063] In this way, instead of projecting all of the basic classes B1 to B200 onto the projection space M, by sequentially adding a predetermined number of basic classes selected for the query set (for example, the same number as the number of new classes selected for the query set, here 5), the number of classes to be classified can be reduced during the period until all of the basic classes are projected, making it easier for the loss to converge and shortening the learning time.

[0064] Next, problems to be solved and means for solving them will be described for the second embodiment of the present invention.

[0065] 8(a) to 8(c) are diagrams illustrating a conventional loss calculation procedure for query samples. As shown in FIG. 8(a), query sample 1 has 205 classes, which are a combination of 200 basic classes B1 to B200 and five new classes N1 to N5, as classification target classes. As shown in FIG. 8(b), query sample 2 has 205 classes, which are a combination of 200 basic classes B1 to B200 and five new classes N6 to N10, as classification target classes. As shown in FIG. 8(c), query sample 3 has 205 classes, which are a combination of 200 basic classes B1 to B200 and five new classes N11 to N15, as classification target classes.

[0066] As such, with conventional loss calculations, the number of classes to be classified for each query sample in a given episode is 205. Because the query loss calculation covers all classes, classes that are far from the task-adaptive representation TAR of the query sample, i.e., have low relevance, are also taken into account in the calculation, which can lead to a decrease in classification accuracy. Another issue is that the loss is difficult to converge, and training takes a long time.

[0067] Figure 9 is a flowchart showing the conventional loss calculation procedure for a query sample. The task adaptive representation TAR of the query sample and the weights W of the classifiers for all classes are used. * Project the task-adaptive representation TAR of the query sample onto the projection space M (S10). The weights W of the classifiers for all classes are calculated as follows: * The Euclidean distance between the query sample and the Euclidean distance is calculated (S20). The probability distribution of all classes is estimated according to the Euclidean distance (S30). The probability distribution of all classes is used to calculate the cross-entropy loss for the class classification of the query sample (S40).

[0068] 10 is a configuration diagram of a machine learning device 210 according to the second embodiment of the present invention. Here, the description will focus on the configuration that is added to XtarNet, omitting the description of the configuration that is common to XtarNet as appropriate.

[0069] The machine learning device 210 includes a base class feature extraction unit 50 , a new class feature extraction unit 52 , a mixed feature calculation unit 60 , an adjustment unit 70 , a learning unit 80 , and a neighboring class selection unit 94 .

[0070] A query set Q consisting of a basic class dataset 14 and a new class dataset 16 is input to a basic class feature extraction unit 50. The basic class feature extraction unit 50 is, for example, a backbone CNN 22. The basic class feature extraction unit 50 extracts and outputs a basic feature vector of each query sample in the query set Q.

[0071] The new class feature extraction unit 52 receives as input the intermediate output of the base class feature extraction unit 50. An example of the new class feature extraction unit 52 is MetaCNN 32. The new class feature extraction unit 52 extracts and outputs a new feature vector for each query sample in the query set Q.

[0072] The mixed feature calculation unit 60 mixes the basic feature vector and the new feature vector of each query sample to calculate a mixed feature vector as a task-adaptive representation TAR, and provides the mixed feature vector to the adjustment unit 70, the neighbor class selection unit 94, and the learning unit 80. The mixed feature calculation unit 60 is, for example, MergeNet36.

[0073] The adjustment unit 70 calculates a task-adjusted classification weight vector set W using the task-adaptive representation TAR of each query sample. * and provides it to the neighborhood class selection unit 94. The adjustment unit 70 is, for example, a TconNet 38.

[0074] The neighbor class selection unit 94 selects the task-adaptive representation TAR of the query sample and the task-adjusted classification weight vector set W for all classes in the projection space M. * Based on the Euclidean distance with the task adaptive representation TAR of the query sample, a predetermined number of classes that are within a predetermined distance from the position of the task adaptive representation TAR of the query sample are selected as neighboring classes, and the weights of the classifiers of the selected predetermined number of neighboring classes are provided to the learning unit 80.

[0075] If the classes within a predetermined distance from the position of the task-adaptive representation TAR of the query sample on the projection space M do not include a class with a correct label, the nearby class selection unit 94 expands the target range and selects nearby classes until the correct class is included.

[0076] The training unit 80 performs training to classify the query sample based on the distance between the position of the task-adaptive representation TAR of the query sample and the weight of the selected classifier in the projection space M, and minimizes the classification loss. The training unit 80 is, for example, the projection space query classifier 42 and the loss optimizer 44.

[0077] 11(a) to 11(c) are diagrams illustrating a loss calculation procedure for a query sample according to the second embodiment.

[0078] As shown in FIG. 11(a), for query sample 1, five neighboring classes B198, B3, N3, B13, and N4, which are close to the TAR of query sample 1, are selected as target classes for loss calculation.

[0079] As shown in FIG. 11(b), for query sample 2, five neighboring classes B198, N3, B9, B200, and B13, which are close to the TAR of query sample 2, are selected as target classes for loss calculation.

[0080] As shown in Fig. 11(c), for query sample 3, the correct class of query sample 3 is not included in the five neighboring classes that are close to the TAR of query sample 3, so the target range is expanded until the correct class is included. In this example, the correct class first appears in the seventh-closest class from the TAR, so the seven neighboring classes B11, B2, B197, B8, B198, B3, and N3 are used as target classes for loss calculation.

[0081] In this way, we select classes that are close to the task-adaptive representation TAR of the query sample, i.e., highly relevant, and calculate the classification loss for the selected classes. This improves the classification accuracy of the query set and also makes it easier for the loss to converge by reducing the number of target classes for loss calculation.

[0082] 12 is a flowchart showing the procedure for calculating the loss for a query sample according to the second embodiment. * is projected onto the projection space M (S50). The task adaptive representation TAR of the query sample and the classifier weights W for all classes are * The Euclidean distance between the two is calculated (S60).

[0083] A predetermined number of classes in the neighborhood of the task-adaptive representation TAR of the query sample are selected (S70). If the correct class is included in the selected classes (Y in S80), proceed to step S100. If the correct class is not included in the selected classes (N in S80), expand the neighborhood range until the correct class is included and select neighboring classes (S90), and proceed to step S100.

[0084] The probability distribution of the selected class is estimated according to the Euclidean distance (S100).The probability distribution of the selected class is used to calculate the cross-entropy loss for the classification of the query sample (S110).

[0085] The various processes of the machine learning devices 200, 210 described above can be realized not only as devices using hardware such as a CPU and memory, but also as firmware stored in a ROM (read-only memory) or flash memory, or software for a computer, etc. The firmware program or software program can be provided by being recorded on a computer-readable recording medium, or can be transmitted and received with a server via a wired or wireless network, or can be transmitted and received as data broadcasting via terrestrial or satellite digital broadcasting.

[0086] As described above, in conventional continuous few-shot learning methods such as XtarNet, when calculating the query loss in meta-learning, all pre-trained base classes are projected onto a projection space (joint classification space) and the query loss is calculated for all base classes, which makes it difficult for the loss to converge and takes a long time to learn. In contrast, according to the machine learning device 200 of the first embodiment, the class to be classified related to the loss calculation during meta-learning is optimized, which makes it easier for the loss to converge and shortens the learning time.

[0087] More specifically, in meta-learning, the base classes of the query set are labeled. By using this base class label information, the base classes selected for each episode's query set are sequentially added to the projection space when calculating the query loss. This reduces the number of target classes until all pre-trained base classes are projected into the projection space. This facilitates loss convergence and shortens the training time.

[0088] Furthermore, in conventional continuous few-shot learning methods such as XtarNet, all pre-trained base classes and new classes are projected onto a projection space (joint classification space) during meta-learning, and the query loss is calculated for all classes. This means that classes with low relevance to the task-adaptive representation of the query sample are also taken into account in the calculation, which may result in a decrease in classification accuracy. Furthermore, the loss is difficult to converge, and learning takes a long time. In contrast, according to the machine learning device 210 of the second embodiment, the classes to be classified in the loss calculation during meta-learning are limited to classes with high relevance to the task-adaptive representation, which makes it easier for the loss to converge and improves classification accuracy.

[0089] The present invention has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of the components and treatment processes, and that such modifications are also within the scope of the present invention. [Explanation of symbols]

[0090] 10 base class dataset, 12 new class dataset, 14 base class dataset, 16 new class dataset, 20 pre-training module, 22 backbone CNN, 23 averaging unit, 24 base class classification weight, 30 meta module group, 32 MetaCNN, 33 averaging unit, 34 new class classification weight, 36 MergeNet, 38 TconNet, 40 projection space construction unit, 42 projection space query classification unit, 44 loss optimization unit, 50 base class feature extraction unit, 52 new class feature extraction unit, 60 mixed feature calculation unit, 70 adjustment unit, 80 learning unit, 90 weight selection unit, 92 base class label information storage unit, 94 neighbor class selection unit, 100 continuous few-shot learning module, 200 machine learning device, 210 machine learning device.

Claims

1. A machine learning device that continuously learns a small number of new classes compared to a base class, a base class feature extraction unit for extracting a feature vector of a base class; a new class feature extraction unit that extracts a feature vector of a new class; a mixed feature calculation unit that mixes the feature vector of the base class and the feature vector of the new class to calculate a mixed feature vector of the base class and the new class; a learning unit that classifies the query samples of the query set based on the distance between the position of the mixed feature vector of the query sample of the query set and the position of the classification weight vector of each class in the projection space, and learns the classification weight vector of a new class so as to minimize the classification loss; a weight selection unit that sequentially adds classification weight vectors of basic classes selected for the query set onto a projection space when learning the query set on an episode-by-episode basis.

2. A machine learning method for continuously learning a small number of new classes compared to a base class, comprising: a base class feature extraction step of extracting a feature vector of a base class; a new class feature extraction step of extracting a feature vector of a new class; a mixed feature calculation step of mixing the feature vector of the base class and the feature vector of the new class to calculate a mixed feature vector of the base class and the new class; a learning step of classifying the query samples of the query set based on the distance between the position of the mixed feature vector of the query sample of the query set and the position of the classification weight vector of each class in the projection space, and learning the classification weight vector of a new class so as to minimize the classification loss; a weight selection step of sequentially adding classification weight vectors of basic classes selected for the query set onto the projection space when learning the query set on an episode-by-episode basis.

3. A machine learning program that continuously learns a small number of new classes compared to a base class, a base class feature extraction step of extracting a feature vector of a base class; a new class feature extraction step of extracting a feature vector of a new class; a mixed feature calculation step of mixing the feature vector of the base class and the feature vector of the new class to calculate a mixed feature vector of the base class and the new class; a learning step of classifying the query samples of the query set based on the distance between the position of the mixed feature vector of the query sample of the query set and the position of the classification weight vector of each class in the projection space, and learning the classification weight vector of a new class so as to minimize the classification loss; and a weight selection step of sequentially adding classification weight vectors of basic classes selected in the query set onto a projection space when learning the query set on an episode-by-episode basis.

Images