Learning apparatus, classifying apparatus, learning method, classifying method, and program

Abstract

To provide another technique for classifying data that is obtained in a target domain different from a training domain.SOLUTION: A learning apparatus includes a control unit which trains a model to be trained, the to-be-trained model including: a data augmentation process that performs data augmentation on original data belonging to a data group which is a group of unsupervised original data in a target domain; a main classifier which obtains a feature vector of data to be processed, and a distribution indicating the probability that the data belongs to a predefined candidate class; and an auxiliary classifier which obtains a feature vector of augmented data and a distribution indicating the probability that the augmented data belongs to the class. Training includes updating the main classifier and the auxiliary classifier based on a result of the main classifier that processes the original data and a result of the main classifier that processes the augmented data, and updating the data augmentation process based on the result of the main classifier that processes the augmented data and a result of the auxiliary classifier.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a learning device, a classification device, a learning method, a classification method, and a program. [Background technology]

[0002] A framework called supervised learning is generally used to train classifiers using machine learning. Supervised learning is a technique in which a large number of pairs of data and their corresponding correct labels are prepared, and a classifier is trained that can predict the correct label when data is input. Such a large number of pairs of data and correct labels are called supervised training data.

[0003] A classifier trained by supervised learning can achieve high classification accuracy in the environment or domain where supervised training data is obtained (hereinafter referred to as the "training domain"). On the other hand, when the environment or domain where new data to which the classifier is to be applied is obtained (hereinafter referred to as the "target domain") differs from the training domain, the classification accuracy of the classifier for the target domain drops significantly.

[0004] As can be seen from this explanation, the target domain is an environment or domain from which data input to a mathematical model trained using training data was obtained, but which is different from the environment or domain from which the training data was obtained, while the training domain is the environment or domain from which the training data, which is data used to train the mathematical model, was obtained.

[0005] For example, a classifier trained using a supervised training dataset for classifying objects created using computer graphics may experience a decrease in classification accuracy when classifying objects in photographs taken in the real world due to differences in appearance between the computer graphics and the photographs. Learning problems that arise when differences exist between domains like this are called domain adaptation problems. Constructing supervised training data in the target domain is often constrained by the costs involved in data collection and labeling, making it difficult to solve the problem by performing supervised learning in the target domain.

[0006] In order to solve such a domain adaptation problem, there are techniques disclosed in, for example, Patent Document 1 and Non-Patent Documents 1 to 3 below.

[0007] The technology disclosed in Patent Document 1 learns a transformation rule from the training domain to the target domain that minimizes the Maximum Mean Discrepancy (MMD) value, which represents the inter-distribution distance between the generation distribution of samples in the training domain and the generation distribution of samples in the target domain. The technology in Patent Document 1 aims to solve the domain adaptation problem by training a classifier using samples of the training domain transformed according to the transformation rule.

[0008] For example, in the technology disclosed in Non-Patent Document 1, a domain classifier that classifies samples in a training domain and a target domain is simultaneously trained. The technology disclosed in Non-Patent Document 1 aims to solve the domain adaptation problem by learning features that prevent the domain classifier from correctly classifying the two domains, thereby acquiring features that are useful for classification independent of the domain.

[0009] For example, the techniques disclosed in Patent Document 1 and Non-Patent Document 1, and similar techniques, require data from two domains, namely, supervised data for the training domain and unsupervised data for the target domain, when training a classifier for a target domain. However, these techniques have problems such as possible privacy issues regarding the supervised training data and the inefficiency of training a classifier using supervised training data for each different target domain. From these perspectives, there is a demand for the ability to train a classifier without using supervised training data. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] Japanese Patent Application Publication No. 2019-101789 [Non-patent literature]

[0011] [Non-Patent Document 1] Ganin, Yaroslav, and Victor Lempitsky. "Unsupervised domain adaptation by backpropagation." International conference on machine learning. PMLR, 2015. [Non-patent document 2] Liang, Jian, Dapeng Hu, and Jiashi Feng. "Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation." International conference on machine learning. PMLR, 2020. [Non-patent document 3] Yang, Shiqi, Shangling Jui, and Joost van de Weijer. "Attracting and dispersing: A simple approach for source-free domain adaptation." Advances in Neural Information Processing Systems 35 (2022): 5802-5815. Summary of the Invention [Problem to be solved by the invention]

[0012] However, there was a need for new techniques to classify data acquired in a target domain that differs from the training domain.

[0013] In view of the above, an object of the present invention is to provide another technique for performing classification on data acquired in a target domain different from the training domain. [Means for solving the problem]

[0014] One aspect of the present invention is a learning device comprising: a control unit that performs learning of a learning target model, which is a mathematical model of a learning target; a data augmentation process that performs probabilistically determined data augmentation on each piece of original data belonging to a dataset, which is a collection of original data that is unsupervised data in a target domain; a main classifier that obtains a feature vector of the data to be processed and a main classification assignment probability distribution that indicates the probability that the data belongs to each of classes that have been previously determined as candidates for assignment; and an auxiliary classifier that obtains a feature vector of extended data that is a result of the data augmentation and an auxiliary classification assignment probability distribution that indicates the probability that the extended data belongs to each of the classes, wherein the parameter values ​​of the main classifier and the parameter values ​​of the auxiliary classifier are different, and during the learning, the main classifier and the auxiliary classifier are updated based on the result of the main classifier using the original data as the processing target and the result of the main classifier using the extended data as the processing target, and the content of the data augmentation process is updated based on the result of the main classifier and the result of the auxiliary classifier using the extended data as the processing target.

[0015] One aspect of the present invention is a classification device comprising: a control unit that learns a learning target model, which is a mathematical model of a learning target; the learning target model includes: a data augmentation process that performs probabilistically determined data augmentation on each original data belonging to a dataset, which is a collection of original data that is unsupervised data in a target domain; a main classifier that obtains a feature vector of the data to be processed and a main classification assignment probability distribution that indicates the probability that the data belongs to each class that has been pre-determined as a candidate for assignment; and an auxiliary classifier that obtains a feature vector of extended data that is a result of the data augmentation and an auxiliary classification assignment probability distribution that indicates the probability that the extended data belongs to each of the classes, wherein the parameter values ​​of the main classifier and the parameter values ​​of the auxiliary classifier are different; and during the learning, the main classifier and the auxiliary classifier are updated based on the result of the main classifier using the original data to be processed and the result of the main classifier using the extended data to be processed, and the content of the data augmentation process is updated based on the result of the main classifier and the result of the auxiliary classifier using the extended data to be processed.

[0016] One aspect of the present invention is a learning method executed by a learning device, comprising: a control unit that learns a learning target model, which is a mathematical model of a learning target; the learning target model including: a data augmentation process that performs probabilistically determined data augmentation on each original data belonging to a dataset, which is a collection of original data that is unsupervised data in a target domain; a main classifier that obtains a feature vector of the data to be processed and a main classification assignment probability distribution that indicates the probability that the data belongs to each of classes previously determined as candidate classes; and an auxiliary classifier that obtains a feature vector of extended data that is a result of the data augmentation and an auxiliary classification assignment probability distribution that indicates the probability that the extended data belongs to each of the classes, wherein the parameter values ​​of the main classifier and the parameter values ​​of the auxiliary classifier are different; and during the learning, the main classifier and the auxiliary classifier are updated based on the result of the main classifier using the original data as the processing target and the result of the main classifier using the extended data as the processing target, and the content of the data augmentation process is updated based on the result of the main classifier and the result of the auxiliary classifier using the extended data as the processing target, and the content of the data augmentation process is updated based on the result of the main classifier using the extended data as the processing target, and the result of the auxiliary classifier

[0017] One aspect of the present invention is a control unit that performs learning of a learning target model, which is a mathematical model to be learned, wherein the learning target model includes a data augmentation process that performs probabilistically determined data augmentation on each original data belonging to a dataset, which is a set of original data that is unsupervised data of a target domain; a main classifier that obtains a feature vector of the data to be processed and a main class membership probability distribution that indicates the probability that the data belongs to each of classes that have been previously determined as candidates for membership; and a main classifier that obtains a feature vector of augmented data that is a result of the data augmentation and an auxiliary classifier that indicates the probability that the augmented data belongs to each of the classes. and an auxiliary classifier that obtains an auxiliary classifier membership probability distribution and an auxiliary classifier that obtains an auxiliary classifier membership probability distribution, wherein the parameter values ​​of the main classifier and the parameter values ​​of the auxiliary classifier are different, and in the learning, the main classifier and the auxiliary classifier are updated based on the result of the main classifier that uses the processing target as original data and the result of the main classifier that uses the processing target as extended data, and the content of the data extension process is updated based on the result of the main classifier that uses the processing target as extended data and the result of the auxiliary classifier,

[0018] One aspect of the present invention is a program for causing a computer to function as the learning device described above.

[0019] One aspect of the present invention is a program for causing a computer to function as the above-described classification device. [Effects of the Invention]

[0020] The present invention makes it possible to provide other techniques for performing classification on data acquired in a target domain different from the training domain. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is an explanatory diagram illustrating a classification system according to an embodiment. [Figure 2]FIG. 2 is a diagram illustrating an example of a hardware configuration of a learning device according to an embodiment. [Figure 3] 10 is a flowchart showing an example of a flow of processing executed by a learning device according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a classification device according to an embodiment. [Figure 5] 10 is a flowchart showing an example of a flow of processing executed by the classification device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] (Embodiment) 1 is an explanatory diagram illustrating a classification system 100 according to an embodiment. The classification system 100 includes a learning device 1 and a classification device 2.

[0023] The learning device 1 includes a control unit 11 including a processor 91, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit) or an NPU (Neural Network Processing Unit), and a memory 92, which are connected via a bus.

[0024] The control unit 11 performs a learning process. The learning process is a process of learning a mathematical model to be learned, which is a mathematical model including a data augmentation process and a classifier (hereinafter referred to as a "learning target model"), until a predetermined condition for terminating the learning (hereinafter referred to as a "learning termination condition") is satisfied. The learning termination condition may be any condition related to the termination of learning, and may be, for example, a condition that the learning target has been updated a predetermined number of times, or a condition that the change in the learning target due to the update is smaller than a predetermined change.

[0025] The data augmentation process is a process of performing probabilistically determined data augmentation on each piece of original data belonging to a data set, which is a collection of original data. Hereinafter, the result of the data augmentation process is referred to as augmented data. Therefore, the augmented data is the result of data augmentation on the original data. "Probabilistically determined" means that the content of the data augmentation is determined probabilistically.

[0026] As is well known, data expansion is a process of transforming or combining data such as image data, video data, and signals such as audio signals. As a technology for transforming or combining data, data expansion has the effect of expanding the amount and distribution of data.

[0027] More specifically, processes that can be performed in data extension will be described using an example in which the target of extension is a signal. Data extension includes, for example, a process of mixing multiple signals, a process of expanding or contracting a signal, a process of changing the intensity, phase, pitch, or reverberation of a signal, a process of inverting a signal, a process of deleting or masking part of a signal, and a process of degrading a signal by noise or blurring. Furthermore, when the signal is particularly image data or video data, data extension includes, for example, a process of changing the brightness of the image data or video data, a process of performing a parallel translation, etc. As such, there are various processes in data extension.

[0028] As described above, there are various types of processing that can be executed as data augmentation. Therefore, when data augmentation is determined probabilistically, it means that one processing to be executed as data augmentation is determined probabilistically from among various types of predetermined processing. Because the data augmentation is determined probabilistically, the processing executed in data augmentation at different execution times is not necessarily the same depending on the probability assigned to each processing predefined as data augmentation processing. Therefore, by executing the data augmentation processing, it is possible to obtain multiple different data from the same original data.

[0029] The content of the data augmentation process can be updated based on the augmentation section loss evaluation value, as will be described later. Therefore, the data augmentation process is a process expressed by parameters. In machine learning, whether or not backpropagation can be performed affects the estimation accuracy. To be able to perform backpropagation, the process to be updated by learning must be differentiable. Therefore, the parameters of the data augmentation process may be differentiable. To this end, the data augmentation process may be, for example, the data augmentation process described in Reference 1, or may be another process whose parameters are differentiable.

[0030] Reference 1: Hataya, Ryuichiro, et al. "Faster autoaugment: Learning augmentation strategies using backpropagation." Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXV 16. Springer International Publishing, 2020.

[0031] The original data is unsupervised data of the target domain. More precisely, the data of the target domain is data acquired in the target domain. Therefore, the unsupervised data of the target domain is unsupervised data acquired in the target domain. The target domain is an environment or domain in which data input to a mathematical model trained using training data was acquired, but which is different from the environment or domain in which the training data was acquired. The training domain is the environment or domain in which training data, which is data used to train the mathematical model, was acquired.

[0032] For example, if the training domain data is a facial photograph of a real person, the target domain data may be a facial photograph of a computer graphics character. The training domain data may also be a facial photograph of a computer graphics character, in which case the target domain data may be a facial photograph of a real person. The training domain data may also be an audio signal recording a female voice, in which case the target domain data may be an audio signal recording a male voice. For example, when training a trained model with the aim of using the trained model for recognizing in-vehicle video, the training domain may be video footage captured on a sunny day, in which case the target domain data may be video footage captured on a rainy day.

[0033] The learning process will be described in more detail. In the learning process, a branching process is executed. The branching process is a process that determines whether a first unit process or a second unit process will be executed in the subsequent stage of the branching process according to a branching rule, which is a predetermined rule regarding branching. In the learning process, the process determined in the branching process between the first unit process and the second unit process (hereinafter referred to as "determination process") is executed in the subsequent stage of the branching process, and after the determination process is executed, a learning completion determination process is executed.

[0034] The learning end determination process is a process for determining whether or not the learning end condition is satisfied. If the execution of the learning end determination process determines that the learning end condition is satisfied, the learning process ends. On the other hand, if the execution of the learning end determination process determines that the learning end condition is not satisfied, a new branching process is executed. In this way, in the learning process, the branching process, the decision process, and the learning end determination process are repeatedly executed one or more times until the learning end condition is satisfied. The model to be trained at the time the learning end condition is satisfied is the trained model to be trained.

[0035] The branching rule is, for example, a rule that determines that a predetermined one of the first unit process and the second unit process is to be executed in a later stage when the branching process is executed an odd number of times, and determines that the other, different from the one, is to be executed in a later stage when the branching process is executed an even number of times.The branching rule is, for example, a rule that determines that a predetermined one of the first unit process and the second unit process is to be executed in a later stage when a predetermined section of the branching process is executed, and determines that the other, different from the one, is to be executed in a later stage in other sections.

[0036] Note that the interval means a range of the number of times a branch process is executed. Therefore, for example, an interval is an interval where the remainder when the number of times a branch process is executed divided by 20 is 0 to 9, and in this case, another interval is an interval where the remainder when the number of times a branch process is executed divided by 20 is 10 to 19.

[0037] <First unit processing> The first unit process will be described. In the first unit process, a data set acquisition process is executed. The data set acquisition process is a process of acquiring a data set by, for example, reading it from a predetermined storage unit (for example, storage unit 13, which will be described later).

[0038] In the first unit process, the model to be trained is executed. As described above, the model to be trained includes a data augmentation process and a classifier, and more specifically, includes a data augmentation process, a main classifier, and an auxiliary classifier. Therefore, by executing the model to be trained, the data augmentation process, the main classifier, and the auxiliary classifier are executed.

[0039] The main classifier is a mathematical model that obtains a main classification feature vector and a main classification membership probability distribution according to a main classification map based on data to be processed by the main classifier (hereinafter referred to as "main classifier target data"). The main classification map is a parameterized map that associates the main classifier target data with the main classification feature vector and the main classification membership probability distribution. The main classification feature vector is a feature vector of the main classifier target data.

[0040] A feature vector is a numerical vector that represents the features of data. More specifically, a feature vector represents the required data features as a vector with n-dimensional elements. n is an arbitrary integer value, and may be, for example, n=256.

[0041] The main class membership probability distribution is information indicating the probability (hereinafter referred to as "main class membership probability") that each of the classes (hereinafter referred to as "candidate classes") previously determined as candidates for the class to which the main classifier target data belongs. The main class membership probability indicated by the main class membership probability distribution may be normalized so that the sum of the main class membership probabilities indicated by the main class membership probability distribution is 1.

[0042] Therefore, for example, if the number of candidate classes is C, the main class membership probability distribution may be represented by a vector having C-dimensional elements, where the sum of the elements is 1.

[0043] The main class membership probability distribution may indicate a non-zero main class membership probability for one of all classes previously determined as candidate classes, and a zero main class membership probability for the other classes. This means that the main class membership probability distribution indicates the most likely class to which the main classifier target data belongs.

[0044] As can be seen from the definition of the main classifier, the main classifier is one of the classifiers included in the model to be trained. The parameters of the main classification map are an example of so-called classifier parameters, and are the parameters of the main classifier in the model to be trained.

[0045] From the above, it can be said that the main classifier is a classifier that obtains the feature vector of the data to be processed and the main class membership probability distribution that indicates the probability that the data belongs to each candidate class.

[0046] The parameter values ​​of the primal classification map are updated through learning. Therefore, in order to be able to apply the backpropagation algorithm as described above in the description of the data augmentation process, the parameter values ​​of the primal classification map may be differentiable.

[0047] The auxiliary classifier is a mathematical model that obtains an auxiliary classification feature vector and an auxiliary classification membership probability distribution according to an auxiliary classification map based on data to be processed by the auxiliary classifier (hereinafter referred to as "auxiliary classifier target data"). The auxiliary classification map is a parameterized map that associates the auxiliary classifier target data with the auxiliary classification feature vector and the auxiliary classification membership probability distribution. The auxiliary classification feature vector is a feature vector of the auxiliary classifier target data. The auxiliary classifier target data is expanded data obtained by a data expansion process, and is used when a second unit process, which will be described in detail later, is executed. The process of determining whether to execute the first unit process or the second unit process is executed is performed by the control unit 11 based on the branching process described above. Note that in the first unit process, the auxiliary classifier does not process data.

[0048] The auxiliary class membership probability distribution is information indicating, for each candidate class, the probability that the auxiliary classifier target data belongs to the auxiliary classifier (hereinafter referred to as "auxiliary class membership probability"). The auxiliary class membership probabilities indicated by the auxiliary class membership probability distribution may be normalized so that the sum of the auxiliary class membership probabilities indicated by the auxiliary class membership probability distribution is 1.

[0049] The auxiliary class membership probability distribution may indicate a non-zero auxiliary class membership probability for one of all classes previously determined as candidate classes, and zero auxiliary class membership probabilities for the other classes. This means that the auxiliary class membership probability distribution indicates the most likely class to which the auxiliary classifier target data belongs.

[0050] As can be seen from the definition of an auxiliary classifier, an auxiliary classifier is one of the classifiers included in the model to be trained. The parameters of the auxiliary classification map are an example of the parameters of a classifier, and are the parameters of the auxiliary classifier in the model to be trained.

[0051] From the above, it can be said that the auxiliary classifier is a classifier that obtains, for each feature vector of the data to be processed and a candidate class, an auxiliary class membership probability distribution that indicates the probability that the data to be processed belongs to that class.

[0052] The parameter values ​​of the auxiliary classification map are updated through learning. Therefore, in order to be able to apply the backpropagation algorithm as described above in the description of the data augmentation process, the parameter values ​​of the auxiliary classification map may be differentiable.

[0053] Note that the parameter values ​​of the main classification map (i.e., the parameter values ​​of the main classifier) ​​and the parameter values ​​of the auxiliary classification map (i.e., the parameter values ​​of the auxiliary classifier) ​​are different from each other. However, the parameters of the main classifier and the auxiliary classifier are both set to the parameter values ​​of the pre-trained classifier only at the initial values. Note that the initial values ​​refer to the values ​​at the start of the learning process before the parameters are updated. Note that the candidate classes in the main classifier and the candidate classes in the auxiliary classifier are the same.

[0054] Furthermore, as can be seen from the above definitions, the main class membership probability and the auxiliary class membership probability are the same in that they indicate the probability that the candidate class is the target class to which the object belongs, with only the parameters being different. Therefore, hereinafter, the probability that the candidate class is the target class to which the object belongs is referred to as the estimated class membership probability. Therefore, both the main class membership probability and the auxiliary class membership probability are examples of the estimated class membership probability.

[0055] As described above, in the first unit process, the model to be trained is executed on the dataset acquired in the dataset acquisition process. Then, in the first unit process, the parameters of the main classifier and auxiliary classifier are updated based on the execution result of the model to be trained. The first unit process will now be described in more detail using an example.

[0056] <<Details of an example of the first unit process>> In the first unit process, a data augmentation process is performed on each original data included in the dataset. In the first unit process, a first main classification process and a second main classification process are then performed on each original data. The first main classification process is a process of running a main classifier on the original data as the processing target.

[0057] It goes without saying that the processing target of the main classifier in the first main classification process is the processing target of the first main classification process. The second main classification process is a process in which the main classifier is executed on the augmented data obtained by the data augmentation process. It goes without saying that the processing target of the main classifier in the second main classification process is the processing target of the second main classification process.

[0058] Next, in the first unit process, a classifier loss evaluation value acquisition process is executed. The classifier loss evaluation value acquisition process is a process for acquiring a classifier loss evaluation value, which is one type of loss used in training the training model, based at least on the results of the first main classification process and the second main classification process. The classifier loss evaluation value is a loss based at least on the results of the first main classification process and the second main classification process. Note that in the classifier loss evaluation value acquisition process, the classifier loss evaluation value may also be acquired based on past classification information.

[0059] The past classification information will now be described. The past classification information is information that has already been stored in a predetermined storage unit, such as the storage unit 13 described below. When the past classification information is used, it is information that is updated after the execution of a first unit process and before the execution of the next first unit process.

[0060] The past classification information is information that indicates, for each processing target (i.e., each original mode) of the first main classification process, information based on the main classification pairs obtained in the past first main classification process. The main classification pairs are pairs of main classification feature vectors and main classification belonging probability distributions. More specifically, the past classification information is information that indicates pairs of past vectors and past probability distributions for each processing target of the first main classification process.

[0061] The past vector is a vector based on the main classification feature vector obtained in the past first main classification process (hereinafter referred to as "past vector"). The past probability distribution is a distribution based on the main classification membership probability distribution corresponding to the main classification feature vector used to obtain the past vector, and is a distribution showing the probability for each candidate class.

[0062] Note that "obtained in a past first main classification process" means that the data was obtained by executing a first main classification process in a first unit process that was executed before the first unit process currently being executed.

[0063] Several examples will be explained using the example of a scene (hereinafter referred to as an "example scene") in which the first single process is repeatedly executed from the 1st to (Q-1)th times (Q is 2 or more) for past classification information because the learning termination condition is not met, and then the first single process is executed for a further Qth time.

[0064] In the illustrated example, the past classification information used in the classifier loss evaluation value acquisition process during the Qth first single process may indicate, for example, first main classification result pairs obtained in each of the Q0th to (Q-1)th first single processes for each processing target of the first main classification process. Each of the first main classification result pairs indicated by the past classification information in this example is an example of a pair of a past vector and a past probability distribution.

[0065] Here, Q0 is an integer equal to or greater than 1 and less than Q. The specific value of Q0 may be, for example, a predetermined value, or may be a value determined according to a predetermined rule. The predetermined rule may be, for example, a predetermined rule that depends on the number of repetitions of the first unit process.

[0066] In the illustrated example, the past classification information used in the classifier loss evaluation value acquisition process during the Qth first single process may indicate, for example, only the (Q-1)th first main classification result pair for each processing target of the first main classification process. Each of the first main classification result pairs indicated by the past classification information in this example is an example of a pair of a past vector and a past probability distribution.

[0067] In the illustrated example, the past classification information used in the classifier loss evaluation value acquisition process during the Qth first single process may indicate, for example, a moving average vector and a moving average distribution for each process target of the first main classification process. The moving average vector is the moving average of the main classification feature vectors obtained in each of the Q0th to (Q-1)th first single processes.

[0068] The moving average distribution is the moving average of the main classification membership probability distribution obtained in each of the first single processes from the Q0th to the (Q-1)th times. In this example, the moving average vector is an example of a past vector, and the moving average distribution is an example of a past probability distribution.

[0069] For simplicity of explanation below, the first unit process executed immediately before the first unit process currently being executed will be referred to as the most recent past process. Also, for simplicity of explanation below, the first unit process currently being executed will be referred to as the current first process. In the above example, the Qth first unit process is the current first process, and the most recent past process relative to that current first process is the (Q-1)th first unit process.

[0070] After obtaining the classifier loss evaluation value, in the first unit process, the parameter values ​​of the main classifier are updated to reduce the classifier loss evaluation value, and the parameter values ​​of the auxiliary classifier are updated based on the updated parameter values ​​of the main classifier.

[0071] In this way, the first unit process is a process for updating the main classifier and auxiliary classifier based on the execution result of the learning target model.

[0072] <Second unit processing> The second unit process will now be described. In the second unit process, a dataset acquisition process is executed. In the second unit process, a model to be trained is executed on the dataset acquired in the dataset acquisition process. In the second unit process, the content of the data augmentation process is updated based on the execution result of the model to be trained. This will now be described in more detail.

[0073] Note that updating the content of the data augmentation process means updating the values ​​of parameters of the data augmentation that can be executed in the data augmentation process, such as updating the mixing ratio of the signal mixture or the degree of data deformation.

[0074] <<More details on the second unit process>> In the second unit process, a data augmentation process is executed. In the second unit process, a second main classification process is executed for each augmented data, and an auxiliary classifier is executed for each augmented data.

[0075] Next, in the second unit processing, a data augmentation unit loss evaluation value, which is one type of loss used in training the training model, is obtained based on the results of the second main classification processing and the auxiliary classifier. The data augmentation unit loss evaluation value is a loss based on the results of the second main classification processing and the auxiliary classifier. Then, in the second unit processing, the content of the data augmentation processing is updated so as to reduce the data augmentation unit loss evaluation value.

[0076] In this way, the second unit process is a process that updates the content of the data augmentation process based on the execution result of the learning target model.

[0077] <Examples of the effects of learning device 1> Theoretical knowledge will be used to explain the effects of the learning device 1. Reference 2 discloses a theory of self-learning based on extended assumptions (hereinafter referred to as "reference theory").

[0078] Reference 2: Wei, Colin, et al. "Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data." International Conference on Learning Representations. 2020.

[0079] In the literature, we show that when an assumption about the data distribution, called the augmented assumption, holds, and the classifier is trained based on a self-learning loss based on the predictive consistency of the classifier and a constraint that imposes predictive diversity on the classifier, there is an upper bound on the classification error of the classifier.

[0080] Here, the self-learning loss based on the prediction consistency of a classifier is a loss that is reduced when the estimated class membership probabilities output by the classifier are consistent between data with and without data augmentation for the same original data. Furthermore, the prediction diversity constraint imposed on a classifier is a constraint that requires that, when the estimated class membership probabilities are calculated for all data by the classifier, a certain percentage or more of class membership probabilities be assigned to all classes on average.

[0081] Since neither of these losses nor constraints requires correct labels for the data, they can be used for domain adaptation problems where supervised learning data cannot be used.

[0082] There are two useful findings in literature theory. The first is that the self-training loss and predictive diversity constraints used in training a classifier can be relaxed as training progresses, and the degree of relaxation can be controlled by the loss value. This means that if we can find just the two values ​​of the self-training loss and predictive diversity constraint, we can train a classifier that guarantees an upper bound on the classification error. Hereafter, this finding will be referred to as the first finding.

[0083] The second finding is that the upper bound on the classification error depends on the nature of the data augmentation used for the self-training loss. Specifically, the greater the change from the original data due to data augmentation, and the lower the prediction consistency of a correct classifier that always correctly predicts the class of the input data, the smaller the upper bound on the classification error. Therefore, if such data augmentation is obtained, the self-training loss and the prediction error of a classifier trained with the prediction consistency constraint will be smaller. Hereinafter, this finding will be referred to as the second finding.

[0084] In the learning process, a main classifier and an auxiliary classifier, each of which is an example of a classifier, are learned based on a first finding, and data augmentation processing is learned based on a second finding.

[0085] Therefore, by executing a learning process, it is possible to provide a technique for classifying data acquired in a target domain different from the training domain, and for learning a classifier without using supervised data.

[0086] Here, the technologies disclosed in Non-Patent Document 2 and Non-Patent Document 3 are compared with the learning device 1. The technologies in Non-Patent Documents 2 and 3 are technologies that realize training of a classifier without using supervised training data by performing training using a classifier that has been trained once using supervised training data (hereinafter referred to as a "pre-trained classifier") and unsupervised data of a target domain.

[0087] More specifically, Non-Patent Document 2 discloses a technology for improving classification accuracy of samples in a target domain without samples from the training domain by training a classifier based on the following two losses. One of the two losses is a classification loss used in classification learning using pseudo-correct labels for samples in the target domain, which are correct labels obtained by clustering based on intermediate features of the classifier being trained. The other of the two losses is a training loss that maximizes the mutual information of the classifier's predictions.

[0088] Furthermore, the technology disclosed in Non-Patent Document 3 discloses a technology for training a classifier that can classify samples of a target domain with higher accuracy than Non-Patent Document 2 by training the classifier based on the following two losses. One of the two losses is a loss that maximizes the similarity of prediction results between samples that exist nearby in the feature space in the intermediate layer of the classifier for samples of the target domain. The other of the two losses is a loss that minimizes the similarity of prediction results between samples outside the command.

[0089] However, the techniques disclosed in Non-Patent Document 2 and Non-Patent Document 3 are not theoretically supported. On the other hand, the learning process is a theoretically supported technique as described above. In the case of a technique without a theoretical support, it is difficult to examine the reliability of the results obtained compared to a technique with a theoretical support. Therefore, the technique of executing the learning process makes it easier to examine the reliability of the results than Non-Patent Document 2 and Non-Patent Document 3.

[0090] Thus, the learning device 1 can provide another technique for performing classification on data acquired in a target domain different from the training domain.

[0091] <Explanation of an example of learning processing using mathematical expressions> An example of the learning process will be described using mathematical expressions, taking the case where past classification information is used as an example.

[0092] <<First unit process>> In the first unit process, the control unit 11 acquires a data set expressed by the following formula (1) by, for example, reading it from a predetermined storage unit (for example, the storage unit 13 described later).

[0093]

number

[0094] Data x i represents the original data identified by the identifier i. B represents the total number of original data belonging to the dataset.

[0095] In the data augmentation process, data augmentation is performed M times for each original data in the data set, and M augmented data sets are obtained. i Among the M pieces of extended data, the extended data identified by the identifier m is expressed by the following formula (2).

[0096]

number

[0097] The main classifier is expressed by, for example, a first function f and a second function g. The first function f is a function of the data x i As input, the main classification feature vector z i The second function g is a function that outputs the main classification feature vector z i When it receives as input, the main classification probability distribution p i is a function that outputs a main classification feature vector z i is expressed by the following formula (3), and the main classification probability distribution p i is expressed by the following equation (4).

[0098]

number

[0099]

number

[0100] As mentioned above, the auxiliary classifier is a mathematical model that differs from the main classifier only in the parameter values ​​and the processing target. Therefore, in consideration of equations (3) and (4), the augmented data x i m The auxiliary classification probability distribution p i m is expressed by the following equation (5).

[0101]

number

[0102] <<<How to obtain the classifier loss evaluation value>>> The acquisition of a classifier loss evaluation value will now be described. First, an example of how past classification information is used when acquiring a classifier loss evaluation value will be described. For simplicity of explanation, in this example, the past classification information indicates only the main classification pairs obtained by the first main classification process in the most recent past process for each processing target of the first main classification process.

[0103] In the learning process, for example, a selection process is performed to obtain the classifier loss evaluation value. The selection process is performed by selecting the main classification feature vector z obtained in the first main classification process in the current first process from the main classification feature vectors indicated by the past classification information. i This is a process of obtaining, for each original mode, the main classification belonging probability distribution corresponding to the top K main classification feature vectors with the highest similarity to the original mode.

[0104] Hereinafter, the set of the K main class membership probability distributions thus obtained is expressed by the following formula (6).

[0105]

number

[0106] An example has been described in which the past classification information indicates, for each original mode, the main classification pairs obtained by the first main classification process in the most recent past process. However, as described above, the past classification information may indicate, for example, the main classification pairs for the past several times for each original mode, or may indicate the moving average vector and the moving average distribution for each original mode.

[0107] Meanwhile, the losses (i.e., classifier loss estimates) for training the main classifier and auxiliary classifiers are composed of the self-training loss and the prediction diversity constraint based on literature theory.

[0108] Regarding the former (i.e., self-training loss), for example, the distribution of the estimated class membership probability of the original data (i.e., the main class membership probability distribution obtained by the first main classification process) p i and the distribution of the estimated class membership probability of the extended data (i.e., the main class membership probability distribution obtained by the second main classification process) p i m It is calculated by finding the similarity between

[0109] However, the predictions made by the main classifier during training may contain many errors. Therefore, the distribution p i and distribution p i m The calculation of the similarity with distribution p i This is similar to a supervised loss where the distribution p i If the predictions of contain many errors, this can adversely affect the accuracy of the trained classifiers (i.e., the main classifier and auxiliary classifiers).

[0110] To avoid this problem, the learning process uses the distribution p i More specifically, the set of past probability distributions C i In such a learning process, the self-learning loss L1 is expressed by, for example, the following equation (7):

[0111]

number

[0112] On the other hand, for the latter (constraint on predictive diversity), for example, if a sample x in the data set i The distribution of the estimated class membership probability of the expanded data (i.e., the main class membership probability distribution obtained by the second main classification process) p i m is the main classification probability distribution p obtained by the first main classification process of other samples in the data set. j(j is different from i) and the similarity is reduced. Therefore, the prediction diversity constraint is expressed, for example, using the loss L2 in the following equation (8).

[0113]

number

[0114] These losses are distributed over the distribution p i m Instead of the distribution p i Therefore, the self-training loss calculated without data augmentation may be the loss L3 expressed by the following equation (9), and the prediction diversity constraint may be expressed using the loss L4 expressed by the following equation (10).

[0115]

number

[0116]

number

[0117] Based on the first finding, the constraint on prediction diversity expressed using loss L2 or loss L4 can be relaxed using the value of the self-learning loss as training (i.e., learning) progresses. Such relaxation can be realized, for example, by multiplying loss L2 by a coefficient α expressed in the following equation (11) or by multiplying loss L4 by a coefficient α expressed in the following equation (12).

[0118]

number

[0119]

number

[0120] However, α maxis a constant that represents the maximum value of α in equation (11), and is a constant that represents the maximum value of α' in equation (12). SG(·) represents the process of treating a value as a constant rather than as a target for gradient calculation during gradient calculations such as gradient descent.

[0121] From the above, the classifier loss evaluation value, which is the loss used to update the main classifier and auxiliary classifier in the learning process, is, for example, the loss L F It may be.

[0122]

number

[0123] Here, β represents a predetermined constant for adjusting the loss balance.

[0124] <<<Updating the main classifier and the auxiliary classifier>>> The update of the main classifier and the update of the auxiliary classifier in the learning process will be described. <<<<<Updating the main classifier>>>> First, we will explain the update of the main classifier in the training process. More specifically, we will explain the update of the parameters η and ψ mentioned above in the training process. The main classifier obtained by repeatedly updating the parameters η and ψ so as to minimize the classifier loss evaluation value of equation (13) is guaranteed to have an upper bound on its classification error. Training (learning) of the main classifier is expressed as the optimization of the objective function expressed by the following equation (14).

[0125]

number

[0126] where the main classifier is L F The parameters η and ψ are configured to be differentiable with respect to ∑ i = ...

[0127] <<<<<Updating the auxiliary classifier>>>> On the other hand, the auxiliary classifier may be updated using a different update method from that of the main classifier. As described above, the auxiliary classifier is a process that differs from the main classifier only in the parameter values. Therefore, hereinafter, the parameter of the auxiliary classifier corresponding to the parameter η of the main classifier is referred to as parameter η', and the parameter of the auxiliary classifier corresponding to the parameter ψ of the main classifier is referred to as parameter ψ'. In this case, the parameters η' and ψ' may be updated, for example, according to the rules of the following equations (15) and (16) so that they each become the exponential moving average of the main classifier.

[0128]

number

[0129]

number

[0130] Here, λ is a predetermined constant.

[0131] <<<Updating of past classification information>>> The following describes updating of past classification information. In the learning process, updating of past classification information may be, for example, as described above, a process of replacing, for each processing target of the first main classification process, a main classification pair obtained by the first main classification process in the current first process with a set of main classification pairs indicated by the past classification information.

[0132] However, updating of the past classification information is not necessarily limited to this. For example, if the past classification information indicates the results of the past few times, the update may be such that the main classification pair obtained for each processing target of the first main classification processing by the first main classification processing in the current first process is replaced with the oldest result for each processing target of the first main classification processing.

[0133] Furthermore, when the past classification information indicates a moving average vector and a moving average distribution for each processing target of the first main classification process, the past classification information may be updated to information indicating, for each processing target of the first main classification process, the moving average of the results obtained by the first main classification process in the current first process and the results indicated by the past classification information.

[0134] In this case, the updated past classification information indicates, for each processing target of the first main classification process, the moving average of the main classification feature vector obtained in the first main classification process in the current first process and the moving average vector indicated by the past classification information, and the moving average of the main classification belonging probability distribution obtained in the first main classification process in the current first process and the moving average distribution indicated by the past classification information.

[0135] <<About the second unit processing>> In the second unit process, the control unit 11 acquires the data set expressed by the above formula (1) by, for example, reading it from a predetermined storage unit (for example, the storage unit 13 described later). In the second unit process, the data x of the acquired data set i Then, data augmentation is performed with a parameter value of θ for the data x i The augmented data $\hat{x_i}$ is obtained from the above. Here, $\hat{x_i}$ means the symbol expressed in the following equation (17).

[0136]

number

[0137] <<<Main classifier and auxiliary classifier in the second unit process>>> We will now explain the main classifier and auxiliary classifier in the second unit process. Similar to classifier training, when the main classifier receives the extended data $\hat{x_i}$ as input, it outputs a main class membership probability distribution based on the parameters η and ψ. Therefore, the main classifier output for the extended data $\hat{x_i}$ is expressed by the following equation (18).

[0138]

number

[0139] The auxiliary classification probability distribution output by the auxiliary classifier for the augmented data $\hat{x_i}$ is expressed by the following equation (19).

[0140]

number

[0141] <<<Data extension loss evaluation value>>> The loss evaluation value of the data augmentation unit will now be described. The loss evaluation value of the data augmentation unit may include a loss that becomes smaller as the main class membership probability is equally assigned to all classes in the main class membership probability distribution output by the main classifier. Such a loss is expressed, for example, by the following equation (20).

[0142]

number

[0143] Here, C represents the total number of candidate classes. $\hat{p_i}[c]$ represents the main classification probability of class c among the main classification classification probability output by the main classifier. Note that $\hat{p_i}[c]$ represents the symbol in the following equation (21).

[0144]

number

[0145] The data augmentation loss evaluation value may include a loss that decreases as the auxiliary class membership probability increases for a certain class in the main class membership probability distribution output by the auxiliary classifier. Such a loss is expressed, for example, by the following equation (22):

[0146]

number

[0147] $\hat{p}'_i}[c]$ denotes the auxiliary classification assignment probability of class c among the auxiliary classification assignment probabilities output by the auxiliary classifier. Note that $\hat{p}'_i}[c]$ represents the symbol in the following equation (23).

[0148]

number

[0149] From the above, the data extension section loss evaluation value may be, for example, the loss expressed by the following equation (24).

[0150]

number

[0151] Here, γ is a constant for adjusting the loss balance.

[0152] <<<The Significance of Two Types of Loss>>> As described above, the data extension portion loss evaluation value includes two types of loss: loss L5 and loss L6. The significance of including two types of loss in this way will be explained below.

[0153] According to the second finding, the upper bound of the classification error is a value that depends on the properties of the data augmentation used for the self-training loss, and the larger the change from the original data due to data augmentation and the smaller the prediction consistency of the correct classifier, the smaller the upper bound of the classification error. To obtain a classifier with a small classification error, it is necessary to update the content of the data augmentation process based on the second finding so as to increase the change from the original data due to data augmentation while maintaining small prediction consistency of the correct classifier.

[0154] Updating the data augmentation process to increase the variation from the original data due to data augmentation is achieved by updating the data augmentation process to generate data that is difficult for the classifier to predict during training. Making predictions difficult for the classifier during training is equivalent to the classifier outputting low-confidence estimates, such as a main classification membership probability distribution output by the classifier that does not assign a high main classification membership probability to one class, but assigns approximately equal main classification membership probabilities to all classes. Therefore, training for such data augmentation can be achieved by training to minimize the value of the loss L5.

[0155] On the other hand, training that minimizes the predictive consistency of the correct classifier is ideally performed by updating the data augmentation process so that the correct classifier outputs a highly confident estimation result that assigns a high probability to one class. However, when attempting to train a classifier using unsupervised data, it is impossible to use the correct classifier. Therefore, in the training process, an auxiliary classifier is used as a substitute for the correct classifier.

[0156] For example, the auxiliary classifier is constructed with parameters that are the moving average of the main classifier. It is known that classifiers constructed in this way often exhibit better classification performance than the original classifier. Therefore, using the auxiliary classifier as a surrogate for the correct classifier is a reasonable technique from the perspective of improving classification accuracy. Therefore, training that keeps the prediction consistency of the correct classifier small can be achieved by training the data augmentation process so that the auxiliary classifier outputs highly confident inference results that assign a high auxiliary class membership probability to one class. Therefore, the data augmentation process can be achieved, for example, by learning to minimize the loss L6 expressed in the above equation (22).

[0157] <<<About learning data augmentation processing>>> Learning of the data augmentation process will now be described. Learning of the data augmentation process will now be described. In learning of the data augmentation process, the parameter θ is repeatedly updated so as to minimize the loss evaluation value of the data augmentation unit shown in the following equation (25).

[0158]

number

[0159] Here, the data extension loss evaluation value L A When the data augmentation process is configured so that the parameter θ is differentiable with respect to θ, the data augmentation process can be trained using gradient descent.

[0160] The classification device 2 performs classification using the trained main classifier obtained by the learning device 1. As described above, the main class membership probability distribution obtained by the main classifier indicates the probability for each candidate class. Therefore, the main class membership probability distribution can be said to be information indicating the class with the highest probability as the class to which the candidate class belongs. Therefore, classification is possible using the trained main classifier.

[0161] <Example of hardware configuration of learning device 1> 2 is a diagram showing an example of the hardware configuration of a learning device 1 according to an embodiment. The learning device 1 includes a control unit 11 having a processor 91 and a memory 92 connected by a bus, and executes a program. By executing the program, the learning device 1 functions as a device including the control unit 11, an interface unit 12, and a storage unit 13.

[0162] More specifically, the processor 91 reads out a program stored in the storage unit 13 and stores the read out program in the memory 92. When the processor 91 executes the program stored in the memory 92, the learning device 1 functions as a device including the control unit 11, the interface unit 12, and the storage unit 13.

[0163] The control unit 11 controls the operation of each functional unit included in the learning device 1. The control unit 11, for example, acquires information stored in the memory unit 13. Specifically, the process of acquiring information stored in the memory unit 13 is reading. The control unit 11, for example, performs a learning process.

[0164] The interface unit 12 includes a communication interface for connecting the learning device 1 to an external device. The interface unit 12 communicates with the external device via wired or wireless communication. The external device is, for example, a device that is the sender of the original data. The interface unit 12 acquires the original data by communicating with the device that is the sender of the original data.

[0165] The external device is, for example, the classification device 2. The interface unit 12 communicates with the classification device 2, enabling the classification device 2 to use a trained main classifier.

[0166] Interface unit 12 may be configured to include input devices such as a mouse, keyboard, or touch panel. Interface unit 12 may be configured as an interface that connects these input devices to learning device 1. In this way, the input devices of interface unit 12 accept input of various information to learning device 1 via wired or wireless connections. Note that information does not necessarily have to be input to the communication interface of interface unit 12, but may also be input to the input devices of interface unit 12.

[0167] Interface unit 12 outputs, for example, various types of information. Interface unit 12 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display, as well as a speaker. Interface unit 12 may be configured as an interface that connects these display devices or speakers to learning device 1. Therefore, the display device and speaker included in interface unit 12 output, for example, information input to an input device of interface unit 12 as an image or sound.

[0168] The memory unit 13 is configured using a computer-readable storage medium (non-transitory computer-readable recording medium) such as a magnetic hard disk drive or a semiconductor storage device. The memory unit 13 stores various information related to the learning device 1. The memory unit 13 stores various information generated by, for example, the operation of the control unit 11. The memory unit 13 stores, for example, the initial values ​​of parameters of the main classifier, the initial values ​​of parameters of the auxiliary classifier, the parameter values ​​of the main classifier, the parameter values ​​of the auxiliary classifier, and the contents of the data augmentation process. Storing the contents of the data augmentation process means, for example, storing the parameter θ described above. The memory unit 13 stores, for example, past classification information. The memory unit 13 may exist on a cloud, for example.

[0169] 3 is a flowchart showing an example of the flow of processing executed by the learning device 1 in the embodiment. The control unit 11 executes learning processing (step S101).

[0170] 4 is a diagram showing an example of the hardware configuration of the classification device 2 according to the embodiment. The classification device 2 includes a control unit 21 having a processor 93 and a memory 94 connected by a bus, and executes a program. By executing the program, the classification device 2 functions as a device including the control unit 21, an interface unit 22, and a storage unit 23.

[0171] More specifically, the processor 93 reads out the program stored in the storage unit 23 and stores the read out program in the memory 94. When the processor 93 executes the program stored in the memory 94, the classification device 2 functions as a device including the control unit 21, the interface unit 22, and the storage unit 23.

[0172] The control unit 21 controls the operation of each functional unit included in the classification device 2. The control unit 21 acquires, for example, information stored in the memory unit 23. Specifically, the process of acquiring information stored in the memory unit 23 is reading. The control unit 21 performs classification using, for example, a trained main classifier.

[0173] The interface unit 22 includes a communication interface for connecting the classification device 2 to an external device. The interface unit 22 communicates with the external device via wired or wireless communication. The external device is, for example, a device that is the sender of the object to be classified by the classification device 2. The interface unit 22 acquires the object to be classified by the classification device 2 by communicating with the device that is the sender of the object to be classified by the classification device 2.

[0174] The external device is, for example, the learning device 1. The interface unit 22 communicates with the learning device 1 to execute the trained main classifier.

[0175] The interface unit 22 may be configured to include input devices such as a mouse, a keyboard, a touch panel, etc. The interface unit 22 may be configured as an interface that connects these input devices to the classification device 2. In this way, the input devices of the interface unit 22 accept input of various information to the classification device 2 via wired or wireless connections. Note that information does not necessarily have to be input to the communication interface of the interface unit 22, but may also be input to the input devices of the interface unit 22.

[0176] The interface unit 22 outputs, for example, various types of information. The interface unit 22 includes a display device such as a CRT display, a liquid crystal display, or an organic EL display, and a speaker. The interface unit 22 may be configured as an interface that connects these display devices or speakers to the classification device 2. Therefore, the display device and speaker included in the interface unit 22 output, for example, information input to an input device of the interface unit 22 as an image or sound.

[0177] The storage unit 23 is configured using a computer-readable storage medium device (non-transitory computer-readable recording medium) such as a magnetic hard disk device or a semiconductor storage device. The storage unit 23 stores various information related to the classification device 2. The storage unit 23 stores various information generated by the operation of the control unit 21, for example. The storage unit 23 may exist on a cloud, for example.

[0178] 5 is a flowchart showing an example of the flow of processing executed by the classification device 2 in the embodiment. The control unit 21 acquires objects to be classified (step S201). The control unit 21 classifies the acquired objects to be classified using a trained main classifier (step S202).

[0179] The learning device 1 configured in this manner executes a learning process. Therefore, as described in the above section <Effects of the learning device 1>, the learning device 1 can provide another technique for classifying data acquired in a target domain different from the training domain.

[0180] Furthermore, the classification device 2 configured in this manner performs classification using the learning results obtained by the learning device 1. Therefore, the classification device 2 can provide another technique for performing classification on data acquired in a target domain different from the training domain.

[0181] The classification system 100 configured in this manner also includes a learning device 1. Therefore, the classification system 100 can provide another technique for performing classification on data acquired in a target domain different from the training domain.

[0182] (Variation) As mentioned above, since predictions made by the main classifier during training may contain many errors, past classification information may be used. This is intended to further improve accuracy. However, predictions made by the main classifier during training do not necessarily contain many errors. Furthermore, according to literature theory, past classification information is not necessary for self-training loss or constraints on prediction diversity. Therefore, past classification information does not necessarily need to be used in the training process.

[0183] As described above, in the learning process executed by the learning device 1, it is not necessary to use past classification information in the first unit process. In the first unit process in which past classification information is not used, the first update process may be executed after the data augmentation process and the execution of the learning target model. The first update process is a process for updating the main classifier and auxiliary classifier based on the results of the main classifier using the processing target as original data and the results of the main classifier using the processing target as augmented data.

[0184] In such a case, the classifier loss evaluation value is calculated by using the main class membership probability distribution for the original data obtained in the first unit process currently being executed, instead of the past classification information used in the process of calculating the classifier loss evaluation value described above. Note that the main class membership probability distribution for the original data is the output of the main classifier to which the original data is input.

[0185] The classifier loss evaluation value is a value calculated based on the results of the main classifier using the processing target as the original data and the results of the main classifier using the processing target as the augmented data. The classifier loss evaluation value is based on two types of values: a first value and a second value. The first value is a value that decreases as the similarity between the first probability distribution and the second probability distribution increases.

[0186] The first probability distribution is a main classification probability distribution for either any one of the original data belonging to the dataset (hereinafter referred to as "first original data") or the extended data that is the result of the data extension process on the first original data. In other words, the first probability distribution is the output of the main classifier to which the first original data or the result of the data extension process on the first original data is input.

[0187] The second probability distribution is a main class membership probability distribution for the first original data in the first unit process currently being executed, or a main class membership probability distribution for the first original data in a first unit process executed before the first unit process currently being executed.

[0188] The second value is a value that decreases as the similarity between the first probability distribution and the third probability distribution decreases.

[0189] The third probability distribution is a main classification probability distribution for any one piece of original data (hereinafter referred to as "second original data") that is different from the original data that is the target of the first main classification process from among the original data that belongs to the dataset. In other words, the third probability distribution is the output of the main classifier to which the second original data, which is any one piece of original data that is different from the first original data from among the original data that belongs to the dataset, is input.

[0190] The classifier loss estimate is a value used in the update process of the main classifier and auxiliary classifiers.

[0191] The data extension loss evaluation value may be, for example, an extension loss, which is a linear sum of a value that decreases as the class dependency of the probability indicated by the main class membership probability distribution for the extended data obtained in the second main classification process decreases, and a value that decreases as the class dependency of the probability indicated by the auxiliary class membership probability distribution for the extended data increases.

[0192] However, as described above, further improvement in accuracy can be expected by also using past classification information. Therefore, in the first unit process, the main classifier and auxiliary classifier may also be updated based on the result of the main classifier in a first unit process executed before the currently executing first unit process, which uses the processing target as original data. In this case, further improvement in accuracy can be achieved.

[0193] As can be seen from the fact that past classification information is not used in the second unit processing, the data extension section loss evaluation value may be an extension loss whether or not past classification information is used. The data extension section loss evaluation value in the above explanation using the mathematical formula is also a type of extension loss.

[0194] The control unit 11 may be implemented using a plurality of information processing devices connected to each other via a network so that they can communicate with each other. In this case, the processes executed by the control unit 11 may be distributed among the plurality of information processing devices.

[0195] The control unit 21 may be implemented using a plurality of information processing devices connected to each other via a network so that they can communicate with each other. In this case, the processes executed by the control unit 21 may be distributed among the plurality of information processing devices.

[0196] Note that all or part of the functions of classification system 100, learning device 1, or classification device 2 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may be transmitted via a telecommunications line.

[0197] The control unit 21 is an example of a classification unit.

[0198] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0199] 100...Classification system, 1...Learning device, 2...Classification device, 11...Control unit, 12...Interface unit, 13...Memory unit, 21...Control unit, 22...Interface unit, 23...Memory unit, 91...Processor, 92...Memory, 93...Processor, 94...Memory

Claims

1. a control unit that performs learning of a learning object model, which is a mathematical model of a learning object; Equipped with The learning model includes: a data augmentation process that performs probabilistically determined data augmentation on each piece of original data belonging to a dataset that is a collection of original data that is unsupervised data in a target domain; a main classifier that obtains a feature vector of the data to be processed and a main classification belonging probability distribution that indicates the probability that the data belongs to each class that is previously determined as a candidate for belonging; and an auxiliary classifier that obtains a feature vector of augmented data that is a result of the data augmentation and an auxiliary classification belonging probability distribution that indicates the probability that the augmented data belongs to each of the classes, the values ​​of the parameters of the main classifier and the auxiliary classifier are different; In the learning, the main classifier and the auxiliary classifier are updated based on the result of the main classifier which uses the processing target as original data and the result of the main classifier which uses the processing target as extended data, and the content of the data extension process is updated based on the result of the main classifier which uses the processing target as extended data and the result of the auxiliary classifier. Learning device.

2. In the learning, the main classifier and the auxiliary classifier are updated by executing a first unit process that updates the main classifier and the auxiliary classifier based on an execution result of the learning target model; In the first unit process, the main classifier and the auxiliary classifier are updated based also on the result of the main classifier in a first unit process executed before the currently executed first unit process, the main classifier using the processing target as original data. The learning device according to claim 1 .

3. In the learning, the main classifier and the auxiliary classifier are updated based on: a first value that decreases as the similarity between a first probability distribution, which is the output of the main classifier to which either first original data, which is any one of original data belonging to the dataset, or a result of a data augmentation process on the first original data, is higher; and a second probability distribution, which is the main classification membership probability distribution for the first original data in a first unit process currently being executed, or the main classification membership probability distribution for the first original data in a first unit process executed before the currently being executed first unit process; and a value that decreases as the similarity between the first probability distribution and a third probability distribution, which is the output of the main classifier to which second original data, which is any one of original data different from the first original data among the original data belonging to the dataset, is input. The learning device according to claim 1 .

4. a control unit that performs learning of a learning target model, which is a mathematical model of a learning target, wherein the learning target model includes: a data augmentation process that performs probabilistically determined data augmentation on each piece of original data belonging to a dataset, which is a set of original data that is unsupervised data in a target domain; a main classifier that obtains a feature vector of the data to be processed and a main classification assignment probability distribution that indicates the probability that the data belongs to each class that is predetermined as a candidate for assignment; and an auxiliary classifier that obtains a feature vector of extended data that is a result of the data augmentation and an auxiliary classification assignment probability distribution that indicates the probability that the extended data belongs to each of the classes, wherein the parameter values ​​of the main classifier and the parameter values ​​of the auxiliary classifier are different, and during the learning, the main classifier and the auxiliary classifier are updated based on the result of the main classifier using the original data to be processed and the result of the main classifier using the extended data to be processed, and the content of the data augmentation process is updated based on the result of the main classifier and the result of the auxiliary classifier using the extended data to be processed; A classification device comprising:

5. a control unit that performs learning of a learning object model, which is a mathematical model of a learning object, the learning object model including: a data augmentation process that performs probabilistically determined data augmentation on each piece of original data belonging to a dataset, the dataset being a set of original data that is unsupervised data of a target domain; a main classifier that obtains a feature vector of the data to be processed and a main classification membership probability distribution indicating the probability that the data belongs to each class that is predetermined as a candidate for membership; and an auxiliary classifier that obtains a feature vector of extended data that is a result of the data augmentation and an auxiliary classification membership probability distribution indicating the probability that the extended data belongs to each of the classes, the main classifier and the auxiliary classifier having different parameter values, and the main classifier and the auxiliary classifier are updated in the learning based on the result of the main classifier using the original data as the processing object and the result of the main classifier using the extended data as the processing object, and the content of the data augmentation process is updated based on the result of the main classifier and the auxiliary classifier using the extended data as the processing object, a control step for performing the learning; A learning method that has

6. a control unit that learns a learning target model, which is a mathematical model of a learning target, wherein the learning target model includes: a data augmentation process that performs probabilistically determined data augmentation on each piece of original data belonging to a dataset, which is a set of original data that is unsupervised data in a target domain; a main classifier that obtains a feature vector of the data to be processed and a main classification membership probability distribution that indicates the probability that the data belongs to each class that is predetermined as a candidate for membership; and an auxiliary classifier that obtains a feature vector of extended data that is a result of the data augmentation and an auxiliary classification membership probability distribution that indicates the probability that the extended data belongs to each of the classes, wherein values ​​of parameters of the main classifier and the auxiliary classifier are different, and during the learning, the main classifier and the auxiliary classifier are updated based on a result of the main classifier using the original data to be processed and a result of the main classifier using the extended data to be processed, and details of the data augmentation process are updated based on a result of the main classifier and a result of the auxiliary classifier using the extended data to be processed. a classification step of performing the classification; A classification method having the following structure:

7. A program for causing a computer to function as the learning device according to any one of claims 1 to 3.

8. A program for causing a computer to function as the classification device according to claim 4.

Images