Training device, training method, and program

Abstract

It provides an efficient and stable training process even when only a small amount of target domain labeled data is available. The training device (1) comprises feature extraction means (11) for extracting source domain feature values ​​from source domain data and for extracting target domain feature values ​​from target domain data, prediction means (12) for predicting source domain class predicted values ​​from the source domain feature values ​​and for predicting the target domain class predicted values ​​from the target domain feature values, filtering means (13) for calculating filtered source domain feature values ​​by filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predicted values ​​and for calculating filtered target domain feature values ​​by filtering out one or more values ​​from the target domain feature values ​​with reference to the target domain class predicted values, and updating means (14) for updating at least one of the feature extraction means and the class prediction means.

Description

[Technical field]

[0001] This application relates to a training device. , lesson Training method , and and programs. [Background technology]

[0002] Machine learning techniques such as neural networks often require large amounts of labeled data to train their parameters, yet it is often the case that only small amounts of labeled data are available.

[0003] So-called domain adaptation techniques have been proposed that may be applicable in such cases (see, for example, non-patent document 1). Domain adaptation techniques utilize a large number of alternative representative datasets (often called source domain datasets) in addition to a target dataset (often called target domain dataset). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Xiang Xu etal., “d-SNE: Domain Adaptation using Stochastic Neighborhood Embedding”, Proceedings of IEEE conference on computer vision and pattern recognition. 2019 Summary of the Invention [Problem to be solved by the invention]

[0005] The domain adaptation technique disclosed in Non-Patent Document 1 has a problem in that the training process becomes unstable due to the influence of outliers, making it impossible to perform the training process efficiently.

[0006] The present invention has been made in consideration of the above-mentioned problems, and one example of its objective is to provide an efficient and stable training process even in situations where only a small amount of target domain labeled data is available. [Means for solving the problem]

[0007] To achieve the above object, the training apparatus comprises one or more feature extraction means for extracting source domain feature values ​​from source domain data and extracting target domain feature values ​​from target domain data; one or more class prediction means for predicting source domain class predicted values ​​from the source domain feature values ​​and predicting the target domain class predicted values ​​from the target domain feature values; one or more filtering means for calculating filtered source domain feature values ​​and filtered target domain feature values ​​by filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predicted values ​​and filtering out one or more values ​​from the target domain feature values ​​with reference to the target domain class predicted values; and updating means for updating at least one of the one or more feature extraction means and the one or more class prediction means with reference to a source domain classification loss calculated with reference to the source domain class predicted values, a target domain classification loss calculated with reference to the target domain class predicted values, and a group loss calculated with reference to the filtered source domain feature values ​​and the filtered target domain feature values.

[0008] To achieve the above object, the classification apparatus comprises a feature extraction means for extracting target domain feature values ​​from target domain data, and a class prediction means for predicting a target domain class prediction value from the target domain feature values, wherein at least one of the feature extraction means and the class prediction means is trained with reference to filtered source domain feature values ​​obtained by filtering out one or more values ​​from the source domain feature values, and filtered target domain feature values ​​obtained by filtering out one or more values ​​from the target domain feature values.

[0009] To achieve the above object, the training method includes: extracting source domain feature values ​​from the source domain data and extracting target domain feature values ​​from the target domain data by one or more feature extraction means; predicting source domain class predicted values ​​from the source domain feature values ​​and predicting the target domain class predicted values ​​from the target domain feature values ​​by one or more class prediction means; calculating filtered source domain feature values ​​and filtered target domain feature values ​​by filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predicted values ​​and filtering out one or more values ​​from the target domain feature values ​​with reference to the target domain class predicted values; and updating at least one of the one or more feature extraction means and the one or more class prediction means with reference to a source domain classification loss calculated with reference to the source domain class predicted values, a target domain classification loss calculated with reference to the target domain class predicted values, and a group loss calculated with reference to the filtered source domain feature values ​​and the filtered target domain feature values.

[0010] To achieve the above object, the classification method includes extracting target domain feature values ​​from target domain data by a feature extraction means, and predicting target domain class prediction values ​​from the target domain feature values ​​by a class prediction means, wherein at least one of the feature extraction means and the class prediction means is trained with reference to filtered source domain feature values ​​obtained by filtering out one or more values ​​from the source domain feature values ​​and filtered target domain feature values ​​obtained by filtering out one or more values ​​from the target domain feature values.

[0011] In order to achieve the above object, a program for causing a computer to function as a training device, the program causing a computer to function as feature extraction means, class prediction means, filtering means, and update means.

[0012] In order to achieve the above object, a program for causing a computer to function as a classification device, the program causing a computer to function as feature extraction means and class prediction means. Effect of the Invention

[0013] Exemplary aspects of the present invention provide an efficient and robust training process even in situations where only a small amount of target domain labeled data is available. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing a configuration of a training device according to a first exemplary embodiment. [Diagram 2] FIG. 2 is a flow diagram illustrating the flow of a training method according to a first exemplary embodiment. [Diagram 3] FIG. 3 is a block diagram showing a configuration of a classification device according to a first exemplary embodiment. [Figure 4] FIG. 4 is a flow diagram illustrating the flow of a classification method according to the first exemplary embodiment. [Diagram 5] FIG. 5 is a diagram showing a configuration of a training device according to a second exemplary embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of input data in the second exemplary embodiment. [Figure 7] FIG. 7 is a schematic diagram illustrating group loss in the second exemplary embodiment. [Figure 8] FIG. 8 is a flow diagram illustrating the flow of a training method according to a second exemplary embodiment. [Figure 9] FIG. 9 is a block diagram showing a configuration of a classification device according to the second exemplary embodiment. [Figure 10] FIG. 10 is a block diagram showing a configuration of a training device according to a third exemplary embodiment. [Figure 11] FIG. 11 is a diagram showing a configuration of a training device according to a fourth exemplary embodiment. [Figure 12] FIG. 12 is a diagram illustrating domain invariance in the fourth exemplary embodiment. [Figure 13] FIG. 13 is a flow diagram illustrating the flow of a training method according to a fourth exemplary embodiment. [Figure 14] FIG. 14 is a block diagram showing a configuration of a training device according to a fifth exemplary embodiment. [Figure 15] FIG. 15 is a block diagram showing a hardware configuration according to each exemplary embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] First Exemplary Embodiment DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first exemplary embodiment according to the present invention will now be described with reference to the accompanying drawings. The first exemplary embodiment is an embodiment on which subsequent exemplary embodiments are based.

[0016] Overview of the First Exemplary Embodiment A training device (also called a learning device) according to this exemplary embodiment trains (also called learns or makes learn) a feature extractor that extracts features from data input to the training device, and trains a class predictor that performs classification based on feature values ​​(also called features).

[0017] A classification device according to a first exemplary embodiment of the present invention comprises a trained feature extractor and a trained class predictor.

[0018] In the first exemplary embodiment, data belonging to a source domain and data belonging to a target domain are used, where "domain" is merely a conceptual distinction between data sets and is not intended to limit the first exemplary embodiment.

[0019] In the first exemplary embodiment, the data size of the data belonging to the source domain is larger than that of the data belonging to the target domain. Moreover, the data belonging to the source domain includes more labeled data compared to the data belonging to the target domain. Here, the term "labeled data" refers to data labeled as "ground truth". The labeled data can be used by the training device for supervised learning or semi-supervised learning.

[0020] The classification device according to the first exemplary embodiment of the present invention is intended to perform classification on data belonging to a target domain. In order to improve the classification accuracy of data belonging to the target domain, the training device according to the first exemplary embodiment performs a training process using not only data belonging to the target domain but also data belonging to the source domain.

[0021] In the classification device that performs the above processing, knowledge obtained from the source domain is also used for training in the target domain. Therefore, the classification device according to the first exemplary embodiment can be expressed as a classification device that performs so-called domain adaptation.

[0022] The type of data input to the training device and the classifier device does not limit the first exemplary embodiment, although it is preferred that the data belonging to the source domain and the data belonging to the target domain belong to the same category.

[0023] As an example, the data belonging to the source domain and the data belonging to the target domain input to the training device and the classification device may be video data, sensing data obtained by various sensors, digitized language data, or other types of data.

[0024] (Training Equipment Configuration) Next, the configuration of a training device 10 according to a first exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the training device 10. As shown in Fig. 1, the training device 10 includes a feature extraction unit 11, a class prediction unit 12, a filter unit (also called a filtering unit) 13, and an update unit 14. In the first exemplary embodiment, the number of the feature extraction unit 11, the class prediction unit 12, and the filter unit 13 may be one, or two or more.

[0025] The feature extraction unit 11 is an example of a feature extraction means as recited in the claims. The class prediction unit 12 is an example of a class prediction means as recited in the claims. The filter unit 13 is an example of a filtering means as recited in the claims. The update unit 14 is an example of an update means as recited in the claims.

[0026] Source domain data and target domain data are input to the feature extraction unit 11. Here, the source domain data is data that belongs to the source domain, and the target domain data is data that belongs to the target domain.

[0027] The feature extraction unit 11 extracts source domain feature values ​​(also called source domain features) from the source domain data, and also extracts target domain feature values ​​(also called target domain features) from the target domain data.

[0028] Here, when the number of feature extraction units 11 is one, it is preferable to configure the feature extraction unit 11 to input the domain label for distinguishing between the source domain data and the target domain data so that the feature extraction unit 11 can distinguish between the source domain data and the target domain data based on the domain label.

[0029] Note that the specific configuration of the feature extraction unit 11 is not limited to the first exemplary embodiment. As an example, the feature extraction unit 11 may be a CNN (convolutional neural network), an RNN (recurrent neural network), or other neural networks or feature extractors. Also, the feature extraction unit 11 may be a technology that does not rely on learning. As an example, the feature extraction unit 11 can be realized as a SIFT (Scale-Invariant Feature Transform), that is, a non-learning type feature extraction method in which each feature value in an image is a key point (x, y coordinates) and the feature value for each image is a set of key points. Also, the feature extraction unit 11 can be realized as a manually selected / designed (rule-based) feature extraction method.

[0030] The class prediction unit 12 predicts a source domain class predicted value from the source domain feature values ​​extracted by the feature extraction unit 11, and outputs the source domain class predicted value. The class prediction unit 12 also predicts a target domain class predicted value from the target domain feature values ​​extracted by the feature extraction unit 11, and outputs the target domain class predicted value.

[0031] Note that the specific configuration of the class predictor 12 is not limited to the first exemplary embodiment. As an example, the class predictor 12 can be configured to (i) output a source domain class prediction value by comparing a source domain feature value with a certain threshold, and (ii) output a target domain class prediction value by comparing a target domain feature value with another threshold.

[0032] The filter unit 13 calculates filtered source domain feature values ​​by filtering one or more values ​​from the source domain feature values ​​with reference to the source domain class predicted values, and calculates filtered target domain feature values ​​by filtering one or more values ​​from the target domain feature values ​​with reference to the target domain class predicted values.

[0033] The filter conditions adopted by the filter unit 13 are not limited to those in the first exemplary embodiment, but the following configurations may be adopted.

[0034] The filter unit 13 may receive source domain class label data. Here, the source domain class label data indicates ground truth labels associated with the source domain data received by the feature extraction unit 11. The filter unit 13 may then filter one or more values ​​from the source domain feature value with reference to the source domain class prediction value and the source domain class label data. More specifically, the filter unit 13 may filter out one or more values ​​that do not match the source domain class label data from the source domain feature value.

[0035] The filter unit 13 may also receive target domain class label data. Here, the target domain class label data indicates ground truth labels associated with the target domain data received by the feature extraction unit 11. The filter unit 13 may then filter one or more values ​​from the target domain feature values ​​by referring to the target domain class prediction value and the target domain class label data. More specifically, the filter unit 13 may filter out one or more values ​​that do not match the target domain class label data from the target domain feature values.

[0036] The update unit 14 updates at least one of the feature extraction unit 11 and the class prediction unit 12 with reference to the source domain classification loss (also referred to as source domain classification loss), the target domain classification loss (also referred to as target domain classification loss), and the group loss (also referred to as grouping loss or grouping loss). As an example, the update unit 14 updates parameters with reference to the source domain classification loss, the target domain classification loss, and the group loss, and provides the updated parameters to the feature extraction unit 11 and the class prediction unit 12.

[0037] The source domain classification loss may be calculated with reference to the source domain class predictions. As an example, the source domain classification loss may be calculated with reference to the source domain class predictions and the source domain class label data.

[0038] The target domain classification loss may be calculated with reference to the target domain class predictions. As an example, the target domain classification loss may be calculated with reference to the target domain class predictions and the target domain class label data.

[0039] The group loss may be computed with reference to the filtered source domain feature values ​​and the filtered target domain feature values.

[0040] The calculation algorithm for calculating the group loss is not limited to the first exemplary embodiment, but the following configuration example can be adopted.

[0041] The group loss may be computed based on intra class metrics determined with reference to filtered feature values ​​in the same class and inter class metrics determined with reference to filtered feature values ​​in different classes.

[0042] (Beneficial Effects) According to the first exemplary embodiment, as described above, the update unit 14 refers to the group loss in addition to the source domain classification loss and the target domain classification loss to update at least one of the feature extraction unit 11 and the class prediction unit 12. Therefore, according to the first exemplary embodiment, even if the amount of target domain labeled data is small, the feature extraction unit 11 and the class prediction unit 12 can be appropriately trained.

[0043] Furthermore, according to the first exemplary embodiment, as described above, the filter unit 13 calculates the filtered source domain feature values ​​and the filtered target domain feature values, and the update unit 14 updates at least one of the feature extraction unit 11 and the class prediction unit 12 with reference to the group loss in addition to the source domain classification loss and the target domain classification loss. Here, as described above, the group loss is obtained with reference to the filtered source domain feature values ​​and the filtered target domain feature values.

[0044] Therefore, according to the first exemplary embodiment, outliers that may be included in the source domain feature values ​​and the target domain feature values ​​can be filtered out by the filter unit 13, without destabilizing the learning process of the feature extraction unit 11.

[0045] Therefore, the training apparatus 10 according to the first exemplary embodiment can provide an efficient and stable training process even with a small amount of labeled data in the target domain.

[0046] (Training method using training equipment) Next, a training method of the training device 10 according to the first exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the training method S1 by the training device 10. As shown in Fig. 2, the training device 10 executes a feature extraction step S11, a class prediction step S12, a filter step S13, and an update step S14.

[0047] (Feature extraction step S11) In a feature extraction step S11, the feature extraction unit 11 extracts source domain feature values ​​from the source domain data and extracts target domain feature values ​​from the target domain data.

[0048] (Class prediction step S12) In a class prediction step S12, the class prediction unit 12 predicts a source domain class prediction value from the source domain feature values, and predicts a target domain class prediction value from the target domain feature values.

[0049] (Filter step S13) In the filtering step S13, the filter unit 13 filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predictors; By filtering out one or more values ​​from the target domain feature values ​​with reference to the target domain class predictor. Output the filtered source domain feature values ​​and the filtered target domain feature values.

[0050] (Update step S14) In the update step S14, the update unit The source domain classification loss calculated with reference to the source domain class predictions, A target domain classification loss calculated with reference to the target domain class prediction value, and Group loss calculated with reference to filtered source domain features and filtered target domain features , and updates at least one of the feature extraction unit 11 and the class prediction unit 12.

[0051] (Beneficial Effects) The training method S1 by the training apparatus 10 described above provides an efficient and stable training process even when only a small amount of target domain labeled data is available.

[0052] (Classification device configuration) Hereinafter, a configuration of the classification device 20 according to the first exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the classification device 20. As shown in Fig. 3, the classification device 20 includes a feature extraction unit 11 and a class prediction unit 12.

[0053] Target domain data is input to the feature extraction unit 11. The feature extraction unit 11 extracts target domain feature values ​​from the target domain data. As described for the training device 10, at least one of the feature extraction unit 11 and the class prediction unit 12: filtered source domain feature values ​​obtained by filtering one or more values ​​from the source domain feature values; The feature vector is trained with reference to filtered target domain feature values ​​obtained by filtering one or more values ​​from the target domain feature values.

[0054] The class prediction unit 12 predicts a target domain class prediction value from the target domain feature value.

[0055] According to the classification device 20, at least one of the feature extraction unit 11 and the class prediction unit 12 is trained with reference to (i) filtered source domain feature values ​​obtained by filtering one or more values ​​from the source domain feature values, and (ii) filtered target domain feature values ​​obtained by filtering one or more values ​​from the target domain feature values.

[0056] Therefore, the classification device 20 according to the embodiment of the first example provides suitable classification processing even when the amount of target domain labeled data is small.

[0057] (Classification method using a classification device) Next, a classification method of the classification device 20 according to the first exemplary embodiment will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing the classification method S2 performed by the classification device 20. As shown in Fig. 4, the classification device 20 executes a feature extraction step S11 and a class prediction step S12.

[0058] (Feature extraction step S11) In the feature extraction step S11, the feature extraction unit 11 extracts target domain feature values ​​from the target domain data. As described for the training device 10, at least one of the feature extraction unit 11 and the class prediction unit 12: filtered source domain feature values ​​obtained by filtering one or more values ​​from the source domain feature values; The feature vector is trained with reference to filtered target domain feature values ​​obtained by filtering one or more values ​​from the target domain feature values.

[0059] (Class prediction step S12) In a class prediction step S12, the class prediction unit 12 predicts a target domain class prediction value from the target domain feature value.

[0060] The classification method by the classifier 20 according to the first exemplary embodiment provides suitable classification processing even when the target domain labeled data is small.

[0061] Second Exemplary Embodiment The second exemplary embodiment will be described below with reference to the drawings. Elements having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and their description will be omitted as appropriate. The outline of the second exemplary embodiment is the same as that of the first exemplary embodiment, so it will not be described here.

[0062] (Training Equipment Configuration) Next, the configuration of a training device 10a according to an exemplary embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the training device 10a. As shown in Fig. 5, the training device 10a includes a first feature extractor 11a, a second feature extractor 11b, a first class predictor 12a, a second class predictor 12b, a first filter unit 13a, a second filter unit 13b, and an update unit 14a.

[0063] The first feature extraction unit 11a and the second feature extraction unit 11b are an example of a feature extraction means as set forth in the claims. The first class prediction unit 12a and the second class prediction unit 12b are an example of a class prediction means as set forth in the claims. The first filter unit 13a and the second filter unit 13b are an example of a filtering means as set forth in the claims. The update unit 14a is an example of an update means as set forth in the claims.

[0064] The first feature extraction unit 11a and the second feature extraction unit 11b can be configured as a single section. The first class prediction unit 12a and the second class prediction unit 12b can be configured as a single section. The first filter unit 13a and the second filter unit 13b can be configured as a single section.

[0065] As shown in FIG. 5, the update unit 14a includes a grouping unit 141, a group loss calculation unit 142, a first classification loss calculation unit 143a, a second classification loss calculation unit 143b, a merge loss calculation unit 144, and a model update unit 145.

[0066] The first classification loss calculation unit 143a and the second classification loss calculation unit 143b can be configured as one section.

[0067] (First feature extraction unit) The first feature extraction unit 11a extracts input data I S The input data I S Although the specific example of does not limit the second exemplary embodiment, the input data I S may be one or more input images. More specifically, as an example, the input data I S As another example, the input image data I S may be a batch of images as shown on the left side of Fig. 6. In the example on the left side of Fig. 6, the input data I S contains a batch of 10 images, each of which represents a digit or multiple digits.

[0068] The first feature extraction unit 11a extracts the source domain data I S From feature value (also called feature quantity) X S The feature value X extracted by the first feature extraction unit 11a is S is supplied to a first class prediction unit 12a and a first filter unit 13a.

[0069] Note that the specific configuration of the first feature extraction unit 11a is not limited to the second exemplary embodiment. As an example, the first feature extraction unit 11a may be a convolutional neural network (CNN), a recurrent neural network (RNN), or any other neural network or feature extractor. Also, the first feature extraction unit 11a may be a technology that does not rely on learning. As an example, the first feature extraction unit 11a can be realized as SIFT (Scale-Invariant Feature Transform), that is, a non-learning type feature extraction method in which each feature value in an image is a key point (x, y coordinates) and the feature value for each image is a set of key points. Also, the first feature extraction unit 11a can be realized as a manually selected / designed (rule-based) feature extraction method.

[0070] In addition, the feature value X S Although the structure of is not limiting to the second exemplary embodiment, the feature value X S may be expressed in vector form. More specifically, as an example, X S may be expressed as a vector as follows:

[0071]

number

[0072] The first feature extractor 11a may extract one feature value consisting of multiple real numbers from each image. Therefore, the feature value may be called a feature vector.

[0073] (Second feature extraction unit) Similarly, the second feature extraction unit 11b receives input data I T The input data I TAlthough the specific example of does not limit the second exemplary embodiment, the input data I T may be one or more input images. More specifically, as an example, the input data I T As another example, the input image data I T may be a set of images as shown on the right side of Fig. 6. In the example on the right side of Fig. 6, the input data I T contains 10 images, each of which represents a number or multiple numbers.

[0074] The second feature extraction unit 11b extracts the target domain data I T From feature value X T The feature value X extracted by the second feature extraction unit 11b is T is supplied to a second class prediction unit 12b and a second filter unit 13b.

[0075] The specific configuration of the second feature extraction unit 11b does not limit the second exemplary embodiment. For example, the second feature extraction unit 11b may be a convolutional neural network (CNN), a recurrent neural network (RNN), or other neural networks or feature extractors. The second feature extraction unit 11b may be a non-learning technology. As an example, the second feature extraction unit 11b may be realized as a scale-invariant feature transform (SIFT), that is, a non-learning feature extraction method in which each feature amount in an image is a key point (x, y coordinates) and the feature amount for each image is a set of key points. The second feature extraction unit 11b may be realized as a manually selected / designed (rule-based) feature extraction method.

[0076] In addition, the feature value X T Although the structure of is not limiting to the second exemplary embodiment, the feature value X T may be expressed in vector form. More specifically, as an example, X Tmay be expressed as a vector as follows:

[0077]

number

[0078] (First class prediction part) The first class prediction unit 12a uses the source domain feature X extracted by the first feature extraction unit 11a. S From the source domain class prediction value P S Predict.

[0079] The source domain class prediction value P output by the first class prediction unit 12a S is supplied to the first filter unit 13a and the first classification loss calculation unit 143a.

[0080] Note that the specific configuration of the first class prediction unit 12a is not limited to the second exemplary embodiment. As an example, S By comparing each element of S A configuration can be adopted in which the corresponding elements of

[0081] As a concrete example, the source domain feature vector X S From this, the first class prediction unit 12a calculates the source domain class prediction value P S can be output as follows:

[0082]

number

[0083] (Second class prediction part) The second class prediction unit 12b predicts the target domain feature value X extracted by the second feature extraction unit 11b. T From the target domain class prediction value P T Predict.

[0084] The target domain class prediction value P output by the second class prediction unit 12b T is supplied to the second filter unit 13b and the second classification loss calculation unit 143b.

[0085] Note that the specific configuration of the second class prediction unit 12b is not limited to the second exemplary embodiment. As an example, the target domain feature vector X T By comparing each element of P with a given threshold, the target domain class prediction value P T A configuration can be adopted in which the corresponding elements of

[0086] As a concrete example, the target domain feature vector X S From this, the second class prediction unit 12b calculates the target domain class prediction value P T can be output as follows:

[0087]

number

[0088] (First filter section) The first filter unit 13a filters the source domain class prediction value P S Refer to the source domain feature value X SFilter out one or more values ​​from the filtered source domain feature values ​​X' S Calculate.

[0089] The filtered source domain feature values ​​X′ calculated by the first filter unit 13a S is supplied to the grouping unit 141.

[0090] The filter conditions adopted by the first filter unit 13a are not limited to those in the second exemplary embodiment, but the following configuration may be adopted.

[0091] The first filter unit 13a filters the source domain class label data Y S Here, the source domain class label data Y S is the source domain input data I S associated with the source domain input data I S Specifically, the input data I S When the image data is as shown on the left side of Figure 6, the source domain class label data Y S may be expressed as follows:

[0092]

number

[0093] The first filter unit 13a then filters (i) the source domain class prediction vector P S and (ii) the source domain class label vector Y S Using each element of and, the source domain feature vector X SMore specifically, the first filter unit 13a may filter out one or more elements from the source domain feature vector X S From the source domain class label vector Y S One or more elements that do not match a corresponding element of may be filtered out.

[0094] As a concrete example, in (Eq.3), P S is given, and in (Eq.5) Y S Given P S The component of Y S Therefore, in this example, the first filter unit 13a does not have to filter out elements of the feature vector because the elements match the corresponding elements of the source domain feature vector X′ calculated by the first filter unit 13a. S may be expressed as follows:

[0095]

number

[0096] The filtered target domain feature value X′ calculated by the second filter unit 13b T is supplied to the grouping unit 141.

[0097] The filter conditions adopted by the second filter unit 13b are not limited to those in the second exemplary embodiment, but the following configuration may be adopted.

[0098] The second filter unit 13b filters the target domain class label data Y THere, the target domain class label data Y T is the target domain input data I T and the target domain input data I T As a specific example, the ground truth labels of input data I T When the image data is as shown on the left side of Figure 6, the target domain class label data Y T may be expressed as follows:

[0099]

number

[0100] Then, the second filter unit 13b filters (i) the target domain class prediction vector P T and (ii) the target domain class label vector Y T With reference to each element of and, the target domain feature vector X T More specifically, the second filter unit 13b may filter out one or more components from the target domain feature vector X T From the target domain class label vector Y T may filter out one or more elements that do not match a corresponding element in

[0101] As a concrete example, in (Eq.4), P T is given, and in (Eq.7) Y T Given P T The second component of Y T does not match the second component of P T The seventh component of is Y T does not match the seventh component of

[0102] Therefore, in this specific example, the second filter unit 13b filters the target domain feature vector X TTherefore, in this example, the filtered target domain feature vector X′ calculated by the second filter unit 13b may be filtered out. T may be expressed as follows:

[0103]

number

[0104] (Grouping section) The grouping unit 141 groups the filtered source domain feature values ​​X' S and the filtered target domain feature value X' T From the,classes, we generate class groups, where each class group contains,features that share the same class label.

[0105] As a concrete example, in (Eq.6), X' S is given, and in (Eq.8) we get X' T Given this, the grouping unit 141 outputs the following class groups (Gr0, Gr1, Gr2, Gr3, Gr4).

[0106]

number

[0107] (Group Loss Calculation Department) The group loss calculation unit 142 refers to the class groups generated by the grouping unit 141 and calculates a group loss (Loss_grouping).

[0108] As an example, the group loss calculation unit 142 calculates the group loss based on (i) the intra-class distance determined with reference to the features of the same class, and (ii) the inter-class distance determined with reference to the features of other classes.

[0109] As a specific example, the group loss calculation unit 142 calculates the group loss using the following formula.

[0110]

number

[0111] More specifically, the calculation of group loss (Loss_grouping) according to (Eq.10) can be expressed as follows:

[0112] The group loss calculation unit 142 may first determine, for each feature value x, the feature value that is farthest from the feature value x in the feature space among all other feature values ​​from the same class group as the feature value x. This type of pair is sometimes called an intra-class pair. The distance between the intra-class pairs of each feature value corresponds to the "maximum of intra-class distance in the feature space" in (Eq.10).

[0113] Then, for each feature value x, the group loss calculation unit 142 may find the other feature value that is closest to the feature value x in the feature space among all other feature values ​​from a class group different from the feature value x. This type of pair is sometimes called an inter-class pair. The distance between the inter-class pairs of each feature corresponds to the "minimum of inter-class distance in the feature space" in (Eq.10).

[0114] Then, the group loss calculation unit 142 may subtract (the minimum value of the inter-class distance in the feature space) from (the maximum value of the inter-class distance in the feature space) for each feature.

[0115] Then, the group loss calculation unit 142 may add a margin. Adding a margin means making the amount (maximum of the inter-class distance in the feature space) smaller than (minimum of the inter-class distance in the feature space) at least a constant value (e.g., if the margin=1, it is desired that for each feature value, (maximum of the inter-class distance in the feature space) is at least one unit smaller than (minimum of the inter-class distance in the feature space)).

[0116] After performing the above calculations for each feature value, the group loss calculation unit 142 calculates XS and X T We can take the sum of all feature values ​​contained in the union with

[0117] Then, the group loss calculation unit 142 may divide the sum by the number of all features. The number of all features is represented as n in (Eq.10). As a specific example, when there is a class group of (Eq.9), n=18 can be used.

[0118] (First class classification loss calculation part) The first class classification loss calculation unit 143a calculates (i) a source domain class prediction value P S ,(ii) source domain class label data Y S Calculate the source domain classification loss (Loss_classification_S) by referring to

[0119] As an example, the first class classification loss calculation unit 143a calculates P S and Y S The source domain classification loss is calculated according to the degree of discrepancy with the

[0120] As a concrete example, P S is given, and by (Eq.5) Y S Given P S All components of Y S Since the first class classification loss calculation unit 143a calculates the source domain classification loss as follows:

[0121]

number

[0122] As an example, the second class classification loss calculation unit 143b calculates P T and Y T The target domain classification loss is calculated according to the degree of discrepancy with the

[0123] As a concrete example, in (Eq.4), P T is given, and in (Eq.7) Y T Given P T The second component of Y T does not match the second component of P T The seventh component of is Y T Since the seventh component of the target domain does not match the seventh component of the target domain, the second class classification loss calculation unit 143b calculates the target domain classification loss as follows.

[0124]

number

[0125] As an example, the merge loss calculation unit 144 calculates the merge loss as follows.

[0126]

number

[0127] (Model Update Part 145) The model update unit 145 determines whether the merge loss has converged. If the merge loss has converged, the model update unit 145 outputs the converged model parameters to a storage medium. If the merge loss value has not converged, the model update unit 145 refers to the merge loss calculated by the merge loss calculation unit 144 and updates the model parameters of the first feature extraction unit 11a, the second feature extraction unit 11b, the first class prediction unit 12a, and the second class prediction unit 12b.

[0128] For example, the model update unit 145 updates the model parameters so as to reduce the merge loss. Note that the update method adopted by the model update unit 145 does not limit the second exemplary embodiment, but the model update unit 145 may update the model parameters according to a back propagation method.

[0129] The model parameters updated by the model update unit 145 are supplied to the first feature extraction unit 11a, the second feature extraction unit 11b, the first class prediction unit 12a, and the second class prediction unit 12b.

[0130] (Beneficial Effects) According to the second exemplary embodiment, as described above, the model update unit 145 updates the model parameters by referring to the group loss in addition to the source domain classification loss and the target domain classification loss. Therefore, according to the second exemplary embodiment, even if only a small amount of target domain labeled data is available, the second feature extraction unit 11b and the second class prediction unit 12b can be trained.

[0131] Also, according to the second exemplary embodiment, as described above, the first filter unit 13a and the second filter unit 13b calculate the filtered source domain features and the filtered target domain features, and the model update unit 145 updates the model parameters with reference to the group loss in addition to the source domain classification loss and the target domain classification loss. Here, as described above, the group loss is obtained with reference to the filtered source domain feature values ​​and the filtered target domain feature values.

[0132] Therefore, according to the second exemplary embodiment, outliers that may be contained in the source domain feature values ​​and the target domain feature values ​​are filtered out by the first filter unit 13a and the second filter unit 13b, and do not destabilize the learning processes of the first feature extraction unit 11a, the second feature extraction unit 11b, the first class prediction unit, and the second class prediction unit 12b.

[0133] The above features will be described below with reference to FIG. 7. The upper half of FIG. 7 is a diagram that illustrates a group loss when the first filter unit 13a and the second filter unit 13b do not perform filtering. As shown in the upper half of FIG. 7, if the first filter unit 13a and the second filter unit 13b do not perform filtering, the group loss becomes inappropriately large due to the contribution of erroneously classified outliers. Therefore, when updating a model using such a group loss, the loss does not converge, and training cannot be performed appropriately.

[0134] On the other hand, the lower half of Fig. 7 is a diagram showing a schematic diagram of the group loss when filtering is performed by the first filter unit 13a and the second filter unit 13b. As shown in the lower half of Fig. 7, when filtering is performed by the first filter unit 13a and the second filter unit 13b, the erroneously classified outliers do not contribute to the group loss. In this way, by updating the model using the group loss, the loss value converges, and training can be performed appropriately.

[0135] Therefore, the training apparatus 10a according to the second exemplary embodiment can provide an efficient and stable training process even with a small amount of labeled data in the target domain.

[0136] (Training method using training equipment) Hereinafter, a training method of the training device 10a according to the second exemplary embodiment will be described with reference to Fig. 8. Fig. 8 is a flow chart showing the flow of the training method S1a by the training device 10a.

[0137] (Step S100) In step S100, the training device 10a receives initial model parameters. The initial model parameters include initial model parameters of the first feature extraction unit 11a, the second feature extraction unit 11b, the first class prediction unit 12a, and the second class prediction unit 12b. The initial model parameters received in this step are provided to the first feature extraction unit 11a, the second feature extraction unit 11b, the first class prediction unit 12a, and the second class prediction unit 12b.

[0138] (Step S10a) In step S10a, the training device 10a receives source domain data. More specifically, the training device 10a receives source domain input data I S and the input data I S Source domain class label data Y associated with S Receive.

[0139] (Step S10b) In step S10b, the training device 10a receives data of the target domain. More specifically, the training device 10a receives the target domain input data I T and input data I T The target domain class label data Y associated with T Receive.

[0140] (Step S11a) In step S11a, the first feature extraction unit 11a extracts the source domain data I S From feature value X S Note that the specific process performed by the first feature extraction unit 11a has been described above, and therefore will not be described here repeatedly.

[0141] (Step S11b) In step S11b, the second feature extraction unit 11b extracts the target domain data I T From feature value X T Note that the specific process performed by the second feature extraction unit 11b has been described above, and therefore will not be described here repeatedly.

[0142] (Step S12a) In step S12a, the first class prediction unit 12a calculates the source domain feature value X S From the source domain class prediction value P S Note that the specific process performed by the first class predictor 12a has been described above, and therefore will not be described here repeatedly.

[0143] (Step S12b) In step S12b, the second class prediction unit 12b calculates the target domain feature value X T From the target domain class prediction value P T Note that the specific process performed by the second class predictor 12b has been described above, and therefore will not be described here repeatedly.

[0144] (Step S13a) In step S13a, the first filter unit 13a calculates the source domain class prediction value P S Refer to the source domain feature value X S Filter out one or more values ​​from the filtered source domain feature values ​​X' SNote that the specific process performed by the first filter unit 13a has been described above, and therefore will not be described here repeatedly.

[0145] (Step S13b) In step S13b, the second filter unit 13b calculates the target domain class prediction value P T Refer to the target domain feature value X T Filter out one or more values ​​from the filtered target domain feature value X' T Note that the specific process performed by the second filter unit 13b has been described above, and will not be described here repeatedly.

[0146] (Step S141) In step S141, the grouping unit 141 selects the filtered source domain feature values ​​X′ S and the filtered target domain feature value X' T From the above, the grouping unit 141 generates and outputs class groups. Here, each class group includes features that share the same class label. Note that the specific processing performed by the grouping unit 141 has been described above, and therefore will not be described here repeatedly.

[0147] (Step S142) In step S142, the group loss calculation unit 142 calculates a group loss (Loss_grouping) by referring to the class groups generated by the grouping unit 141. Note that the specific processing performed by the group loss calculation unit 142 has been described above, and therefore will not be described here repeatedly.

[0148] (Step S143a) In step S143a, the first class classification loss calculation unit 143a calculates the source domain class prediction value P S and source domain class label data Y SThe source domain classification loss (Loss_classification_S) is calculated using the above. Note that the specific processing performed by the first class classification loss calculation unit 143a has been described above, and therefore will not be described here repeatedly.

[0149] (Step S143b) In step S143b, the second class classification loss calculation unit 143b calculates the target domain class prediction value P T and the target domain class label data Y T The target domain classification loss (Loss_classification_T) is calculated using the above. Note that the specific processing performed by the second class classification loss calculation unit 143b has been described above, and therefore will not be described here repeatedly.

[0150] (Step S144) In step S144, the merge loss calculation unit 144 calculates the merge loss (Loss_merge) by referring to the source domain classification loss (Loss_classification_S), the target domain classification loss (Loss_classification_T), and the group loss (Loss_grouping). Note that the specific processing performed by the merge loss calculation unit 144 has been described above, and will not be described again here.

[0151] (Step S145) In step S145, the model update unit 145 determines whether the merge loss has converged. If the merge loss has converged (YES in step S145), the process proceeds to step S147. If the merge loss has not converged (NO in step S145), the process proceeds to step S146.

[0152] (Step S146) In step S146, the model update unit 145 refers to the merge loss calculated by the merge loss calculation unit 144 and updates the model parameters of the first feature extraction unit 11a, the second feature extraction unit 11b, the first class prediction unit 12a, and the second class prediction unit 12b.

[0153] (Step S147) In step S147, the model update unit 145 refers to the merge loss calculated by the merge loss calculation unit 144 and stores the model parameters of the first feature extraction unit 11a, the second feature extraction unit 11b, the first class prediction unit 12a, and the second class prediction unit 12b in a storage medium.

[0154] Note that the order of steps in the training method S1a does not limit the second exemplary embodiment. For example, in the training method S1a, steps S143a and S143b may be performed before steps S13a and S13b, respectively. Alternatively, steps S143a and S143b may be performed after at least one of steps S141 and S142.

[0155] (Beneficial Effects) The training method S1a by the training apparatus 10a described above provides an efficient and stable training process even when only a small amount of target domain labeled data is available.

[0156] (Classification device configuration) Next, the configuration of a classification device 20a according to a second exemplary embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of the classification device 20a. As shown in Fig. 9, the classification device 20a includes a feature extraction unit 11b and a class prediction unit 12b. Here, the feature extraction unit 11b is configured similarly to the above-mentioned second feature extraction unit 11b, and the class prediction unit 12b is configured similarly to the above-mentioned second class prediction unit 12b.

[0157] According to the classification device 20a, the feature extraction unit 11b and the class prediction unit 12b are trained with reference to (i) filtered source domain feature values ​​obtained by filtering one or more values ​​from source domain feature values, and (ii) filtered target domain feature values ​​obtained by filtering one or more values ​​from target domain feature values.

[0158] Therefore, the classification device 20a according to the second exemplary embodiment provides suitable classification processing even when the target domain labeled data is small.

[0159] (Notes on the second exemplary embodiment) (Note 1: Regarding the feature extraction part) In one configuration example, the first feature extraction unit 11a and the second feature extraction unit 11b may be completely independent, i.e., the first feature extraction unit 11a and the second feature extraction unit 11b do not share model parameters or layers.

[0160] In another exemplary configuration, the first feature extractor 11a and the second feature extractor 11b may share one or some of the model parameters, i.e., the first feature extractor 11a and the second feature extractor 11b may share one or some layers.

[0161] As an example, the first feature extraction unit 11a and the second feature extraction unit 11b may share the first few layers from which low-level features such as edges, corners, etc. are extracted. Alternatively, the first feature extraction unit 11a and the second feature extraction unit 11b may share the last few layers from which abstract features such as higher-level features and semantic meaning are extracted.

[0162] In yet another exemplary configuration, the first feature extraction unit 11a and the second feature extraction unit 11b may share all of the model parameters, i.e., the first feature extraction unit 11a and the second feature extraction unit 11b may share all layers.

[0163] (Note 2: About the class prediction section) In one configuration example, the first class predictor 12a and the second class predictor 12b may be completely independent, i.e., the first class predictor 12a and the second class predictor 12b do not share model parameters and layers.

[0164] In another exemplary configuration, the first class predictor 12a and the second class predictor 12b may share one or some of the model parameters, i.e., the first class predictor 12a and the second class predictor 12b may share one or some layers.

[0165] In yet another exemplary configuration, the first class predictor 12a and the second class predictor 12b may share all model parameters, i.e., the first class predictor 12a and the second class predictor 12b may share all layers.

[0166] (Note 3: Regarding group losses) The computation of the group loss can be accomplished by any method for computing the distance or similarity between two features in a class group. The group loss can be the L1 norm, L2 norm, cosine similarity, or some other measure that requires learning, etc.

[0167] In addition, after calculating the group loss, the group loss calculation unit 142 may perform some post-processing operations, such as rescaling the group loss, so that the group loss is rescaled to a safer range, such as within [0,1). As a specific example, the group loss calculation unit 142 may calculate a post-processed group loss (Loss_grouping_pp) by rescaling the group loss (Loss_grouping) as follows:

[0168]

number

[0169] (Note 4: About merge loss) The merge loss can be the direct sum of all subtask losses, such as the source domain classification loss (Loss_classification_S), the target domain classification loss (Loss_classification_T), and the group loss (Loss_grouping), or it can be a weighted sum of the subtask losses.

[0170] The weights in the sum may change as training progresses (e.g., weight the classification loss higher in early training iterations, then decay as the number of training iterations increases). The weights may be weights that require learning.

[0171] (Note 5: Regarding filtering rules) The rules for deciding whether to filter out a feature value may vary. As a first example, the rules may depend on the accuracy of the prediction given by the first class predictor 12a or the second class predictor 12b.

[0172] More specifically, if the label data (true labels of features) is [0,1,1,1] while the prediction given by the class predictor for the same batch of features is [0,1,0,1], then only the third feature is predicted incorrectly and therefore only the third feature can be filtered out by the filter.

[0173] As a second example, the rule may depend on the confidence of the prediction. More specifically, if (i) the true label of a feature value is [0,1,1,1] and the predicted confidence of the feature value of the same batch is [0.7,0.3], [0.45,0.55], [0.6,0.4], [0.2,0.8], and (ii) the confidence threshold is set to 0.6, the model determines that for the first feature, the confidence score of 0.7 (indicating the confidence that the feature belongs to its true class (i.e., index 0)) is greater than the threshold, and therefore the feature value can be safely passed to the next stage. For the second feature, the model is confident that the feature should belong to its true class (i.e., index 1), but the confidence is lower than the threshold (0.55<0.6), and therefore the filter unit may reject this feature value. For the third feature, the confidence that the feature value belongs to its true class (i.e., index 1) is only 0.4, and therefore this feature value may also be rejected. The confidence threshold may be manually selected or may be a threshold that requires learning.

[0174] <Third embodiment> Hereinafter, the third exemplary embodiment of the present invention will be described in detail with reference to the drawings. The elements having the same functions as those described in the above exemplary embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate. Furthermore, the outline of the third exemplary embodiment is the same as that of the above-mentioned embodiment, so it will not be described here.

[0175] (Training Equipment Configuration) Next, the configuration of a training device 10b according to a third exemplary embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of the training device 10b. As shown in Fig. 10, the third exemplary embodiment differs from the second embodiment in that a source domain classification loss (Loss_classification_S) is input to a first filter unit 13a, and a target domain classification loss (Loss_classification_T) is input to a second filter unit 13b.

[0176] In the third exemplary embodiment, the first filter unit 13a further refers to the source domain classification loss, and the second filter unit 13b refers to the target domain classification loss.

[0177] As an example, the first filter unit 13a may include: (i) a source domain class prediction vector P S (ii) each element of the source domain class label vector Y S and (iii) the source domain classification loss for each element of the source domain feature vector X S One or more elements may be filtered out from

[0178] Similarly, the second filter unit 13b filters (i) the target domain class prediction vector P T (ii) each element of the target domain class label vector Y T (iii) Refer to the target domain classification loss to find the target domain feature vector X T One or more elements may be filtered out from

[0179] As another example, the first filter unit 13a may be switched off during the training process when the source domain classification loss is less than a threshold TH1, which may be selected manually.

[0180] Similarly, the second filter unit 13b may be switched off during the training process if the target domain classification loss is less than a threshold TH2. The threshold TH2 may or may not be the same value as TH1. The threshold TH2 may be manually selected.

[0181] Even if a feature is misclassified after several training epochs, such a feature is not too far from other features and is therefore worth considering. To determine the stage at which misclassified but valuable features are used in the group loss calculation, the classification loss can be used as guidance, as explained above.

[0182] (Beneficial Effects) According to the third exemplary embodiment, the misclassified but valuable features can be appropriately utilized in the training process.

[0183] (Classification device configuration) The classification device 20b according to the third exemplary embodiment has a similar configuration to the classification device 20a according to the second exemplary embodiment, except that, as described above, the first filter unit 13a further refers to the source domain classification loss in the training process, and the second filter unit 13b further refers to the target domain classification loss in the training process in the third exemplary embodiment.

[0184] Therefore, the classification device 20b according to the third exemplary embodiment provides a suitable classification process.

[0185] <Fourth embodiment> Hereinafter, the fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Elements having the same functions as those described in the above exemplary embodiment are given the same reference numerals, and their description will be omitted as appropriate. Furthermore, the outline of the fourth exemplary embodiment is the same as that of the above exemplary embodiment, so it will not be described here.

[0186] (Training Equipment Configuration) Hereinafter, the configuration of the training device 10c according to the fourth exemplary embodiment will be described with reference to Fig. 11. Fig. 11 is a block diagram showing the configuration of the training device 10c. As shown in Fig. 11, the training device 10c includes a domain discrimination unit 15 and a domain loss calculation unit 16 in addition to the components included in the training device 10a according to the second exemplary embodiment.

[0187] (Domain discrimination part) The domain discrimination unit 15 performs discrimination processing to discriminate between a target domain and a source domain. That is, the domain discrimination unit 15 performs domain prediction to indicate whether a feature is from the source domain or the target domain.

[0188] As an example, the domain discrimination unit 15 performs the following process. The domain discrimination unit 15 uses the source domain feature value X S and the target domain feature value X extracted by the second feature extraction unit 11b. T Then, the domain discrimination unit 15 receives the source domain feature value X without referring to information about which domain the feature belongs to. S Let X be the target domain feature value. T Then, the domain discrimination unit 15 outputs the result of the discrimination process.

[0189] As a concrete example, the feature value X S Given a feature value X S The result of the discrimination process is DP S can be printed as follows:

[0190]

number

[0191]

number

[0192] As an example, the domain loss calculation unit 16 may calculate (i) source domain label data D indicating the ground truth domain label of the source domain. S and (ii) target domain label data D indicating the ground truth domain label of the target domain. T Specifically, the source domain label data D S and target domain label data D T may be expressed as follows:

[0193]

number

number

[0194] As a concrete example, in (Eq,15), DP S is given, and in (Eq.17) D S is given, and in (Eq.16) DP T is given, and in (Eq.18) D T Given, DP S The two components of D S Unlike the corresponding components of DP T The four components of DT Since the components are different from the corresponding components of the domain loss, the domain loss calculation unit 16 calculates the domain loss (Loss_domain) as follows:

[0195]

number

[0196] As an example, the merge loss calculation unit 144 according to the fourth exemplary embodiment calculates the merge loss as follows.

[0197]

number

[0198] In addition, the model update unit 145 may update model parameters of the domain discrimination unit 15 in addition to the first feature extraction unit 11a, the second feature extraction unit 11b, the first class prediction unit 12a, and the second class prediction unit 12b, by referring to the merge loss (Loss_merge).

[0199] In addition, the model updating unit 145 may update the first feature extraction unit 11a, the second feature extraction unit 11b, the first class prediction unit 12a, and the second class prediction unit 12b by referring to the merge loss (Loss_merge), and may also update the domain discrimination unit 15 by referring to the domain loss (Loss_domain).

[0200] (Training using training devices) From a training perspective, the training device 10c performs the following processes: First, the training device 10c trains the domain discriminator 15 so that the domain discriminator 15 can discriminate whether a feature is from a source domain or a target domain. Second, the training device 10c trains the first feature extractor 11a and the second feature extractor 11b to extract features that may confuse the trained domain discriminator 15.

[0201] By repeating the above process, the training device 10c can achieve a situation where the well-trained domain discriminator 15 cannot distinguish whether a feature is from the source domain or the target domain, which means that the feature has very good domain confusion property, i.e., domain invariant property.

[0202] Fig. 12 is a diagram showing a process in which the first feature extractor 11a and the second feature extractor 11b realize domain invariance through learning by the training device 10c. As shown in Fig. 12, features that exist separately in the feature space at the early stage of training do not exist separately in the feature space at the later stage of training. In this way, domain invariance is achieved by the training performed by the training device 10c.

[0203] (Beneficial Effects) As described above, according to the fourth exemplary embodiment, the training device 10c performs the following steps: S and X TThis allows for the realization of domain invariance, which results in desirable target domain properties.

[0204] Thus, according to the fourth exemplary embodiment, the training apparatus 10c provides an efficient and stable training process even when only a small amount of target domain label data is available.

[0205] (Training method using training equipment) A training method for the training device 10c according to the fourth exemplary embodiment will be described below with reference to Fig. 13. Fig. 13 is a flow chart showing the flow of a training method S1c for the training device 10c.

[0206] 13, the training method S1c according to the fourth exemplary embodiment includes steps S15 and S16 in addition to the steps included in the training method S1a according to the second exemplary embodiment. Hereinafter, only the differences between the training method S1c according to the fourth exemplary embodiment and the training method S1a according to the second exemplary embodiment will be described.

[0207] (Step S15) In step S15, the domain discrimination unit 15 performs a domain discrimination process to discriminate the target domain from the source domain. The specific process performed by the domain discrimination unit 15 has been described above, and will not be repeated here.

[0208] (Step S16) In step S16, the domain loss calculation unit 16 calculates and outputs the domain discrimination loss by referring to the result of the discrimination process by the domain discrimination unit 15. The specific process performed by the domain loss calculation unit 16 has been described above, and therefore will not be described repeatedly here.

[0209] (Step S144) In step S144, the merge loss calculation unit 144 according to the fourth exemplary embodiment calculates the merge loss (Loss_merge) by referring to (i) the source domain classification loss (Loss_classification_S), (ii) the target domain classification loss (Loss_classification_T), (iii) the group loss (Loss_grouping), and (iv) the domain loss (Loss_domain). Note that the specific processing performed by the merge loss calculation unit 144 has been described above, and will not be described again here.

[0210] Note that the order of steps in the training method S1c does not limit the fourth exemplary embodiment. For example, in the training method S1c, steps S15 and S16 may be performed before at least any of steps S13a, S13b, S141, and S142.

[0211] (Beneficial Effects) According to the fourth exemplary embodiment, the training method S1c provides an efficient and stable training process even when only a small amount of target domain labeled data is available.

[0212] (Classification device configuration) The configuration of the classification device according to the fourth exemplary embodiment will be described below. The classification device 20c according to the fourth exemplary embodiment has a similar configuration to the classification device 20a shown in FIG.

[0213] According to a fourth exemplary embodiment, the training device 10c performs a training on the extracted features X S and X T achieve domain invariance, which results in favorable target domain properties.

[0214] Therefore, the classification device 20c according to the fourth exemplary embodiment provides suitable classification processing even when the target domain labeled data is small.

[0215] (Points to note regarding the fourth embodiment) To compute the domain loss, class labels are not required. Thus, instead of using only labeled target domain data, the training device 10c can also use unlabeled (in the sense of class labels) target data. As long as the data is from the target dataset, the training device 10c can know that the domain label of the data is "target".

[0216] The domain-invariant features described above can be made more general by including unlabeled (in the sense of class labels) target data: features may not only be domain-invariant to all pieces of label data, but also domain-invariant to all pieces of unlabeled target data.

[0217] The above configuration, which uses unlabeled (in the sense of class labels) target data, can achieve the domain invariant property adequately in situations where a limited amount of class-labeled target data is available.

[0218] <Fifth embodiment> Hereinafter, the fifth exemplary embodiment of the present invention will be described in detail with reference to the drawings. The elements having the same functions as those described in the above exemplary embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate. Furthermore, the outline of the fifth exemplary embodiment is the same as that of the above-mentioned embodiment, so it will not be described here.

[0219] (Training Equipment Configuration) Next, the configuration of a training device 10d according to a fifth exemplary embodiment will be described with reference to Fig. 14. Fig. 14 is a block diagram showing the configuration of the training device 10d. As shown in Fig. 14, the training device 10d according to the fifth exemplary embodiment differs from the training device 10c according to the fourth exemplary embodiment in that a source domain classification loss (Loss_classification_S) is input to the first filter unit 13a, and a target domain classification loss (Loss_classification_T) is input to the second filter unit 13b.

[0220] That is, the training device 10d according to the fifth exemplary embodiment includes both the configuration described in the third exemplary embodiment and the configuration described in the fourth exemplary embodiment.

[0221] Therefore, the training apparatus 10d according to the fifth exemplary embodiment provides both the effects of the configuration of the third exemplary embodiment and the effects of the configuration of the fourth exemplary embodiment.

[0222] (Classification device configuration) The classification device 20d according to the fifth exemplary embodiment has a configuration similar to that of the classification device 20a shown in Fig. 9. The training device 10d according to the fifth exemplary embodiment includes both the configuration described in the third exemplary embodiment and the configuration described in the fourth exemplary embodiment. In addition, the classification device 20d according to the fifth exemplary embodiment includes a feature extraction unit 11b and a class prediction unit 12b trained by the training device 10d.

[0223] Therefore, the classification device 20d according to the fifth exemplary embodiment provides both the effects of the classification device 20b according to the third exemplary embodiment and the effects of the classification device 20c according to the fourth exemplary embodiment.

[0224] <Example of configuration realized by software> Some or all of the functions of the training devices 10, 10a, 10b, 10c, and 10d and the classification devices 20, 20a, 20b, 20c, and 20d may be realized by hardware such as an IC chip (integrated circuit), or by software.

[0225] In the latter case, the training devices 10, 10a, 10b, 10c, 10d and the classification devices 20, 20a, 20b, 20c, 20d are realized by a computer that executes instructions of a program, which is software that realizes the above-mentioned functions. FIG. 15 shows an example of such a computer (hereinafter, referred to as "computer C"). The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to function as any one of the training devices 10, 10a, 10b, 10c, 10d and the classification devices 20, 20a, 20b, 20c, 20d. In the computer C, the processor C1 reads out and executes the program P from the memory C2, thereby realizing the functions of the training devices 10, 10a, 10b, 10c, 10d and the classification devices 20, 20a, 20b, 20c, 20d.

[0226] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), or a physics processing unit (PPU). As an example, the memory C2 may be a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.

[0227] The computer C may further include a RAM (random access memory) into which the program P is loaded when the program P is executed and in which various data is temporarily stored. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0228] The program P can be stored in a non-transitory tangible storage medium M readable by the computer C. Examples of the storage medium M include a tape, a disk, a card, a semiconductor memory, and a programmable logic circuit. The computer C can acquire the program P via the storage medium M. The program P may be transmitted via a transmission medium. Examples of the transmission medium include a communication network and a broadcast wave. The computer C can also acquire the program P via such a transmission medium.

[0229] [Supplementary Note 1] The present invention is not limited to the above-described exemplary embodiments, and may be modified in various ways by those skilled in the art within the scope of the claims. For example, exemplary embodiments obtained by appropriately combining the technical means disclosed in the above exemplary embodiments are also included in the technical scope of the present invention.

[0230] [Supplementary Note 2] All or part of the above exemplary embodiments can be expressed as follows: However, the present invention is not limited to the following exemplary aspects.

[0231] [supplement] Aspects of the present invention can also be expressed as follows: (Aspect 1) one or more feature extraction means for extracting source domain feature values ​​from the source domain data and for extracting target domain feature values ​​from the target domain data; one or more class prediction means for predicting a source domain class prediction value from a source domain feature value and a target domain class prediction value from a target domain feature value; filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predictor; Filtering out one or more values ​​from the target domain feature values ​​with reference to the target domain class predictor. one or more filtering means for calculating filtered source domain feature values ​​and filtered target domain feature values ​​by The source domain classification loss, calculated with reference to the source domain class predictions, the target domain classification loss, calculated with reference to the target domain class predictions; and Group loss calculated with reference to the filtered source domain feature values ​​and the filtered target domain feature values updating means for updating at least one of said one or more feature extraction means and said one or more class prediction means with reference to A training device comprising:

[0232] The above configuration provides an efficient and stable training process even when only a small amount of target domain labeled data is available. (Aspect 2) The one or more filtering means filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predictions and source domain class label data; 2. The training apparatus of aspect 1, further comprising: filtering out one or more values ​​from the target domain feature values ​​with reference to the target domain class prediction values ​​and the target domain class label data.

[0233] According to the above configuration, it is possible to provide a more efficient and stable training process.

[0234] (Aspect 3) 3. The training apparatus of claim 1 or 2, wherein the one or more filtering means further reference the source domain classification loss and the target domain classification loss.

[0235] According to the above configuration, it is possible to provide a more efficient and stable training process.

[0236] (Aspect 4) The updating means a grouping means for outputting class groups from the filtered source domain feature values ​​and the filtered target domain feature values; 4. The training apparatus of any one of aspects 1 to 3, wherein each of the class groups includes feature values ​​that share the same class label.

[0237] According to the above configuration, it is possible to provide a more efficient and stable training process.

[0238] (Aspect 5) 5. The training apparatus according to aspect 4, wherein the update means further comprises a group loss calculation means for calculating the group loss by referring to the class group.

[0239] According to the above configuration, it is possible to provide a more efficient and stable training process.

[0240] (Aspect 6) The group loss calculation means The intra-class distance is determined by referring to the features in the same class; The inter-class distance is determined by referring to the features in different classes. 6. The training apparatus according to claim 5, wherein the group loss is calculated based on:

[0241] According to the above configuration, it is possible to provide a more efficient and stable training process.

[0242] (Aspect 7) The update means includes: Calculating the source domain classification loss with reference to the source domain class prediction value and source domain class label data; Calculate the target domain classification loss by referring to the target domain class predictions and the target domain class label data. 7. The training apparatus of any one of aspects 1-6, further comprising one or more classification loss computation means.

[0243] According to the above configuration, it is possible to provide a more efficient and stable training process.

[0244] (Aspect 8) The update means includes: a merge loss calculation means for calculating a merge loss by referring to (i) the source domain classification loss, (ii) the target domain classification loss, and (iii) the group loss; The update means includes: The training apparatus of any one of aspects 1 to 7, further comprising: updating at least one of the one or more feature extraction means and the one or more class prediction means with reference to the merge loss.

[0245] According to the above configuration, it is possible to provide a more efficient and stable training process.

[0246] (Aspect 9) one or more domain discrimination means for executing a discrimination process for discriminating a target domain from a source domain; one or more domain loss calculation means for outputting a domain discrimination loss as a result of the discrimination process; Further comprising: The updating means further refers to a domain discrimination loss, The updating means further updates the domain discrimination means. 9. The training device of any one of aspects 1 to 8.

[0247] According to the above configuration, it is possible to provide a more efficient and stable training process.

[0248] (Aspect 10) A feature extraction means for extracting target domain feature values ​​from the target domain data; A class prediction means for predicting a target domain class prediction value from the target domain feature value; Equipped with At least one of the feature extraction means and the class prediction means filtered source domain feature values ​​obtained by filtering out one or more values ​​from the source domain feature values; filtered target domain feature values ​​obtained by filtering out one or more values ​​from the target domain feature values; have been trained with reference to Classification device.

[0249] According to the above configuration, even when only a small amount of target domain labeled data is available, it is possible to provide a suitable classification process.

[0250] (Aspect 11) extracting source domain feature values ​​from the source domain data and target domain feature values ​​from the target domain data by one or more feature extraction means; predicting, by one or more class prediction means, a source domain class prediction value from the source domain feature values ​​and a target domain class prediction value from the target domain feature values; filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predictor; Filter out one or more values ​​from the target domain feature values ​​by looking up the target domain class predictions calculating filtered source domain feature values ​​and filtered target domain feature values ​​by The source domain classification loss, calculated with reference to the source domain class predictions, the target domain classification loss, calculated with reference to the target domain class predictions; and Group loss calculated with reference to the filtered source domain feature values ​​and the filtered target domain feature values updating at least one of the one or more feature extraction means and the one or more class prediction means with reference to A training method including:

[0251] The method provides an efficient and stable training process even when only a small amount of target domain labeled data is available.

[0252] (Aspect 12) extracting target domain feature values ​​from the target domain data by a feature extraction means; predicting a target domain class prediction value from the target domain feature value by a class prediction means; Including, At least one of the feature extraction means and the class prediction means filtered source domain feature values ​​obtained by filtering out one or more values ​​from the source domain feature values; filtered target domain feature values ​​obtained by filtering out one or more values ​​from the target domain feature values; have been trained with reference to Classification method.

[0253] The method provides an efficient and stable training process even when only a small amount of target domain labeled data is available.

[0254] (Aspect 13) A program for causing a computer to function as the training device described in aspect 1, characterized in that the program causes the computer to function as the feature extraction means, the class prediction means, the filtering means, and the update means.

[0255] (Aspect 14) A program for causing a computer to function as the classification device according to aspect 10, the program causing a computer to function as the feature extraction means and the class prediction means.

[0256] (Aspect 15) A training device having at least one processor, The processor, extracting source domain feature values ​​from the source domain data and target domain feature values ​​from the target domain data by one or more feature extraction means; predicting, by one or more class prediction means, a source domain class prediction value from the source domain feature values ​​and a target domain class prediction value from the target domain feature values; filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predictor; Filter out one or more values ​​from the target domain feature values ​​by looking up the target domain class predictions calculating filtered source domain feature values ​​and filtered target domain feature values ​​by The source domain classification loss, calculated with reference to the source domain class predictions, the target domain classification loss, calculated with reference to the target domain class predictions; and Group loss calculated with reference to the filtered source domain feature values ​​and the filtered target domain feature values updating at least one of the one or more feature extraction means and the one or more class prediction means with reference to A training device that performs the following:

[0257] The training device may further include a memory. The memory may store a program for causing the processor to execute the process. Alternatively, the program may be stored in a computer-readable non-transitory tangible storage medium.

[0258] (Aspect 16) A classification device comprising at least one processor, The processor, extracting target domain feature values ​​from the target domain data by a feature extraction means; predicting a target domain class prediction value from the target domain feature value by a class prediction means; Run At least one of the feature extraction means and the class prediction means filtered source domain feature values ​​obtained by filtering out one or more values ​​from the source domain feature values; filtered target domain feature values ​​obtained by filtering out one or more values ​​from the target domain feature values; have been trained with reference to Classification device.

[0259] The classification device may further include a memory. The memory may store a program for causing the processor to execute the process. Alternatively, the program may be stored in a computer-readable non-transitory tangible storage medium. [Explanation of symbols]

[0260] 10, 10a, 10b, 10c, 10d training equipment 20, 20a, 20b, 20c, 20d classification device 11, 11a, 11b Feature Extraction Section 12, 12a, 12b Class Prediction Section 13, 13a, 13b Filter section 14, 14a update section 141 Grouping Section 142 Group Loss Calculation Department 143a, 143b Classification loss calculation section 144 Merge Loss Calculation 145 Model Update Department 15 Domain Discrimination Part 16 Domain Loss Calculation Section

Claims

1. one or more feature extraction means for extracting source domain feature values ​​from the source domain data and for extracting target domain feature values ​​from the target domain data; one or more class prediction means for predicting a source domain class prediction value from a source domain feature value and a target domain class prediction value from a target domain feature value; filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predictor; one or more filtering means for calculating filtered source domain feature values ​​and filtered target domain feature values ​​by filtering out one or more values ​​from the target domain feature values ​​with reference to the target domain class predictions; The source domain classification loss, calculated with reference to the source domain class predictions, the target domain classification loss, calculated with reference to the target domain class predictions; and updating means for updating at least one of the one or more feature extraction means and the one or more class prediction means with reference to a group loss calculated with reference to the filtered source domain feature values ​​and the filtered target domain feature values; A training device comprising:

2. The one or more filtering means include: filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predictions and source domain class label data; The training apparatus of claim 1 , further comprising: a training device configured to filter out one or more values ​​from the target domain feature values ​​by referring to the target domain class prediction values ​​and the target domain class label data.

3. The training apparatus of claim 1 or 2, wherein the one or more filtering means further refer to the source domain classification loss and the target domain classification loss.

4. The update means a grouping means for outputting class groups from the filtered source domain feature values ​​and the filtered target domain feature values; A training apparatus according to claim 1 , wherein each of the class groups comprises feature values ​​that share the same class label.

5. The training apparatus according to claim 4 , wherein the update means further comprises a group loss calculation means for calculating the group loss by referring to the class groups.

6. extracting source domain feature values ​​from the source domain data and extracting target domain feature values ​​from the target domain data by one or more feature extraction means; predicting, by one or more class prediction means, a source domain class prediction value from the source domain feature values ​​and a target domain class prediction value from the target domain feature values; filtering out one or more values ​​from the source domain feature values ​​with reference to the source domain class predictor; calculating filtered source domain feature values ​​and filtered target domain feature values ​​by filtering out one or more values ​​from the target domain feature values ​​with reference to the target domain class prediction values; The source domain classification loss, calculated with reference to the source domain class predictions, the target domain classification loss, calculated with reference to the target domain class predictions; and updating at least one of the one or more feature extractors and the one or more class predictors with reference to a group loss calculated with reference to the filtered source domain feature values ​​and the filtered target domain feature values; A training method including:

7. 2. A program for causing a computer to function as the training device according to claim 1, the program causing the computer to function as the feature extraction means, the class prediction means, the filtering means, and the update means.

Images