Learning method, information processing device, and learning program

Abstract

To provide a learning method for configuring a machine learning model that extracts the feature values of the image data and classifies image data classes based on the feature values and is capable of effectively solving the new problem of OSDG.SOLUTION: The learning method includes; a step of acquiring the feature values of multiple pieces of learning data using the learning with the correct label of the class as the input; a step of calculating a loss function based on the feature values; and a step of updating the parameters of the machine learning model so that the loss function gets smaller. Here, the class is constituted of multiple known classes that indicate that the image data is of a particular type and an unknown class that indicates that the image data does not belong to any type. The loss function is configured to include a first loss function and a second loss function.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a learning method, an information processing device, and a learning program, and in particular to a technique for extracting features of image data and classifying the image data into classes based on the features. [Background technology]

[0002] In machine learning, it is known that performance degrades when the domain of the test data is different from the domain of the training data. DG (Domain Generalization) is a technique for maintaining performance even when the domain of the test data (target domain) is different from the domain of the training data (source domain). Many applications that use machine learning are required to achieve good performance in a new environment different from the training data, and therefore DG has been attracting particular attention in recent years. For example, Patent Document 1 can be cited as a conventional technique related to machine learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-091443 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, when applying DG to the training of a device that performs class classification, it is required that all of the classes into which data in the target domain is classified appear as the source domain. However, in practical applications (e.g., when applied to self-driving cars), it is also necessary to distinguish that none of the classes appear during training.

[0005] Therefore, the inventor of the present disclosure defined the problem of implementing DG as OSDG (Open Set Domain Generalization), assuming that known classes indicating a specific type (a type of interest) and unknown classes indicating that they do not belong to any type indicated by the known classes (a type of no interest) exist in both the source domain and the target domain. In order to solve OSDG, it is necessary to simultaneously implement separation of known classes and unknown classes and DG.

[0006] Traditional DG is typically performed by smoothing the data distribution across multiple source domains in feature space. However, due to the presence of unknown classes in OSDG, traditional DG cannot be effectively solved by simply applying it to OSDG.

[0007] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a learning method capable of effectively solving OSDG, an information processing device, and a learning program that causes a computer to execute the learning method, in relation to a technology for extracting features of image data and classifying the image data into classes based on the features. [Means for solving the problem]

[0008] The first disclosure relates to a method for learning a machine learning model that extracts features of image data for classifying the image data into classes. The learning method according to the first disclosure includes the steps of: inputting training data to which a correct label of the class has been assigned, acquiring the features of a plurality of the training data; calculating a loss function based on the features; and updating parameters of the machine learning model so as to reduce the loss function. Here, the classes are made up of a plurality of known classes which indicate that the image data is of a specific type, and an unknown class which indicates that the image data does not belong to any of the types. The loss function includes, as terms, a first loss function that gives, for each of a plurality of first anchor data selected from the training data whose correct label is the unknown class, a distance between the feature of the first anchor data and the feature of the appropriately selected training data, and takes a larger value as the distance is smaller than a predetermined margin; and a second loss function that gives, for each of a plurality of second anchor data selected from the training data whose correct label is the known class, the distance between the feature of the second anchor data and the feature of the appropriately selected training data, and takes a larger value as the distance for the training data whose correct label is the same as the second anchor data is larger and the distance for the training data whose correct label is different from the second anchor data is smaller.

[0009] The second disclosure relates to a learning method that further has the following characteristics in addition to the learning method according to the first disclosure. The first anchor data is xa, the training data selected for the first anchor data is xn, the number of samples of the training data xn is K, the correct labels given to the first anchor data xa and the training data xn, respectively, are ya and yn, a set of the multiple known classes is C, the unknown class is u, the margin is α, the function giving the distance is d, and the feature amounts of the anchor data xa and the training data xn, respectively, are f(xa) and f(xn). The first loss function is expressed as Ld shown in the following formula (1).

number

[0010] The third disclosure relates to a learning method further having the following characteristics in addition to the learning method according to the first or second disclosure. The second loss function is a triplet loss function that uses the second anchor data as an anchor and is configured with the same margin as the margin of the first loss function.

[0011] The fourth disclosure relates to a learning method for a machine learning model that extracts features of image data and classifies the image data into classes based on the features. The learning method according to the fourth disclosure includes the steps of: inputting training data to which a correct label of the class has been assigned, and acquiring outputs for a plurality of the training data and the features of the plurality of training data; calculating a loss function based on the outputs and the features; and updating parameters of the machine learning model so as to reduce the loss function. Here, the classes are made up of a plurality of known classes which indicate that the image data is of a specific type, and an unknown class which indicates that the image data does not belong to any of the types. The loss function includes, as terms, a main loss function which assumes a smaller value as the output coincides with the correct label; a first loss function which, for each of a plurality of first anchor data selected from the training data whose correct label is the unknown class, gives a distance between the feature amount of the first anchor data and the feature amount of the appropriately selected training data, and assumes a larger value as the distance is smaller than a predetermined margin; and a second loss function which, for each of a plurality of second anchor data selected from the training data whose correct label is the known class, gives the distance between the feature amount of the second anchor data and the feature amount of the appropriately selected training data, and assumes a larger value as the distance for the training data whose correct label is the same as the second anchor data is larger and the distance for the training data whose correct label is different from the second anchor data is smaller.

[0012] The fifth disclosure relates to a learning method that further has the following characteristics in comparison with the learning method according to the fourth disclosure. The first anchor data is xa, the training data selected for the first anchor data is xn, the number of samples of the training data xn is K, the correct labels given to the first anchor data xa and the training data xn, respectively, are ya and yn, a set of the multiple known classes is C, the unknown class is u, the margin is α, the function giving the distance is d, and the feature amounts of the anchor data xa and the training data xn, respectively, are f(xa) and f(xn). The first loss function is expressed as Ld shown in the following formula (1).

number

[0013] The sixth disclosure relates to a learning method further having the following characteristics with respect to the learning method according to the fourth or fifth disclosure. The second loss function is a triplet loss function that uses the second anchor data as an anchor and is configured with the same margin as the margin of the first loss function.

[0014] The seventh disclosure relates to a learning method according to any one of the fourth to sixth disclosures, further having the following characteristics. The primary loss function includes a loss function that trains the machine learning model to be able to classify classes of the image data regardless of the domain of the image data.

[0015] The eighth disclosure relates to an information processing device having a feature amount extraction processing unit that extracts feature amounts of image data, and a class classification processing unit that classifies the image data into classes based on the feature amounts. The classes are composed of a plurality of known classes indicating that the image data is of a specific type, and an unknown class indicating that the image data does not belong to any of the types. The feature extraction processing unit and the class classification processing unit are composed of a machine learning model. The machine learning model is trained to reduce a loss function using a plurality of training data to which correct labels of the classes are assigned. Here, the loss function includes, as terms, a main loss function which assumes a smaller value as the output coincides with the correct label; a first loss function which, for each of a plurality of first anchor data selected from the training data whose correct label is the unknown class, gives a distance between the feature amount of the first anchor data and the feature amount of the appropriately selected training data, and assumes a larger value as the distance is smaller than a predetermined margin; and a second loss function which, for each of a plurality of second anchor data selected from the training data whose correct label is the known class, gives the distance between the feature amount of the second anchor data and the feature amount of the appropriately selected training data, and assumes a larger value as the distance for the training data whose correct label is the same as the second anchor data is larger and the distance for the training data whose correct label is different from the second anchor data is smaller.

[0016] The ninth disclosure relates to an information processing device further having the following features with respect to the information processing device according to the eighth disclosure. The first anchor data is xa, the training data selected for the first anchor data is xn, the number of samples of the training data xn is K, the correct labels given to the first anchor data xa and the training data xn, respectively, are ya and yn, a set of the multiple known classes is C, the unknown class is u, the margin is α, the function giving the distance is d, and the feature amounts of the anchor data xa and the training data xn, respectively, are f(xa) and f(xn). The first loss function is expressed as Ld shown in the following formula (1).

number

[0017] A tenth disclosure relates to an information processing device according to the eighth or ninth disclosure, further comprising the following features. The second loss function is a triplet loss function that uses the second anchor data as an anchor and is configured with the same margin as the margin of the first loss function.

[0018] An eleventh disclosure relates to an information processing device according to any one of the eighth to tenth disclosures, further having the following features. The primary loss function includes a loss function that trains the machine learning model to be able to classify classes of the image data regardless of the domain of the image data.

[0019] A twelfth disclosure is a learning program for causing a computer to execute the learning method according to any one of the first to seventh disclosures. Effect of the Invention

[0020] According to the learning method, information processing device, and learning program for causing a computer to execute the learning method according to the present disclosure, the loss function includes a first loss function and a second loss function as terms. This makes it possible to configure a feature space that can be divided so as to remove image data of an unknown class. In turn, it is possible to configure a machine learning model capable of effectively solving OSDG, and to provide an information processing device capable of effectively solving OSDG. [Brief description of the drawings]

[0021] [Figure 1] FIG. 13 is a conceptual diagram for explaining OSDG when the target data is image data. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a machine learning model that is trained by a training method according to the present embodiment. [Diagram 3] 4 is a flowchart showing a learning method according to the present embodiment. [Figure 4] FIG. 1 is a conceptual diagram expressing a feature space achieved by a first loss function. [Diagram 5] FIG. 13 is a conceptual diagram of a feature space when learning is performed to reduce a first loss function. [Figure 6] FIG. 13 is a conceptual diagram of a feature space when learning is performed to reduce a second loss function. [Figure 7] 1 is a table showing examples of a machine learning model trained by the learning method according to the present embodiment. [Figure 8] FIG. 13 is a diagram showing an example of image data of a VLCS. [Figure 9] FIG. 1 is a diagram showing an example of image data of DomainNet. [Figure 10] 1 is a block diagram showing an example of the configuration of an information processing device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, the embodiments of the present disclosure will be described with reference to the drawings. However, when the number, quantity, amount, range, etc. of each element is mentioned in the embodiments shown below, the idea of ​​the present disclosure is not limited to the mentioned number unless it is specifically stated or the number is clearly specified in principle. Furthermore, the configurations etc. described in the embodiments shown below are not necessarily essential to the idea of ​​the present disclosure unless it is specifically stated or the number is clearly specified in principle. In addition, the same reference numerals are attached to the same or corresponding parts in each drawing, and the duplicated explanation is appropriately simplified or omitted.

[0023] 1.OSDG(Open Set Domain Generalization) The learning method according to the present embodiment is a learning method for a machine learning model that extracts features of image data and classifies the image data into classes based on the features. In particular, the learning method relates to a machine learning model that aims to solve OSDG. Below, we will explain OSDG when the target data is image data.

[0024] OSDG is a problem of classifying data into classes, independent of the domain of the target data, and is composed of known classes, which indicate that the classes are specific classes (classes of interest), and unknown classes, which indicate that the classes do not belong to any of the known classes (classes of no interest).

[0025] FIG. 1 is a conceptual diagram for explaining OSDG when the target data is image data. In FIG. 1, classes indicate the types of objects that appear in the image data. The classes are composed of known classes of "dog", "horse", and "person", and an unknown class. Here, the unknown class indicates that the type of object that appears in the image data is not one of "dog", "horse", or "person". FIG. 1 also shows multiple image data (photos, pictures, sketches, etc.) from different domains.

[0026] In other words, in Fig. 1, OSDG is a problem of classifying objects in image data as "dogs," "horses," and "people" without depending on the domain, while removing image data that contains other objects. For example, there is a case where you want to recognize (are interested in) that "dogs," "horses," and "people" are included in image data, but are not interested in other objects that are included. Such a case is raised as a practical problem when trying to perform object recognition from image data.

[0027] When constructing a machine learning model to solve the OSDG shown in Figure 1, multiple image data from different domains with correct labels are used as training data, as shown in Figure 1. What is different from conventional DGs here is that the classes to be classified include unknown classes, and the training data includes image data whose correct labels are unknown classes. In addition, the target data also includes image data of unknown classes.

[0028] In addition, OSDG, like conventional DG, also includes the use of image data from domains that do not exist in the training data as target data. For example, in Figure 1, image data of a photograph (first row, photo), a painting (second row, art), and a sketch (third row, sketch) are given as training data, while the target data is image data of cartoon-style or deformed objects. Hereinafter, the domain of the training data is also referred to as the "source domain" and the domain of the target data is also referred to as the "target domain."

[0029] In general, when classifying image data, features of the image data are extracted, and the image data is classified into classes based on the positions of the extracted features in a feature space. Conventional DGs typically operate to arrange the feature space by using training data across multiple source domains. On the other hand, simply applying a conventional DG to an OSDG does not guarantee that the feature space is arranged appropriately for unknown classes in the target domain. This is because the conventional DG does not require unknown classes in the source domain and the target domain. For this reason, in the conventional DG, there is a risk that image data in the target domain that should be classified into an unknown class will be mapped to a position in the feature space corresponding to a known class.

[0030] To solve OSDG, at least two mechanisms are required. One (the first mechanism) is to construct a feature space that can be divided so as to remove image data of unknown classes. The other (the second mechanism) is to perform distribution matching to align the positions in the feature space of known classes across multiple domains. In conventional DGs, the latter mechanism makes it possible to perform domain-independent class classification, but as mentioned above, the former mechanism is insufficient.

[0031] Therefore, the learning method according to the present embodiment provides the first mechanism. For the second mechanism, a conventional DG can be adopted. This allows the construction of a machine learning model that can effectively solve the OSDG.

[0032] 2. Machine learning models The machine learning model that is trained by the learning method according to this embodiment will be described below. Fig. 2 is a block diagram showing an example of the configuration of the machine learning model 10 that is trained by the learning method according to this embodiment.

[0033] The machine learning model 10 includes a feature extraction processing unit 11 and a class classification processing unit 12. The feature extraction processing unit 11 receives image data as input and outputs features. The features output by the feature extraction processing unit 11 are transmitted to the class classification processing unit 12. The class classification processing unit 12 receives the features as input and outputs classes. In other words, the machine learning model 10 performs class classification of image data.

[0034] The machine learning model 10 has parameters that define the processing of each of the feature amount extraction processing unit 11 and the class classification processing unit 12. That is, the machine learning model 10 learns by updating the parameters.

[0035] The machine learning model 10 is typically configured with a convolutional neural network. In this case, the feature extraction processing unit 11 is configured with a convolutional layer and a pooling layer, and the classification processing unit 12 is configured with a fully connected layer. The parameters are the filters of the convolutional layer and the weights of the fully connected layer.

[0036] However, the machine learning model 10 may be configured by other means. For example, the feature extraction processing unit 11 may be configured by a convolutional neural network, and the class classification processing unit 12 may be configured by an SVM or a k-NN method.

[0037] The machine learning model 10 is typically provided by a program, and the processing of the machine learning model 10 is realized by a processor. In this case, the parameters of the machine learning model 10 may be provided as part of the program, or may be stored in a memory and read by the processor. Also, the parameters may be updated by updating the program or by updating the memory.

[0038] 3. Learning Methods In the learning method according to this embodiment, a loss function is calculated, and the parameters of the machine learning model 10 are updated so as to reduce the loss function. Hereinafter, the learning method according to this embodiment will be described.

[0039] FIG. 3 is a flowchart showing the learning method according to this embodiment.

[0040] In step S100, learning data is input to the machine learning model 10, and outputs for a plurality of learning data and feature quantities of the plurality of learning data are obtained. After step S100, the process proceeds to step S110.

[0041] In step S110, a loss function is calculated based on the output and feature amount acquired in step S100. Here, the learning method according to this embodiment is characterized by the loss function calculated to provide the first mechanism. Details of the calculated loss function will be described later. After step S110, the process proceeds to step S120.

[0042] In step S120, the gradient of the loss function acquired in step S110 is calculated. A suitable known technique may be adopted as the method for calculating the gradient of the loss function. For example, when the machine learning model 10 is configured by a convolutional neural network, the method for calculating the gradient of the loss function is typically the backpropagation method. After step S120, the process proceeds to step S130.

[0043] In step S130, the parameters of the machine learning model 10 are updated based on the gradient calculated in step S120 so as to reduce the loss function. That is, the parameters are updated by the gradient descent method. Here, the hyperparameters related to the parameter update may be suitably determined according to the environment in which the learning method according to the present embodiment is applied. For example, the parameters may be updated by the momentum method.

[0044] In step S140, it is determined whether or not a learning termination condition is satisfied. The termination condition may be, for example, that the number of repeated parameter updates is equal to or greater than a predetermined value, or that the loss function is equal to or less than a predetermined value.

[0045] If the learning end condition is satisfied (step S140; Yes), the learning is terminated. If the learning end condition is not satisfied (step S140; No), the process returns to step S100 and the learning is repeated.

[0046] The learning method shown in FIG. 3 is realized as a program (learning program).

[0047] 4. Loss Function In order to provide the first mechanism, the inventors of the present disclosure have come up with the idea of ​​constructing a feature space by metric learning. Typical loss functions related to metric learning (triplet loss function and contrastive loss function) are configured to make the same classes closer to each other in the feature space and different classes farther apart in the feature space. This is expected to construct a feature space that can be easily divided into classes.

[0048] However, it is not clear how to handle unknown classes in loss functions related to distance learning. What is clear is that when a triplet loss function is adopted, training data of unknown classes will not form positive pairs with training data of known classes. However, it is not easy to determine whether it is acceptable to form positive pairs with training data of unknown classes.

[0049] A simple approach is to use the training data of the unknown class only to form negative pairs in the triplet loss function. However, the inventors of the present disclosure have found through an ablation study that this approach alone is insufficient for OSDG because the features of the training data of the unknown class are not clearly separated in the feature space.

[0050] Therefore, in addition to the above approach, the inventor of the present disclosure came up with the idea of ​​introducing a first loss function for constructing a feature space that maintains a distance from the unknown class in the feature space. FIG. 4 is a conceptual diagram expressing a feature space achieved by the first loss function. Each figure shown in FIG. 4 indicates a feature in the feature space. Here, the same figures indicate the same class. Also, the dotted line shown in FIG. 4 indicates an example of a desirable classification boundary for OSDG.

[0051] As shown in Figure 4, distance learning can construct a feature space in which identical known classes are close to each other. However, if the first loss function is not introduced, the known classes and unknown classes are close to each other, making it difficult to separate them. On the other hand, by introducing the first loss function, the unknown classes are kept sufficiently distant from other classes, making it possible to separate the unknown classes more clearly.

[0052] The loss function calculated in the learning method according to this embodiment will be described in detail below.

[0053] The loss function calculated in the learning method according to this embodiment includes a first loss function and a second loss function as terms.

[0054] First, the first loss function will be described. The first loss function is expressed by Ld shown in the following formula (1).

[0055]

number

[0056] Here, the training data is represented by x, the correct label given to the training data x by y, the set of known classes by C, the unknown class by u, the feature of the training data x by f(x), and the function giving the distance in the feature space by d.

[0057] In other words, N is a set of combinations of training data (hereinafter also referred to as "first anchor data") xa whose correct label ya is unknown class u, and training data xn selected with the number of samples K for the first anchor data xa. However, the selected training data xn must be closer to the first anchor data xa than a predetermined margin α. This is a condition that prevents the selection of training data that has already achieved the purpose of the first loss function and does not contribute to changes in the first loss function. This makes it possible to make the learning process more efficient.

[0058] Here, i is a number for distinguishing each of the selected learning data. The margin α specifies how much distance should be maintained between the unknown class and the feature space. The margin α may be appropriately set according to the environment in which the learning method according to this embodiment is applied.

[0059] In the configuration of N, the first anchor data xa are selected from the learning data whose correct label is an unknown class, or all of them are selected, and for each of the first anchor data xa, learning data xn is selected with the number of samples K. In addition, in formula (1), |N| represents the number of elements of N.

[0060] In formula (1), the function d that gives the distance may be a suitable function depending on the environment in which the learning method according to the present embodiment is applied. For example, cosine similarity is exemplified as d.

[0061] As shown in formula (1), the first loss function Ld becomes larger as the distance between the feature value f(xa) of the first anchor data xa and the feature value f(xn) of the training data xn for each first anchor data xa becomes smaller than the margin α. In other words, by performing learning (updating the parameters of the machine learning model 10) so as to reduce the first loss function Ld, a feature space that maintains a distance from the unknown class in the feature space is constructed. FIG. 5 shows a conceptual diagram of the feature space when learning is performed so as to reduce the first loss function Ld.

[0062] Next, the second loss function will be described. The second loss function is expressed by Lt shown in the following formula (2).

[0063]

number

[0064] As shown in equation (2), the second loss function Lt is a triplet loss function that uses training data whose correct label is a known class (hereinafter also referred to as “second anchor data”) as an anchor and is composed of the same margin as the margin α related to the first loss function.

[0065] In the triplet set T, the positive data xp is training data of the same class as the second anchor data xa, and may be one or more training data selected randomly. This makes it possible to improve the efficiency of the training process without selecting all training data of the same class as the second anchor data xa as the positive data xp.

[0066] In addition, for each second anchor data xa, negative data xn is selected with the number of samples K. In the above triplet set T, a condition for semi-hardly selecting negative data xn is given, but a suitable condition may be adopted (for example, a condition for hard selection may be given). In addition, the number of samples K does not have to be the same as the first loss function.

[0067] The second loss function (triplet loss function) Lt becomes larger as the distance between the feature value f(xa) of the second anchor data xa and the feature value f(xp) of the positive data xp becomes larger and the distance between the feature value f(xa) of the second anchor data xa and the feature value f(xn) of the negative data xn becomes smaller. In addition, the second anchor data xa is characterized in that learning data whose correct answer label is a known class is selected.

[0068] That is, by performing learning (updating the parameters of the machine learning model 10) so as to reduce the second loss function Lt, the following inequality (3) is satisfied. Then, for known classes, the feature space is configured so that the same classes are closer to each other in the feature space, and different classes are farther away in the feature space. FIG. 6 shows a conceptual diagram of the feature space when learning is performed so as to reduce the second loss function Lt.

[0069]

number

[0070] When training is performed to reduce the second loss function, the feature space for the unknown class is merely configured to be distant from the known class in relation to the known class. For this reason, simply training to reduce the second loss function will only result in the feature space conceptually shown in the upper part of Figure 4 being configured. Therefore, by introducing the first loss function, a feature space that maintains a distance from the unknown class in the feature space is configured as shown in Figure 5. In this way, it is possible to configure the feature space conceptually shown in the lower part of Figure 4.

[0071] As described above, the first mechanism can be given by including the first loss function and the second loss function as terms in the loss function. Then, to give the second mechanism, a conventional DG is adopted. That is, in the learning method according to this embodiment, the loss function to be calculated is given by L shown in the following formula (4).

[0072]

number

[0073] Here, L DGis a loss function related to conventional DG. However, the DG method may be a suitable method (e.g., DeepAll, JiGen, MMLD, etc.) depending on the environment in which the learning method according to the present embodiment is applied. L DG is the loss function according to the DG method adopted. DG is a loss function that trains the machine learning model 10 so as to be able to classify the class of image data as DG. Therefore, in the learning method according to this embodiment, L DG is configured as a loss function (principal loss function) that takes a smaller value as the output for the training data matches the correct label of the training data.

[0074] Here, λ is a positive real number and is a hyperparameter that gives the degree of contribution of the first loss function and the second loss function. λ may be appropriately set depending on the environment to which the learning method according to the present embodiment is applied.

[0075] 5. Working Example Fig. 7 shows an example of the machine learning model 10 trained by the training method according to the present embodiment. The example shown in Fig. 7 shows the accuracy (%) for two cases: when the training data and image data as the target data are provided by VLCS, which is a benchmark database, and when they are provided by DomainNet, which is also a benchmark database.

[0076] Here, VLCS is a combination of four different databases (PASCAL VOC 2007, LabelMe, Caltech-101, Sun09) and consists of five categories of objects that appear in image data. DomainNet contains 345 categories of objects that appear in image data, consisting of six domains (Sketch, Real, Quickdraw, Painting, Infograph, Clipart). Figures 8 and 9 show examples of image data in VLCS and DomainNet.

[0077] In addition, to set the problem as an OSDG, the image data classes related to VLCS and DomainNet are divided into three sets Ck, Csu, and Cuu. Ck is a set of classes that are known classes in both the source domain and the target domain. Csu is a set of classes that are unknown classes in the source domain. Cuu is a set of classes that are unknown classes in the target domain.

[0078] In VLCS, we set |Ck|=3, |Csu|=1, and |Cuu|=1, and in DomainNet, we set |Ck|=10, |Csu|=167, and |Cuu|=168. Specifically, in VLCS, "car", "chair", and "person" are known classes, "dog" is an unknown class in the source domain, and "bird" is an unknown class in the target domain. In DomainNet, Csu and Cuu contain up to 2000 image data to balance the classes.

[0079] FIG. 7 shows a comparison of the accuracy rate when three conventional DG methods, DeepAll, JiGen, and MMLD, are used. The comparison is performed when the learning method according to the present embodiment is not applied (first row, L DG When only the second loss function is applied (second stage, w / L triplet , L DG + λLt), when the first and second loss functions are applied (third row, w / L metric , L DG +λLm).

[0080] As shown in FIG. 7, the accuracy rate can be improved by applying the learning method according to this embodiment. In particular, the accuracy rate is improved in all cases by applying the second loss function. Furthermore, it has been revealed that applying the first loss function and the second loss function has a higher overall effect of improving the accuracy rate than applying only the second loss function. In this way, by applying the learning method according to this embodiment, it is possible to effectively solve OSDGs.

[0081] 6. Information Processing Device By using the machine learning model 10 trained by the learning method according to the present embodiment, it is possible to configure an information processing device that extracts features of image data and classifies the image data based on the features, and that is capable of solving OSDG effectively. FIG 10 shows an example of the configuration of the information processing device 100.

[0082] The information processing device 100 receives image data as input and outputs a class of the image data. The information processing device 100 is a computer including a memory 110 and a processor 120. The information processing device 100 is, for example, a server (which may be virtually configured) configured on a communication network (typically the Internet).

[0083] The memory 110 stores data 111 and a program 112 that is executable by the processor 120. The processor 120 reads the data 111 and the program 112 from the memory 110, and executes processing in accordance with the program 112 based on the data 111.

[0084] Here, the machine learning model 10 trained by the learning method according to the present embodiment is provided as the program 112. That is, the feature extraction processing unit 11 and the class classification processing unit 12 are realized by a processor 120 that executes processing according to the program 112. At this time, parameters of the trained machine learning model 10 may be stored as data 111 or may be stored as part of the program 112.

[0085] The processor 120 reads the program 112 related to the machine learning model 10 and executes processing in accordance with the program 112 related to the machine learning model 10, thereby realizing an information processing device 100 capable of effectively solving an OSDG.

[0086] 7. Variations The learning method according to the present embodiment can also be applied to a case where the machine learning model 10 is trained only on the feature extraction processing unit 11. For example, this is the case where the feature extraction processing unit 11 is taken out (extracted) from the machine learning model 10 that has been trained as a conventional DG and trained.

[0087] At this time, the output of the target machine learning model 10 becomes the feature amount of the image data. The loss function calculated in the learning method according to the present embodiment is configured to include the first loss function and the second loss function as terms.

[0088] As a result, by providing the feature extraction processing unit 11 trained by the learning method according to this embodiment as the DG, it is possible to configure the machine learning model 10 capable of effectively solving the OSDG. Alternatively, the class classification processing unit 12 may be configured by the SVM or k-NN method, and the machine learning model 10 may be configured in combination with the feature extraction processing unit 11 trained by the learning method according to this embodiment. [Explanation of symbols]

[0089] 10 Machine Learning Models 11 Feature extraction processing section 12 Classification processing unit 100 Information processing device 110 Memory 111 Data 112 Programs 120 processors

Claims

1. A method for learning a machine learning model that extracts features of image data for classifying the image data into classes, comprising: A step of inputting learning data to which a correct answer label of the class is assigned, and acquiring the feature amounts of a plurality of the learning data; Calculating a loss function based on the feature amount; updating parameters of the machine learning model to reduce the loss function; Including, the classes are composed of a plurality of known classes indicating that the image data is of a specific type, and an unknown class indicating that the image data does not belong to any of the types; The loss function is a first loss function that gives a distance between the feature amount of the first anchor data and the feature amount of the appropriately selected training data for each of a plurality of first anchor data selected from the training data whose correct answer label is the unknown class, and that takes a larger value as the distance is smaller than a predetermined margin; a second loss function that gives, for each of a plurality of second anchor data selected from the training data, the correct label of which is the known class, the distance between the feature of the second anchor data and the feature of the appropriately selected training data, and that assumes a larger value as the distance related to the training data whose correct label is the same as the second anchor data is larger and the distance related to the training data whose correct label is different from the second anchor data is smaller; Contains as a term A learning method comprising:

2. 2. The learning method according to claim 1, The first anchor data is xa, the learning data selected for the first anchor data is xn, the number of samples of the learning data xn is K, the correct labels given to the first anchor data xa and the learning data xn, respectively, are ya and yn, a set of a plurality of known classes is C, the unknown class is u, the margin is α, the function giving the distance is d, and the feature amounts of the first anchor data xa and the learning data xn, respectively, are f(xa) and f(xn), the first loss function is represented by Ld shown in the following formula (1): A learning method comprising: [0010] 【number】 【number】

3. The learning method according to claim 1 or 2, The second loss function is a triplet loss function that uses the second anchor data as an anchor and has the same margin as the margin of the first loss function. A learning method comprising:

4. A learning method for a machine learning model that extracts features of image data and classifies a class of the image data based on the features, comprising: A step of inputting learning data to which a correct label of the class has been assigned, and acquiring outputs for a plurality of the learning data and the feature amounts of the plurality of the learning data; calculating a loss function based on the output and the feature amount; updating parameters of the machine learning model to reduce the loss function; Including, the classes are composed of a plurality of known classes indicating that the image data is of a specific type, and an unknown class indicating that the image data does not belong to any of the types; The loss function is A primary loss function whose value becomes smaller as the output matches the correct label; a first loss function that gives a distance between the feature amount of the first anchor data and the feature amount of the appropriately selected training data for each of a plurality of first anchor data selected from the training data whose correct answer label is the unknown class, and that takes a larger value as the distance is smaller than a predetermined margin; a second loss function that gives, for each of a plurality of second anchor data selected from the training data, the correct label of which is the known class, the distance between the feature of the second anchor data and the feature of the appropriately selected training data, and that assumes a larger value as the distance related to the training data whose correct label is the same as the second anchor data is larger and the distance related to the training data whose correct label is different from the second anchor data is smaller; Contains as a term A learning method comprising:

5. The learning method according to claim 4, The first anchor data is xa, the learning data selected for the first anchor data is xn, the number of samples of the learning data xn is K, the correct labels given to the first anchor data xa and the learning data xn, respectively, are ya and yn, a set of a plurality of known classes is C, the unknown class is u, the margin is α, the function giving the distance is d, and the feature amounts of the first anchor data xa and the learning data xn, respectively, are f(xa) and f(xn), the first loss function is represented by Ld shown in the following formula (1): A learning method comprising: [0025] 【number】

6. The learning method according to claim 4 or 5, The second loss function is a triplet loss function that uses the second anchor data as an anchor and has the same margin as the margin of the first loss function. A learning method comprising:

7. A learning method according to any one of claims 4 to 6, comprising: The primary loss function includes a loss function that trains the machine learning model so as to be able to classify the classes of the image data regardless of the domain of the image data. A learning method comprising:

8. An information processing device having a feature extraction processing unit that extracts a feature of image data, and a class classification processing unit that classifies a class of the image data based on the feature, the classes are composed of a plurality of known classes indicating that the image data is of a specific type, and an unknown class indicating that the image data does not belong to any of the types; The feature extraction processing unit and the class classification processing unit are configured using a machine learning model, The machine learning model is trained to reduce a loss function using a plurality of training data to which a correct label of the class is assigned, The loss function is A primary loss function that becomes smaller as the output for the training data matches the correct label; a first loss function that gives a distance between the feature amount of the first anchor data and the feature amount of the appropriately selected training data for each of a plurality of first anchor data selected from the training data whose correct answer label is the unknown class, and that takes a larger value as the distance is smaller than a predetermined margin; a second loss function that gives, for each of a plurality of second anchor data selected from the training data, the correct label of which is the known class, the distance between the feature of the second anchor data and the feature of the appropriately selected training data, and that assumes a larger value as the distance related to the training data whose correct label is the same as the second anchor data is larger and the distance related to the training data whose correct label is different from the second anchor data is smaller; Contains as a term 23. An information processing apparatus comprising:

9. 9. The information processing device according to claim 8, The first anchor data is xa, the learning data selected for the first anchor data is xn, the number of samples of the learning data xn is K, the correct labels given to the first anchor data xa and the learning data xn, respectively, are ya and yn, a set of a plurality of known classes is C, the unknown class is u, the margin is α, the function giving the distance is d, and the feature amounts of the first anchor data xa and the learning data xn, respectively, are f(xa) and f(xn), the first loss function is represented by Ld shown in the following formula (1):

23. An information processing apparatus comprising: [0030] 【number】

10. 10. The information processing device according to claim 8, The second loss function is a triplet loss function that uses the second anchor data as an anchor and has the same margin as the margin of the first loss function.

23. An information processing apparatus comprising:

11. The information processing device according to any one of claims 8 to 10, The primary loss function includes a loss function that trains the machine learning model so as to be able to classify the classes of the image data regardless of the domain of the image data.

23. An information processing apparatus comprising:

12. A learning program that causes a computer to execute the learning method according to any one of claims 1 to 7.

Images