Learning methods, information processing devices, and recording media containing the learning program.

Abstract

This disclosure provides a learning method, an information processing apparatus, and a recording medium containing a learning program. The learning method constitutes a machine learning model that extracts features from image data and classifies the image data into categories based on these features, enabling efficient solution of the OSDG (Optical Characteristic of Image Data) as a new problem. The learning method of this disclosure includes: a step of obtaining features from multiple learning data provided with correct answer labels for each category as input; a step of calculating a loss function based on the features; and a step of updating the parameters of the machine learning model by reducing the loss function. Here, the categories include multiple known categories representing that the image data belongs to a specific type and unknown categories representing that the image data does not belong to any type. Furthermore, the loss function is configured to include a first loss function and a second loss function.

Description

Technical Field

[0001] This disclosure relates to techniques for extracting features from image data and classifying the image data into categories based on these features. Background Technology

[0002] In machine learning, it is known that performance degrades when the domain of the test data differs from the domain of the learning data. Domain generalization (DG) is a technique used to maintain performance even when the domain of the test data (target domain) differs from the domain of the learning data (source domain). In many applications utilizing machine learning, there is a demand for good performance in new environments different from the learning data; therefore, DG has received particular attention in recent years. As an existing technology related to machine learning, patent document 1 can be cited as an example.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-091443 Summary of the Invention

[0006] Previously, when applying Data Grading (DG) to the learning of devices performing category classification, it was required that all categories in the target domain be classified appear as source domains. However, in practical applications (e.g., in autonomous vehicles), it is also necessary to distinguish cases where no category appeared during the learning process.

[0007] Therefore, the inventors of this disclosure define the problem of implementing DG (Open Set Domain Generalization) in which there are known and unknown categories in both the source and target domains as OSDG (Open Set Domain Generalization). The known category represents a specific type (the type of interest), and the unknown category represents any type that does not belong to the type represented by the known category (the type of interest). To solve OSDG, it is required to simultaneously implement the separation of known and unknown categories and DG.

[0008] Typically, conventional Dependency Gaussians (DGs) work by aligning the distributions of data across multiple source domains in the feature space. However, undefined classes exist in OSDGs, so simply applying conventional DGs to OSDGs is ineffective.

[0009] This disclosure was made in view of the above-mentioned problems, and its purpose is to provide a technique for efficiently solving OSDG, which relates to the technique of extracting feature quantities of image data and classifying the categories of image data according to the feature quantities.

[0010] The first disclosure relates to a learning method for a machine learning model, which extracts feature quantities of the image data for classifying categories of image data.

[0011] The first disclosed learning method includes: a step of obtaining the feature quantities of a plurality of said learning data, which are provided with correct answer labels of said category, as input; a step of calculating a loss function based on said feature quantities; and a step of updating the parameters of said machine learning model in a manner that reduces said loss function.

[0012] Here, the categories include multiple known categories indicating that the image data belongs to a specific kind and unknown categories indicating that the image data does not belong to any of the aforementioned kinds.

[0013] Additionally, the loss function includes, as a term, a first loss function that provides the distance between the feature value of the first anchor data selected from the learning data whose correct answer label is the unknown category and the feature value of the appropriately selected learning data, wherein the smaller the distance is than a predetermined margin, the larger the value of the first loss function; and a second loss function that provides the distance between the feature value of the second anchor data selected from the learning data whose correct answer label is the known category and the feature value of the appropriately selected learning data, wherein the larger the distance involved in learning data whose correct answer label is the same as the second anchor data and the smaller the distance involved in learning data whose correct answer label is different from the second anchor data, the larger the value of the second loss function.

[0014] The learning method disclosed in the second disclosure has the following additional characteristics compared to the learning method disclosed in the first disclosure.

[0015] When the first anchor data is set as xa, the learning data selected for the first anchor data is set as xn, the number of samples of the learning data xn is set as K, the correct answer labels provided for the first anchor data xa and the learning data xn are set as ya and yn respectively, the set of multiple known categories is set as C, the unknown category is set as u, the margin is set as α, the function that provides the distance is set as d, and the feature quantities of the anchor data xa and the learning data xn are set as f(xa) and f(xn) respectively, the first loss function is represented by Ld as shown in the following equation (1).

[0016]

Mathematical Formula 1

[0017]

[0018] N={(x ax ni )|y a ∈{u), y ni ∈C∪{u},d(f(x a ), f(x) ni ))<α}, i={1, 2,…,K}

[0019] The learning method disclosed in the third disclosure has the following characteristics compared to the learning methods disclosed in the first or second disclosure.

[0020] The second loss function is a triplet loss function constructed with the second anchor data as the anchor and the same residual as the first loss function.

[0021] The fourth disclosure relates to a learning method for a machine learning model that extracts features from image data and classifies the categories of the image data based on the features.

[0022] The fourth disclosed learning method includes: a step of taking learning data provided with correct answer labels of the said category as input, obtaining outputs for multiple learning data and the feature quantities of the multiple learning data; a step of calculating a loss function based on the outputs and the feature quantities; and a step of updating the parameters of the machine learning model in a manner that reduces the loss function.

[0023] Here, the categories include multiple known categories indicating that the image data belongs to a specific kind and unknown categories indicating that the image data does not belong to any of the aforementioned kinds.

[0024] Additionally, the loss function includes, as a term, a main loss function, the smaller the value of the main loss function, the more consistent the output is with the correct answer label; a first loss function, which provides the distance between the feature value of the first anchor data selected from the learning data whose correct answer label is the unknown category and the feature value of the appropriately selected learning data, the larger the value of the first loss function, the smaller the distance is compared to a predetermined margin; and a second loss function, which provides the distance between the feature value of the second anchor data selected from the learning data whose correct answer label is the known category and the feature value is the appropriately selected learning data, the larger the distance involved in the learning data whose correct answer label is the same as the second anchor data and the smaller the distance involved in the learning data whose correct answer label is different from the second anchor data, the larger the value of the second loss function.

[0025] The learning method disclosed in the fifth disclosure has the following additional characteristics compared to the learning method disclosed in the fourth disclosure.

[0026] When the first anchor data is set as xa, the learning data selected for the first anchor data is set as xn, the number of samples of the learning data xn is set as K, the correct answer labels provided for the first anchor data xa and the learning data xn are set as ya and yn respectively, the set of multiple known categories is set as C, the unknown category is set as u, the margin is set as α, the function that provides the distance is set as d, and the feature quantities of the anchor data xa and the learning data xn are set as f(xa) and f(xn) respectively, the first loss function is represented by Ld as shown in the following equation (1).

[0027]

Mathematical Formula 2

[0028]

[0029] N={(x a x ni )|y a ∈{u), y ni ∈C∪{u},d(f(x a ), f(x) ni ))<α}, i={1, 2,…,K}

[0030] The learning method disclosed in the sixth disclosure has the following characteristics compared to the learning methods disclosed in the fourth or fifth disclosure.

[0031] The second loss function is a triplet loss function constructed with the second anchor data as the anchor and the same residual as the first loss function.

[0032] The learning method disclosed in the 7th disclosure has the following additional characteristics compared to any of the learning methods disclosed in the 4th to 6th disclosures.

[0033] The main loss function includes a loss function that learns the machine learning model in a way that allows the image data to be classified into categories regardless of the domain of the image data.

[0034] The 8th disclosure relates to an information processing apparatus, comprising: a feature extraction processing unit for extracting features from image data; and a category classification processing unit for classifying the image data according to the features.

[0035] The categories include multiple known categories representing that the image data belongs to a specific type, and unknown categories representing that the image data does not belong to any of the stated categories. Furthermore, the feature extraction processing unit and the category classification processing unit are composed of machine learning models. The machine learning model is learned using multiple training data sets provided with the correct answer labels for each category, in a manner that reduces the loss function.

[0036] Here, the loss function includes, as a term, a main loss function, the smaller the value of the main loss function, the more consistent the output is with the correct answer label; a first loss function, which provides the distance between the feature value of the first anchor data selected from the learning data whose correct answer label is the unknown category and the feature value of the appropriately selected learning data, the larger the value of the first loss function, the smaller the distance is compared to a predetermined margin; and a second loss function, which provides the distance between the feature value of the second anchor data selected from the learning data whose correct answer label is the known category and the feature value is the appropriately selected learning data, the larger the distance involved in the learning data whose correct answer label is the same as the second anchor data and the smaller the distance involved in the learning data whose correct answer label is different from the second anchor data, the larger the value of the second loss function.

[0037] The information processing apparatus disclosed in the 9th disclosure has the following features compared to the information processing apparatus disclosed in the 8th disclosure.

[0038] When the first anchor data is set as xa, the learning data selected for the first anchor data is set as xn, the number of samples of the learning data xn is set as K, the correct answer labels provided for the first anchor data xa and the learning data xn are set as ya and yn respectively, the set of multiple known categories is set as C, the unknown category is set as u, the margin is set as α, the function that provides the distance is set as d, and the feature quantities of the anchor data xa and the learning data xn are set as f(xa) and f(xn) respectively, the first loss function is represented by Ld as shown in the following equation (1).

[0039]

Mathematical Expression 3

[0040]

[0041] N={(x a x ni )|y a ∈{u},y ni ∈C∪{u},d(f(x a ), f(x) ni ))<α}, i={1, 2,…,K}

[0042] The information processing apparatus disclosed in the 10th disclosure has the following features compared to the information processing apparatus disclosed in the 8th or 9th disclosure.

[0043] The second loss function is a triplet loss function constructed with the second anchor data as the anchor and the same residual as the first loss function.

[0044] The information processing apparatus disclosed in the 11th disclosure has the following features compared to the information processing apparatus disclosed in any one of the 8th to 10th disclosures.

[0045] The main loss function includes a loss function that learns the machine learning model in a way that allows the image data to be classified into categories regardless of the domain of the image data.

[0046] The 12th disclosure is a computer-readable recording medium encoded with a learning program that enables the computer to execute the learning method disclosed in any one of the 1st to 7th disclosures.

[0047] According to this disclosure, the loss function includes a first loss function and a second loss function as terms. Therefore, a feature space capable of segmenting image data by removing unknown categories can be constructed. Furthermore, a machine learning model capable of efficiently solving OSDG can be constructed, and an information processing apparatus capable of efficiently solving OSDG can be provided. Attached Figure Description

[0048] Figure 1 This is a conceptual diagram used to illustrate the case where the data being used as an object is image data.

[0049] Figure 2 This is a block diagram illustrating a structural example of a machine learning model that learns using the learning method of this embodiment.

[0050] Figure 3 This is a flowchart illustrating the learning method of this embodiment.

[0051] Figure 4 It is a conceptual graph representing the feature space realized by the first loss function.

[0052] Figure 5 This is a conceptual diagram of the feature space when learning is performed by reducing the first loss function.

[0053] Figure 6 This is a conceptual diagram of the feature space when learning is performed by reducing the second loss function.

[0054] Figure 7 This is a table illustrating an embodiment of a machine learning model that learns using the learning method of this implementation.

[0055] Figure 8 This is a diagram showing an example of VLCS image data.

[0056] Figure 9 This is a diagram illustrating an example of image data from a DomainNet.

[0057] Figure 10 This is a block diagram illustrating a structural example of the information processing apparatus of this embodiment.

[0058] Symbol Explanation

[0059] 10: Machine learning model; 11: Feature extraction processing unit; 12: Category classification processing unit; 100: Information processing device; 110: Memory; 111: Data; 112: Program; 120: Processor. Detailed Implementation

[0060] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, in the embodiments shown below, when the number, quantity, amount, range, etc., of each element are mentioned, the idea of ​​the present disclosure is not limited to the mentioned quantity, except where specifically stated or clearly determined in principle. Furthermore, the structures, etc., described in the embodiments shown below are not essential to the idea of ​​the present disclosure, except where specifically stated or clearly determined in principle. In addition, in each figure, the same or equivalent parts are given the same symbols, and their repeated descriptions are appropriately simplified or omitted.

[0061] 1. OSDG (Open Set Domain Generalization)

[0062] The learning method described in this embodiment is a machine learning model that extracts features from image data and classifies the image data according to these features. In particular, it relates to a machine learning model aimed at solving OSDG (Optical Distributed Image Data). Hereinafter, the OSDG is explained when the data being solved is image data.

[0063] OSDG addresses the following problem: Classification of data includes known classes representing specific kinds (kinds of interest) and unknown classes representing any kind that does not belong to the known classes (kinds of no interest), regardless of the domain of the data being the object.

[0064] Figure 1 This is a conceptual diagram used to illustrate the case where the data being used is image data. Figure 1 In this context, the category represents the type of object captured in the image data. Furthermore, the categories include known categories such as "dog," "horse," and "person," as well as unknown categories. Here, an unknown category indicates that the object captured in the image data is not any of the categories "dog," "horse," or "person." Additionally, in... Figure 1 The image data (photographs, drawings, sketches, etc.) with different domains are shown.

[0065] In other words, Figure 1 In the context of OSDG (Object Recognition and Classification), the problem is to classify objects in image data as "dog," "horse," or "person," regardless of the domain, while removing image data containing other objects. For example, it's desirable to identify objects in image data that are "dog," "horse," or "person" (this is of interest), but not concerned with the presence of other objects. A practical example of this is the desire to perform object recognition from image data.

[0066] In constructing the solution Figure 1 In the case of the OSDG machine learning model shown, such as Figure 1 As shown, multiple image datasets from different domains, each labeled with the correct answer, are used as training data for learning. Unlike previous Data Grading (DG) methods, this method includes unknown categories within the classification categories, and the training data includes image data with correct answer labels belonging to unknown categories. Additionally, the object data also includes image data with unknown categories.

[0067] Furthermore, similar to previous DGs, OSDG includes cases where image data from domains not present in the learning data is treated as object data. For example, in Figure 1 In this context, the learning data includes image data such as photographs (paragraph 1, photo), drawings (paragraph 2, art), and sketches (paragraph 3, sketch). On the other hand, the object data consists of image data of cartoon-style, distorted objects. Hereinafter, the domain of the learning data will be referred to as the "source domain," and the domain of the object data will be referred to as the "target domain."

[0068] Generally, in classifying image data, features are extracted from the image data, and the image data is classified according to the position of the extracted features in the feature space. Typically, conventional Direct Generation (DG) works by aligning the feature space using learning data spanning multiple source domains. However, even when conventional DG is simply applied to OSDG, proper alignment of the feature space cannot be guaranteed for unknown categories in the target domain. This is because in conventional DG, unknown categories are not required in either the source or target domains. Therefore, in conventional DG, image data of the target domain that should be classified as an unknown category may be captured at a position in the feature space corresponding to a known category.

[0069] To solve the OSDG, at least two mechanisms are required. One (the first mechanism) is a feature space that enables segmentation of image data by removing unknown categories. The other (the second mechanism) is a distribution matching mechanism that ensures consistent positions of known categories across multiple domains in the feature space. In previous DGs, the latter mechanism enabled domain-independent category classification, while the former mechanism, as mentioned earlier, was insufficient.

[0070] Therefore, the learning method of this embodiment provides a first mechanism. Furthermore, regarding the second mechanism, a conventional DG can be employed. Thus, a machine learning model capable of effectively solving the OSDG can be constructed.

[0071] 2. Machine Learning Model

[0072] The following describes the machine learning model that learns using the learning method described in this embodiment. Figure 2 This is a block diagram illustrating a structural example of a machine learning model 10 that learns using the learning method of this embodiment.

[0073] The machine learning model 10 includes a feature extraction processing unit 11 and a category classification processing unit 12. The feature extraction processing unit 11 takes image data as input and outputs features. The features output by the feature extraction processing unit 11 are passed to the category classification processing unit 12. The category classification processing unit 12 takes the features as input and outputs categories. In other words, the machine learning model 10 performs category classification on the image data.

[0074] The machine learning model 10 has parameters for the feature extraction processing unit 11 and the category classification processing unit 12, respectively. That is, the machine learning model 10 learns by updating the parameters.

[0075] Typically, the machine learning model 10 is composed of a convolutional neural network. In this case, the feature extraction processing unit 11 includes convolutional layers and pooling layers, and the category classification processing unit 12 is composed of a fully concatenated layer. Furthermore, the parameters are the filters of the convolutional layers and the weights of the fully concatenated layer.

[0076] The machine learning model 10 can also be constructed using other methods. For example, the feature extraction processing unit 11 can be constructed using a convolutional neural network, and the category classification processing unit 12 can be constructed using SVM or k-NN methods.

[0077] Furthermore, typically, the machine learning model 10 is provided through a program, and the processing of the machine learning model 10 is implemented by a processor. In this case, the parameters of the machine learning model 10 can be provided either as part of the program or stored in memory and read by the processor. Additionally, parameter updates can be performed either through program updates or through memory updates.

[0078] 3. Learning methods

[0079] The learning method of this embodiment calculates a loss function and updates the parameters of the machine learning model 10 by reducing the loss function. The learning method of this embodiment will be described below.

[0080] Figure 3 This is a flowchart illustrating the learning method of this embodiment.

[0081] In step S100, the learning data is input into the machine learning model 10 to obtain the output and feature values ​​of the multiple learning data. After step S100, the process proceeds to step S110.

[0082] In step S110, a loss function is calculated based on the output and feature values ​​obtained in step S100. Here, the learning method of this embodiment is characterized by a loss function calculated to provide the first mechanism. Details regarding the calculated loss function will be described later. After step S110, the process proceeds to step S120.

[0083] In step S120, the gradient of the loss function obtained in step S110 is calculated. The gradient of the loss function can be calculated using appropriate known techniques. For example, in the case where the machine learning model 10 is constructed using a convolutional neural network, the gradient of the loss function is typically calculated using the backpropagation method. After step S120, the process proceeds to step S130.

[0084] In step S130, the parameters of the machine learning model 10 are updated to reduce the loss function based on the gradient calculated in step S120. That is, the parameters are updated using gradient descent. Here, the hyperparameters related to the parameter update can be appropriately determined according to the environment in which the learning method of this embodiment is applied. For example, the momentum method can also be used for parameter updating.

[0085] In step S140, it is determined whether the learning termination condition is met. The termination condition is, for example, that the number of parameter updates is more than a predetermined value or the loss function is less than a predetermined value.

[0086] If the learning termination condition is met (step S140; Yes), the learning ends. If the learning termination condition is not met (step S140; No), return to step S100 and repeat the learning process.

[0087] also, Figure 3 The learning method shown enables processing performed by one or more computers when a computer program (learning program) is executed for one or more computers. Furthermore, this also provides the ability to... Figure 3 The learning method shown is encoded as a computer program that can be executed by one or more computers on a computer-readable recording medium.

[0088] 4. Loss Function

[0089] The inventors of this disclosure have conceived of constructing a feature space through metric learning to provide a first mechanism. Typical loss functions associated with metric learning (triplet loss, contrastive loss) are configured to make similar categories closer to each other in the feature space and different categories farther apart. Thus, it is expected that a feature space capable of easily segmenting each category can be constructed.

[0090] However, it's unclear how to handle unknown categories in loss functions related to distance learning. What is clear is that, with a triplet loss function, learning data for unknown categories does not form a direct pair with learning data for known categories. However, it's not easy to determine whether a direct pair can be formed using learning data for unknown categories.

[0091] A simple approach is to use the training data of the unknown class only to form negative pairs in the triplet loss function. However, the inventors of this disclosure have found through ablation studies that this approach alone cannot clearly separate the features of the training data of the unknown class in the feature space, which is insufficient for OSDG.

[0092] Therefore, in addition to the above-mentioned approach, the inventors of this disclosure have also conceived of introducing a first loss function for constructing a feature space that maintains a distance from the unknown category in the feature space. Figure 4 It is a conceptual graph representing the feature space realized by the first loss function. Figure 4 The figures shown represent features in the feature space. Here, identical figures represent the same category. Furthermore, Figure 4 The dashed lines shown represent examples of the best-identifiable boundaries for OSDG.

[0093] like Figure 4As shown, a feature space can be constructed by distance learning to make known categories close to each other. However, without the first loss function, known categories are close to unknown categories, making them difficult to separate. On the other hand, by incorporating the first loss function, unknown categories maintain sufficient distance from other categories, allowing for clearer separation of unknown categories.

[0094] The loss function calculated in the learning method of this embodiment will be explained in detail below.

[0095] In the learning method of this embodiment, the calculated loss function includes a first loss function and a second loss function as terms.

[0096] First, the first loss function is explained. The first loss function is represented by Ld as shown in the following equation (1).

[0097]

Mathematical Expression 4

[0098]

[0099] N={(x a x ni )|y a ∈{u},y ni ∈C∪{u},d(f(x a ), f(x) ni ))<α}i={1,2,…,K}

[0100] Here, the learning data is represented by x, the correct answer label provided for the learning data x is represented by y, the set of known categories is represented by C, the unknown category is represented by u, the feature quantity of the learning data x is represented by f(x), and the function that provides the distance in the feature quantity space is represented by d.

[0101] In other words, N is the set of learning data xa (hereinafter also referred to as "first anchor data") with the correct answer label ya set to the unknown category u, and the combination of learning data xn selected for the first anchor data xa with a sample size K. The selected learning data xn is conditional upon the distance to the first anchor data xa being less than a predetermined margin α. This is a condition that suppresses the selection of learning data that has already achieved the purpose of the first loss function and does not contribute to changes in the first loss function. Therefore, efficient learning processing can be achieved.

[0102] Here, 'i' is an additional number used to distinguish the selected learning data. Additionally, the margin α specifies the distance to the unknown category in the feature space. The margin α can be appropriately provided depending on the environment in which the learning method of this embodiment is applied.

[0103] Furthermore, in the structure of N, multiple or all of the first anchor data xa are selected from the learning data where the correct answer is labeled as an unknown category. For each first anchor data xa, learning data xn is selected with a sample size K. In addition, in equation (1), |N| represents the number of features in N.

[0104] In equation (1), the distance function d can be an appropriate function depending on the environment in which the learning method of this embodiment is applied. For example, cosine similarity can be exemplified as d.

[0105] As shown in equation (1), for each first anchor data xa, the smaller the distance ratio α between the feature quantity f(xa) of the first anchor data xa and the feature quantity f(xn) of the learning data xn, the larger the value of the first loss function Ld. That is, learning is performed by reducing the first loss function Ld (updating the parameters of the machine learning model 10), thereby constructing a feature space that maintains a distance in the feature space from the unknown category. Figure 5 A conceptual diagram of the feature space is shown for learning in a way that reduces the first loss function Ld.

[0106] Next, the second loss function will be explained. The second loss function is represented by Lt as shown in the following equation (2).

[0107]

Mathematical Expression 5

[0108]

[0109] T={(x a x p x ni )|y a =y p ∈C, y ni ∈C∪{u},y a ≠y ni d(f(x) a ), f(x) p ))<d(f(x a ), f(x) ni ))+α}i={1,2,…,K}

[0110] As shown in equation (2), the second loss function Lt is a triplet loss function anchored by the learning data (hereinafter also referred to as "second anchor data") with the correct answer label as the known category. It consists of the same residual α as the residual associated with the first loss function.

[0111] Furthermore, in the structure of the triplet set T, the positive data xp is the learning data with the same class as the second anchor data xa, and can be one or more randomly selected learning data. Therefore, it is not necessary to select all learning data with the same class as the second anchor data xa as positive data xp, thus achieving greater efficiency in the learning process.

[0112] Furthermore, for each of the second anchor data xa, negative data xn is selected with a sample size K. Additionally, in the aforementioned triplet set T, a semi-hard condition for selecting negative data xn is provided, but appropriate conditions can also be used (e.g., a hard selection condition can also be provided). Furthermore, the sample size K may not be the same as the first loss function.

[0113] The greater the distance between the feature value f(xa) of the second anchor data xa and the feature value f(xp) of the positive data xp, and the smaller the distance between the feature value f(xa) of the second anchor data xa and the feature value f(xn) of the negative data xn, the larger the value of the second loss function (triple loss function) Lt. Furthermore, the characteristic of the second anchor data xa lies in selecting learning data where the correct answer label is a known category.

[0114] In other words, learning is performed in a way that reduces the second loss function Lt (updating the parameters of machine learning model 10), thereby satisfying the following inequality (3). Moreover, with respect to known categories, the feature space is constructed in a way that is close to each other in the feature space with the same category and further away from each other in the feature space with different categories. Figure 6 A conceptual diagram of the feature space is shown for learning in a way that reduces the second loss function Lt.

[0115]

Mathematical Expression 6

[0116] d(f(x a ), f(x) p ))+α≤d(f(x a ), f(x) ni ))…(3)

[0117] When learning by reducing the second loss function, regarding the unknown category, in its relationship with the known category, it only constructs the feature space in a way that is far removed from the known category. Therefore, learning solely by reducing the second loss function only constitutes... Figure 4 The upper part conceptually represents the feature space. Therefore, by introducing the first loss function, a feature space is constructed as shown above. Figure 5 The feature space shown maintains the distance in the feature space from the unknown category. Thus, it is possible to construct a feature space that... Figure 4 The lower part conceptually illustrates the feature space.

[0118] As explained above, the loss function includes both a first loss function and a second loss function, thereby providing a first mechanism. Furthermore, to provide a second mechanism, the conventional DG is employed. That is, in the learning method of this embodiment, the calculated loss function is provided using L as shown in the following equation (4).

[0119]

Mathematical Expression 7

[0120] L = L DG +λ*Lm…(4)

[0121] Lm=Ld+Lt

[0122] Here, L DG This is a loss function related to conventional DG. However, the DG method can be tailored to the environment in which the learning method of this implementation is applied (e.g., DeepAll, JiGen, MMLD, etc.). DG This is the loss function corresponding to the DG technique used. Furthermore, L DG The loss function of machine learning model 10 is learned by using DG in a way that allows it to classify categories of image data. Therefore, in the learning method of this embodiment, L DG It has a structure that serves as the loss function (main loss function), in which the more consistent the label is with the correct answer in the learning data, the smaller the output value for the learning data.

[0123] Furthermore, λ is a positive real number and is a hyperparameter that provides the degree of contribution of the first and second loss functions. λ can be appropriately provided according to the environment in which the learning method of this embodiment is applied.

[0124] 5. Examples

[0125] Figure 7 An embodiment of a machine learning model 10 that learns using the learning method of this embodiment is shown. Figure 7 In the illustrated embodiments, the accuracy (%) is shown for two cases: one where image data as learning data and object data is provided using VLCS as a benchmark database, and the other where it is provided using DomainNet as a benchmark database.

[0126] Here, VLCS is a combination of four different databases (PASCAL VOC 2007, LabelMe, Caltech-101, and Sun09), encompassing five types of objects captured in the image data. Additionally, DomainNet, regarding objects captured in the image data, includes 345 types comprised of six domains (Sketch, Real, Quickdraw, Painting, Infograph, and Clipart). Figure 8 as well as Figure 9 Examples of image data from VLCS and DomainNet are shown.

[0127] Furthermore, since the problem is defined as OSDG, the image data related to VLCS and DomainNet are divided into three sets: Ck, Csu, and Cuu. Ck is the set of categories with known categories in both the source and target domains. Csu is the set of categories with unknown categories in the source domain. Cuu is the set of categories with unknown categories in the target domain.

[0128] Then, in VLCS, |Ck| = 3, |Csu| = 1, and |Cuu| = 1; in DomainNet, |Ck| = 10, |Csu| = 167, and |Cuu| = 168. Specifically, in VLCS, "car," "chair," and "person" are set as known categories, "dog" is set as an unknown category in the source domain, and "bird" is set as an unknown category in the target domain. Additionally, in DomainNet, Csu and Cuu include a maximum of 2000 image data points in a balanced manner across all categories.

[0129] exist Figure 7 In the previous DG, comparisons of correct answer rates were shown for various cases using DeepAll, JiGen, and MMLD. The comparisons also included cases where the learning method of this implementation was not applied (paragraph 1, only L). DG ), the case where only the second loss function is applied (paragraph 2, w / L) triplet L DG +λLt), applying the first loss function and the second loss function (paragraph 3, w / L) metric L DG +λLm).

[0130] like Figure 7As shown, by applying the learning method of this embodiment, the correct answer rate can be improved. In particular, by applying the second loss function, the correct answer rate is improved in all cases. Furthermore, it is clear that by applying both the first and second loss functions, the overall improvement in the correct answer rate is greater than that achieved by applying only the second loss function. Thus, by applying the learning method of this embodiment, the OSDG can be solved effectively.

[0131] 6. Information processing device

[0132] By using the machine learning model 10 learned through the learning method of this embodiment, an information processing device can be constructed to extract feature quantities of image data and classify the categories of image data according to the feature quantities. This information processing device can effectively solve OSDG. Figure 10 An example of the structure of the information processing device 100 is shown.

[0133] The information processing device 100 takes image data as input and outputs a category of image data. The information processing device 100 is a computer equipped with a memory 110 and a processor 120. The information processing device 100 may be, for example, a server configured on a communication network (typically the Internet) (or it may be configured virtually).

[0134] The memory 110 stores data 111 and a program 112 that can be executed by the processor 120. The processor 120 reads the data 111 and the program 112 from the memory 110 and performs processing according to the program 112 based on the data 111.

[0135] Here, as program 112, a machine learning model 10 learned by the learning method of this embodiment is provided. That is, the feature extraction processing unit 11 and the category classification processing unit 12 are implemented by a processor 120 that performs processing following program 112. At this time, the parameters of the learned machine learning model 10 can be stored either as data 111 or as part of program 112.

[0136] The processor 120 reads out the program 112 related to the machine learning model 10 and executes the processing that follows the program 112 related to the machine learning model 10, thereby realizing the information processing device 100 that can effectively solve the OSDG.

[0137] 7. Variations

[0138] The learning method of this embodiment can also be applied to the case where the machine learning model 10 is learned only by the feature extraction processing unit 11. For example, in the conventional DG, the machine learning model 10 that has been learned is learned by extracting a portion of the feature extraction processing unit 11.

[0139] At this time, the output of the machine learning model 10, which is the object, is the feature quantity of the image data. Moreover, the loss function calculated in the learning method of this embodiment is configured to include a first loss function and a second loss function as terms.

[0140] Therefore, the feature extraction processing unit 11, which is provided as a DG and has been learned by the learning method of this embodiment, can be used to construct a machine learning model 10 that can effectively solve the OSDG. Alternatively, the category classification processing unit 12 can be constructed using SVM or k-NN methods, and combined with the feature extraction processing unit 11, which has been learned by the learning method of this embodiment, to construct the machine learning model 10.

Claims

1. A learning method, which is a machine learning model for solving OSDG (Open Set Domain) generalization by extracting features of the image data for classifying categories of image data, the learning method being characterized by comprising: The step of obtaining the feature values ​​of multiple sets of learning data by taking the learning data provided with the correct answer labels of the said categories as input; The steps of calculating the loss function based on the aforementioned features; and The step of updating the parameters of the machine learning model in a manner that reduces the loss function. The categories include multiple known categories indicating that the image data belongs to a specific kind and unknown categories indicating that the image data does not belong to any of the said kinds. The loss function includes, as a term: The first loss function provides the distance between the feature value of the first anchor data and the feature value of the appropriately selected learning data for each of the multiple first anchor data selected from the learning data whose correct answer label is the unknown category. The smaller the distance is compared to a predetermined margin, the larger the value of the first loss function. as well as The second loss function provides the distance between the feature value of the second anchor data and the feature value of the appropriately selected learning data for each of the multiple second anchor data selected from the learning data whose correct answer label is the known category. The larger the distance between the learning data whose correct answer label is the same as the second anchor data, and the smaller the distance between the learning data whose correct answer label is different from the second anchor data, the larger the value of the second loss function. The first loss function is introduced to construct a feature space that maintains a distance from the unknown category in the feature space.

2. The learning method according to claim 1, characterized in that, When the first anchor data is set as xa, the learning data selected for the first anchor data is set as xn, the number of samples of the learning data xn is set as K, the correct answer labels provided for the first anchor data xa and the learning data xn are set as ya and yn respectively, the set of multiple known categories is set as C, the unknown category is set as u, the margin is set as α, the function that provides the distance is set as d, and the feature quantities of the first anchor data xa and the learning data xn are set as f(xa) and f(xn) respectively, the first loss function is represented by Ld as shown in the following equation (1). 。 3. The learning method according to claim 1 or 2, characterized in that, The second loss function is a triplet loss function constructed with the second anchor data as the anchor and the same residual as the first loss function.

4. A learning method, which is a machine learning model for solving OSDG (Open Set Domain) generalization, that extracts features from image data and classifies the categories of the image data based on the features, and the learning method is characterized by comprising: The steps include taking learning data with correct answer labels for the provided categories as input, obtaining outputs for multiple sets of learning data, and the feature values ​​of the multiple sets of learning data; The steps of calculating the loss function based on the output and the feature quantities; and The step of updating the parameters of the machine learning model in a manner that reduces the loss function. The categories include multiple known categories indicating that the image data belongs to a specific kind and unknown categories indicating that the image data does not belong to any of the said kinds. The loss function includes, as a term: The more consistent the output is with the label of the correct answer, the smaller the value of the main loss function. The first loss function provides the distance between the feature value of the first anchor data and the feature value of the appropriately selected learning data for each of the multiple first anchor data selected from the learning data where the correct answer label is the unknown category. The smaller the distance is compared to a predetermined margin, the larger the value of the first loss function. The second loss function provides the distance between the feature value of the second anchor data and the feature value of the appropriately selected learning data for each of the multiple second anchor data selected from the learning data whose correct answer label is the known category. The larger the distance between the learning data whose correct answer label is the same as the second anchor data, and the smaller the distance between the learning data whose correct answer label is different from the second anchor data, the larger the value of the second loss function. The first loss function is introduced to construct a feature space that maintains a distance from the unknown category in the feature space.

5. The learning method according to claim 4, characterized in that, When the first anchor data is set as xa, the learning data selected for the first anchor data is set as xn, the number of samples of the learning data xn is set as K, the correct answer labels provided for the first anchor data xa and the learning data xn are set as ya and yn respectively, the set of multiple known categories is set as C, the unknown category is set as u, the margin is set as α, the function that provides the distance is set as d, and the feature quantities of the first anchor data xa and the learning data xn are set as f(xa) and f(xn) respectively, the first loss function is represented by Ld as shown in the following equation (1). 。 6. The learning method according to claim 4 or 5, characterized in that, The second loss function is a triplet loss function constructed with the second anchor data as the anchor and the same residual as the first loss function.

7. The learning method according to claim 4 or 5, characterized in that, The main loss function includes a loss function that learns the machine learning model in a way that allows the image data to be classified into categories regardless of the domain of the image data.

8. An information processing apparatus comprising a feature extraction processing unit for extracting feature quantities of image data and a category classification processing unit for classifying the image data according to the feature quantities, characterized in that, The categories include multiple known categories indicating that the image data belongs to a specific kind and unknown categories indicating that the image data does not belong to any of the said kinds. The feature extraction processing unit and the category classification processing unit are composed of machine learning models aimed at solving the OSDG, i.e., open set domain generalization. The machine learning model is learned using multiple training datasets provided with correct answer labels for the stated categories, in a manner that reduces the loss function. The loss function includes, as a term: The more consistent the output of the learning data is with the label of the correct answer, the smaller the value of the main loss function. The first loss function provides the distance between the feature value of the first anchor data and the feature value of the appropriately selected learning data for each of the multiple first anchor data selected from the learning data whose correct answer label is the unknown category. The smaller the distance is compared to a predetermined margin, the larger the value of the first loss function. as well as The second loss function provides the distance between the feature value of the second anchor data and the feature value of the appropriately selected learning data for each of the multiple second anchor data selected from the learning data whose correct answer label is the known category. The larger the distance between the learning data whose correct answer label is the same as the second anchor data, and the smaller the distance between the learning data whose correct answer label is different from the second anchor data, the larger the value of the second loss function. The first loss function is introduced to construct a feature space that maintains a distance from the unknown category in the feature space.

9. The information processing apparatus according to claim 8, characterized in that, When the first anchor data is set as xa, the learning data selected for the first anchor data is set as xn, the number of samples of the learning data xn is set as K, the correct answer labels provided for the first anchor data xa and the learning data xn are set as ya and yn respectively, the set of multiple known categories is set as C, the unknown category is set as u, the margin is set as α, the function that provides the distance is set as d, and the feature quantities of the first anchor data xa and the learning data xn are set as f(xa) and f(xn) respectively, the first loss function is represented by Ld as shown in the following equation (1). 。 10. The information processing apparatus according to claim 8 or 9, characterized in that, The second loss function is a triplet loss function constructed with the second anchor data as the anchor and the same residual as the first loss function.

11. The information processing apparatus according to claim 8 or 9, characterized in that, The main loss function includes a loss function that learns the machine learning model in a way that allows the image data to be classified into categories regardless of the domain of the image data.

12. A computer-readable recording medium encoded with a learning program that causes a computer to execute the learning method according to any one of claims 1 to 7.

Images