Learning device, inference device, learning system, learning method, and program

Abstract

In order to provide a technology that serves to further improve the accuracy of inference based on data individually included in heterogeneous domains, a learning device according to the present invention is provided with: a conversion means for generating converted features from a feature obtained from input data; and a learning means for training the conversion means by referring to the result of inference by an inference means referring to the features converted by the conversion means. The conversion means is provided with: a first conversion means for generating first converted features; and a second conversion means for generating second converted features. The learning means trains the conversion means so as to reduce the difference between a distribution of the first converted features and a distribution of the second converted features.

Description

Learning device, inference device, learning system, learning method, and recording medium

[0001] The present disclosure relates to a learning device, an inference device, a learning system, a learning method, and a recording medium.

[0002] A technique is known in which, when making inferences about input data, input features are transformed and the transformed features are used for inference. For example, Non-Patent Document 1 on heterogeneous domain adaptation discloses a configuration in which features of a target domain are transformed by a feature transformer and inferences are made based on the transformed features. Here, parameters of the feature transformers for the target domain and the source domain are mutually complementary.

[0003] Wei, P., et. al., "A general domain specific feature transfer framework for hybrid domain adaptation", IEEE Transactions on Knowledge and Data Engineering, 31(8),1440-1451, 2019

[0004] The configuration disclosed in Non-Patent Document 1 updates each feature transformer based on the loss in a feature space that includes features transformed based on data contained in each of the heterogeneous domains. This technology is expected to improve the accuracy of inference processing by referencing the transformed features. However, the technology described in Non-Patent Document 1 leaves room for improvement in terms of improving inference accuracy.

[0005] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one of its objectives is to provide a technology that can further improve the accuracy of inferences based on data contained in each of the heterogeneous domains.

[0006] A learning device according to one aspect of the present invention comprises: a conversion means for generating converted features from features obtained from input data including first data belonging to a first domain and second data belonging to a second domain; and a learning means for training the conversion means by referring to an inference result by an inference means that references the features converted by the conversion means, wherein the conversion means comprises first conversion means for generating first converted features from features obtained from the first data, and second conversion means for generating second converted features from features obtained from the second data, and the learning means trains the conversion means so that the difference between the distribution of the first converted features and the distribution of the second converted features is small.

[0007] An inference device according to one aspect of the present invention comprises an acquisition means for acquiring features from input data, a conversion means for generating features converted from the features, the conversion means being trained by referring to an inference result by an inference means that references the features converted by the conversion means, and an inference means for performing inference by referring to the converted features.

[0008] A learning system according to one aspect of the present invention comprises: a first learning device including: first conversion means for generating first converted features from features obtained from input first data; first inference means for making inferences by referring to the first converted features; and first learning means for training the first conversion means by referring to the inference results by the first inference means; a second learning device including: second conversion means for generating second converted features from features obtained from input second data; second inference means for making inferences by referring to the second converted features; and second learning means for training the conversion means by referring to the inference results by the second inference means; and an integration device for generating new parameters based on parameters referenced by the first inference means and the second inference means, respectively; and each of the first inference means and the second inference means updates each of the parameters to the new parameters created by the integration device.

[0009] A learning method according to one aspect of the present invention includes a conversion process that generates converted features from features obtained from input data, and a learning process that trains a conversion means that executes the conversion process by referring to an inference result based on the converted features, wherein the conversion process comprises a first conversion process that generates first converted features from features obtained from the first data, and a second conversion process that generates second converted features from features obtained from the second data, and the learning process trains the conversion means that executes the conversion process so as to reduce the difference between the distribution of the first converted features and the distribution of the second converted features.

[0010] A learning program according to one aspect of the present invention is a program for causing a computer to function as a learning device, and causes the computer to function as a conversion means for generating converted features from features obtained from input data, and a learning means for training the conversion means by referring to inference results by inference means that reference the features converted by the conversion means.

[0011] According to one aspect of the present invention, it is possible to further improve the accuracy of inference based on data contained in each of the different domains.

[0012] FIG. 1 is a block diagram showing the configuration of a learning device according to an exemplary embodiment. FIG. 2 is a flow diagram showing the flow of a learning method according to an exemplary embodiment. FIG. 3 is a block diagram showing the configuration of a learning system according to an exemplary embodiment. FIG. 4 is a block diagram showing the configuration of a learning device according to an exemplary embodiment. FIG. 5 is a flow diagram showing the flow of a learning method according to an exemplary embodiment. FIG. 6 is a block diagram showing the configuration of a learning device according to an exemplary embodiment. FIG. 7 is a flow diagram showing the flow of a learning method according to an exemplary embodiment. FIG. 8 is a flow diagram showing the flow of a learning process in a learning device according to an exemplary embodiment. FIG. 9 is a block diagram showing the configuration of an inference device according to an exemplary embodiment. FIG. 10 is a flow diagram showing the flow of an inference method according to an exemplary embodiment. FIG. 11 is a block diagram showing the configuration of a learning system according to an exemplary embodiment. FIG. 12 is a block diagram showing the configuration of a computer that functions as a learning device according to each exemplary embodiment.

[0013] [First Exemplary Embodiment] A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0014] <Configuration of Learning Device 1> The configuration of the learning device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the learning device 1. As shown in Fig. 1, the learning device 1 includes a conversion unit 11 and a learning unit 12. The learning device 1 according to this exemplary embodiment also functions as a feature transformation device that generates transformed features from features obtained from input data.

[0015] (Conversion Unit 11) The conversion unit 11 generates transformed features from features obtained from input data. The "input data" refers to data in which any item is associated with information related to that item. The "features obtained from the input data" can be obtained from any item and information related to that item. The input data can be data related to multiple objects, with information about each object associated with any item. Examples of the items include items related to physical information such as the target's age, gender, and medical history, as well as items related to social information such as the target's marital status, occupation, annual income, savings, and debt. The features can be numerical values ​​obtained from the information about the items. For example, if the information related to an item is numerical, the numerical values ​​can be used as features. If the information related to an item is not numerical, the information can be quantified and the numerical values ​​can be acquired as features. However, the above example does not limit the present exemplary embodiment.

[0016] The transform unit 11 includes a first transform unit and a second transform unit. The first transform unit generates a first transformed feature. The first transformed feature is generated from a feature obtained from the first data. The second transform unit generates a second transformed feature. The second transformed feature is generated from a feature obtained from the second data.

[0017] (Learning unit 12) The learning unit 12 trains the conversion unit 11 by referring to the inference result by the inference unit that references the feature converted by the conversion unit 11. The inference unit may be provided in a device different from the learning device 1.

[0018] The learning unit 12 trains the conversion unit 11 so as to reduce the difference between the inference result by the inference unit that refers to the converted feature amount and the correct label of the inference. That is, the learning unit S12 calculates the classification loss L ｓ (hereinafter, "loss value L ｓ ") is calculated, and the loss value L ｓ The learning unit 12 can cause the conversion unit 11 to learn by updating the parameters that the conversion unit 11 refers to.

[0019] The learning unit 12 trains the conversion unit 11 by referring to the distribution of the first transformed feature quantity and the distribution of the second transformed feature quantity in addition to the inference result. The learning unit 12 trains the conversion unit 11 so that the difference between the distribution of the first transformed feature quantity and the distribution of the second transformed feature quantity becomes small. Specifically, the domain loss λL obtained by referring to the distribution of the first transformed feature quantity f1 and the distribution of the second transformed feature quantity f2 is ｄ Learning is based on this.

[0020] The learning unit 12 performs learning based on the value of the loss function L (loss function value VLF) shown below: L=L s +λL d Domain loss λL ｄ can be the distance between the distributions. ｄ The calculation method of is not particularly limited, and can be calculated using, for example, Maximum Mean Discrepancy (MMD) and a covariance matrix. In addition, in the common feature space, the domain loss λL is calculated by adversarial learning that identifies whether the feature originates from the first data or the second data. ｄ For example, the domain loss λL ｄcan be the negative loss function of the discriminative parameters in adversarial learning.

[0021] When the domain loss λLd is calculated using a covariance matrix, the domain loss λLd can be expressed by the following formula (1). where D1 denotes the covariance matrix of the first data, and D2 denotes the covariance matrix of the second data.

[0022] As described above, the learning device 1 according to this exemplary embodiment is configured to cause the conversion unit 11 to learn by referring to the inference results of the inference unit that referenced the features converted by the conversion unit 11. Therefore, the learning device 1 according to this exemplary embodiment can provide a technology that can further improve the inference accuracy for inference based on data contained in each of the heterogeneous domains. Furthermore, the learning device 1 according to this exemplary embodiment makes the feature space including the converted features separable by inference.

[0023] <Flow of Learning Method S1> The flow of learning method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of learning method S1. Note that each step in this learning method may be executed by a processor provided in learning device 1, or by a processor provided in another device, or each step may be executed by a processor provided in a different device.

[0024] The conversion process S11 generates converted features from the features obtained from the input data.

[0025] The conversion process S11 includes a first conversion process and a second conversion process. The first conversion process generates first converted features. The first converted features are generated from features obtained from the first data. The second conversion process generates second converted features. The second converted features are generated from features obtained from the second data.

[0026] The learning process S12 trains the conversion unit 11 used for the conversion process S11. The learning process S12 is executed by referring to the inference result of the inference performed by referring to the feature converted by the conversion process S11.

[0027] The learning process S12 trains the conversion unit 11 that executes the conversion process S11 by referring to the distribution of the first transformed feature quantity and the distribution of the second transformed feature quantity in addition to the inference result. The learning process S12 trains the conversion unit 11 that executes the conversion process S11 so that the difference between the distribution of the first transformed feature quantity and the distribution of the second transformed feature quantity becomes small. Specifically, the domain loss λL obtained by referring to the distribution of the first transformed feature quantity f1 and the distribution of the second transformed feature quantity f2 is ｄ Learning is based on this.

[0028] The learning process S12 performs learning based on the value of the loss function L (loss function value VLF) shown below: L=L s +λL d Domain loss λL ｄ can be the distance between the distributions. ｄ The calculation method of is not particularly limited, and can be calculated using, for example, Maximum Mean Discrepancy (MMD) and a covariance matrix. In addition, in the common feature space, the domain loss λL is calculated by adversarial learning that identifies whether the feature originates from the first data or the second data. ｄ For example, the domain loss λL ｄ can be the negative loss function of the discriminative parameters in adversarial learning.

[0029] The learning process S12 trains the conversion process so that the difference between the inference result obtained by the inference process that references the converted feature amount and the correct label of the inference is reduced. That is, the learning process S12 calculates the classification loss L ｓ (hereinafter, "loss value L ｓ ") is calculated, and the loss value L ｓThe learning process S12 can be a process of training the conversion means so that the conversion unit 11 can learn by updating the parameters that the conversion unit 11 refers to in order to execute the conversion process S11.

[0030] The learning process S12 trains the conversion process S11 by referring to the distribution of the first transformed feature quantity and the distribution of the second transformed feature quantity in addition to the inference result. The learning process S12 trains the conversion process S11 so that the difference between the distribution of the first transformed feature quantity and the distribution of the second transformed feature quantity becomes small. Specifically, the domain loss λL obtained by referring to the distribution of the first transformed feature quantity and the distribution of the second transformed feature quantity is ｄ The learning process S12 performs learning based on the value of the loss function L (loss function value VLF) shown below.

[0031] L=L s +λL d Domain loss λL ｄ can be the distance between the distributions. ｄ The calculation method of is not particularly limited, and can be calculated using, for example, Maximum Mean Discrepancy (MMD) and a covariance matrix. In addition, in the common feature space, the domain loss λL is calculated by adversarial learning that identifies whether the feature originates from the first data or the second data. ｄ For example, the domain loss λL ｄ can be the negative loss function of the discriminative parameters in adversarial learning.

[0032] As described above, the learning method S1 according to this exemplary embodiment employs a configuration in which the conversion unit 11 used for the conversion process S11 is trained by referring to the inference results of inference performed by referring to the feature amounts converted by the conversion process S11. Therefore, the learning method S1 according to this exemplary embodiment can provide a technology that can further improve inference accuracy while referring to the converted feature amounts.

[0033] (Specific example of learning device 1) The learning device 1 according to this exemplary embodiment can further include an inference unit. The inference unit derives an inference result based on the feature converted by the conversion means. The learning device 1 can cause the conversion means 11 and the inference means to train simultaneously based on a common loss function. This simplifies the learning process, enabling efficient learning. In the learning device 1, the series of processes including the conversion process S11 and the learning process S12 can be repeated multiple times.

[0034] (Specific Example of Conversion Processing S11) As the conversion processing S11 according to this exemplary embodiment, for example, any linear conversion or conversion of feature quantities by a neural network can be performed.

[0035] (Specific Example of Learning Process S12) The learning process S12 can further refer to a loss value during conversion in the conversion process S11. The conversion loss is a loss caused by the conversion process S11, and can be calculated, for example, from the difference between the distribution of the converted feature quantities and a predetermined arbitrary distribution.

[0036] <Application Examples> The learning device 1 can be used, for example, in the technical field of heterogeneous domain adaptation. Domain adaptation is a technology that improves target learning and solves the aforementioned problems by transferring knowledge from fully labeled data (source data) to learning on related target data (target data) from a different domain. Heterogeneous domain adaptation refers to domain adaptation when the feature spaces of the first data and the second data are different. The first data and the second data may have features related to common items; in this case, the domain adaptation is called hybrid domain adaptation.

[0037]

[0033] A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are denoted by the same reference numerals, and their description will be omitted as appropriate.

[0038] <Configuration of Learning System 100> Next, the configuration of learning system 100 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of learning system 100. As shown in Fig. 3, a first learning device 1A and a second learning device 1B are connected to an integration device 3. The first learning device 1A and the second learning device 1B are independent devices.

[0039] "First data" and "second data" are data in which any item is associated with information related to that item. The first data is input to the first learning device 1A and processed by the first learning device 1A. The second data is input to the second learning device 1B and processed by the second learning device 1B.

[0040] (First Learning Device 1A) The first learning device 1A includes a first conversion unit 11A, a first inference unit 13A, and a first learning unit 12A.

[0041] The first conversion unit 11A generates a first converted feature from the feature obtained from the input first data.

[0042] The first inference unit 13A performs inference by referring to the first transformed feature amount. The inference is performed based on arbitrary parameters. The first inference unit 13A updates the parameters it refers to with new parameters created by the integration device 3, which will be described later.

[0043] The first learning unit 12A refers to the inference result by the first inference means and causes the first conversion unit 11A to learn.

[0044] (Second Learning Device 1B) The second learning device 1B includes a second conversion unit 11B, a second inference unit 13B, and a second learning unit 12B.

[0045] The second conversion unit 11B generates a second converted feature from the feature obtained from the input second data.

[0046] The second inference unit 13B performs inference by referring to the second converted feature amount. The inference is performed based on arbitrary parameters. The second inference unit 13B updates the parameters it refers to with new parameters created by the integration device 3, which will be described later.

[0047] The second learning unit 12B refers to the inference result by the second inference means and causes the second conversion unit 11B to learn.

[0048] (Integration Device 3) The integration device 3 generates new parameters based on the parameters referred to by the first inference means 13A and the second inference means 13B.

[0049] As described above, the learning system 100 according to this exemplary embodiment is configured to generate new parameters based on parameters referenced by the first inference unit 13A and the second inference unit 13B. This allows the learning system 100 to integrate and learn from the first and second data even when the data are owned by different people. Because the first and second data belong to different domains, the relationship between the owners of the first and second data is not particularly limited and may be the same or different industries. In particular, the learning system 100 according to this exemplary embodiment can learn data belonging to different domains, making learning suitable even when the owners of the data are in different industries. (Specific Example of the Learning System 100) The learning system 100 according to this exemplary embodiment can typically be used for federated learning between the first and second learning devices 1A and 1B, which are independent of each other. Each learning device independently trains its conversion unit and inference unit. Each learning device can begin learning based on the initial global model created by the integration device 3. Each learning device transmits the parameters of the local model for inference processing obtained as a result of learning to the integrating device 3.

[0050] The integrating device 3 refers to the parameters of each local model and calculates more appropriate parameters. The calculation method is not particularly limited, but for example, the average value of the parameters obtained by the first learning device 1A and the parameters obtained by the second learning device 1B can be used as the new parameters. In federated learning, there is no need to transmit the first data and the second data themselves to outside the first learning device 1A and the second learning device 1B, which has the effect of ensuring the confidentiality of each data.

[0051] (Specific Examples of the First Learning Unit 12A and the Second Learning Unit 12B) The learning of the first learning unit 12A and the second learning unit 12B can be performed both before and after the parameters are updated to new parameters by the integrating device 3. For example, the learning by each learning unit can be performed independently of each other before the parameters referenced by each inference unit are updated to new parameters by the integrating device 3. Furthermore, for example, the learning by each learning unit can be performed independently of each other after the parameters referenced by each inference unit are updated to new parameters by the integrating device 3.

[0052] Furthermore, when the learning unit performs adversarial training, the discrimination parameters referenced by the learning unit can be updated. In this case, the discrimination parameters are integrated by the integration device 3. In combination with federated learning, each device learns, which has the effect of obtaining a conversion model and an inference model that improve inference accuracy.

[0053]

[0033] A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are denoted by the same reference numerals, and their description will be omitted as appropriate.

[0054] <Overview of the Learning Device 1C> In this exemplary embodiment, the "input data" refers to data in which any item is associated with information related to that item. Furthermore, the "input data" in this exemplary embodiment includes first data belonging to a first domain and second data belonging to a second domain.

[0055] <Configuration of Learning Device 1C> The configuration of the learning device 1C according to this exemplary embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the learning device 1. As shown in Fig. 4, the learning device 1C includes a control unit 10C, a storage unit 20C, a communication unit 30, and an input / output unit 40.

[0056] (Conversion unit 11C) The conversion unit 11C includes a first conversion unit 111C and a second conversion unit 112C. The first conversion unit 111C generates a first converted feature f1. The first converted feature f1 is generated from a feature obtained from the first data d1. The second conversion unit 112C generates a second converted feature f2. The second converted feature f2 is generated from a feature obtained from the second data d2.

[0057] (Learning unit 12C) The learning unit 12C trains the conversion unit 11C by referring to the inference result and the distribution of the first converted feature value f1 and the distribution of the second converted feature value f2. The learning unit 12C trains the conversion unit 11C so that the difference between the distribution of the first converted feature value f1 and the distribution of the second converted feature value f2 becomes small.

[0058] (Inference Unit 13C) The inference unit 13C includes a first inference unit 131C and a second inference unit 132C. The first inference unit 131C performs inference by referring to the first transformed feature quantity f1. The second inference unit 132C performs inference by referring to the second transformed feature quantity f2. Here, the first inference unit 131C and the second inference unit 132C perform inference based on the same parameters. For this reason, while FIG. 4 illustrates a configuration in which the inference unit 13C includes the first inference unit 131C and the second inference unit 132C, this is not limiting, and the inference unit 13C may perform inference on both the first transformed feature quantity f1 and the second transformed feature quantity f2.

[0059] (Storage Unit 20C) The storage unit 20C stores various types of information referenced by the control unit 10C and various types of information derived by the control unit 10C. As an example, the storage unit 20C stores the following: First data d1 Second data d2 First transformed feature value f1 Second transformed feature value f2 Loss function value VLF First inference result R1 Second inference result R2

[0060] (Communication Unit 30) The communication unit 30 communicates with devices external to the learning device 1C via a communication line. While the specific configuration of the communication line does not limit this exemplary embodiment, examples of the communication line include a wireless local area network (LAN), a wired LAN, a wide area network (WAN), a public line network, a mobile data communication network, or a combination thereof. The communication unit 30 transmits data provided by the control unit 10C to other devices, and provides data received from other devices to the control unit 10C.

[0061] (Input / Output Unit 40) Input / output devices such as a keyboard, mouse, display, printer, and touch panel are connected to the input / output unit 40. The input / output unit 40 accepts various types of information input to the learning device 1C from the connected input devices. The input / output unit 40 also outputs various types of information to connected output devices under the control of the control unit 10C. An example of the input / output unit 40 is an interface such as a USB (Universal Serial Bus).

[0062] As described above, the learning device 1C according to this exemplary embodiment employs a configuration for converting features obtained from data belonging to different domains. Therefore, in addition to the effects of the learning device 1 according to exemplary embodiment 1, the learning device 1C according to this exemplary embodiment can map features obtained from data belonging to domains with different feature spaces into a common feature space.

[0063] Furthermore, the learning device 1C according to this exemplary embodiment is configured to train the conversion unit 11C so that the difference between the distribution of the first transformed feature f1 and the distribution of the second transformed feature f2 is small. This makes it possible to generate an inference model capable of inferring data belonging to domains with different feature spaces. In particular, when one of the first data and the second data is unlabeled data, it is possible to learn an inference model capable of inferring data that is also unlabeled.

[0064] <Flow of Learning Method S1C> The flow of the learning method S1C according to the present exemplary embodiment 3 will be described with reference to Fig. 5. Fig. 5 is a flow chart showing the flow of the learning method S1C.

[0065] The first conversion process S111 generates a first converted feature f1. The first converted feature f1 is generated from a feature obtained from the first data d1. The second conversion process S112 generates a second converted feature f2. The second converted feature f2 is generated from a feature obtained from the second data d2. The order of the first conversion process S111 and the second conversion process S112 is not particularly limited as long as they are performed before each inference process. For example, the first conversion process S111 and the second conversion process S112 can be performed simultaneously.

[0066] The first inference process S131 performs inference by referring to the first transformed feature value f1. The second inference process S132 performs inference by referring to the second transformed feature value f2. The order of the first inference process S131 and the second inference process S132 is not particularly limited as long as they are performed after each transformation process and before the learning process. For example, the first inference process S131 and the second inference process S132 can be performed simultaneously.

[0067] The learning process S12C trains the conversion unit 11 that executes the conversion process S11C by referring to the inference result and the distribution of the first converted feature value f1 and the distribution of the second converted feature value f2. The learning process S12C trains the conversion unit 11 that executes the conversion process S11C so that the difference between the distribution of the first converted feature value f1 and the distribution of the second converted feature value f2 becomes small.

[0068] As described above, the training method S1C according to this exemplary embodiment employs a process for converting features obtained from data belonging to different domains. Therefore, in addition to the effects of the training method S1 according to the first exemplary embodiment, the training method S1C according to this exemplary embodiment can map features obtained from data belonging to different domains into a common feature space.

[0069] Furthermore, in the learning method S1C according to this exemplary embodiment, a process is employed in which the transformation unit 11C is trained so as to minimize the difference between the distribution of the first transformed feature value f1 and the distribution of the second transformed feature value f2. This allows for the generation of an inference model capable of inferring data belonging to domains with different feature space. In particular, when one of the first data and the second data is unlabeled data, an inference model capable of inferring data without labels can be trained.

[0070] (Specific Example of Learning Method S1C) As a specific example of learning method S1C, a method of learning based on common features and specific features obtained from the features obtained from the first data d1 and the second data d2, respectively, will be described. The common features are features related to common items that are common to both the first data d1 and the second data d2. The specific features are features related to unique items that are not common to both the first data d1 and the second data d2.

[0071] (Specific Example of Conversion Unit 11C) The first data d1 and the second data d2 contain information about unique items that are not common to each other and belong to different domains. The conversion process executed by the conversion unit 11C maps the features obtained from the first data d1 and the second data d2 into a common feature space. Below, specific examples of the first conversion process S111C and the second conversion process S112C executed by the first conversion unit 111C and the second conversion unit 112C included in the conversion unit 11C will be described.

[0072] An example of the conversion process in the conversion unit 11C includes a first conversion process S111C and a second conversion process S112C in which information relating to each unique item of the first data d1 and the second data d2 is mutually referenced.

[0073] The first conversion unit 111C executes a first conversion process S111C to generate a first converted feature f1 from a feature and a pseudo feature obtained from the first data. The pseudo feature is a pseudo feature related to a unique item of the second data d2 calculated from a common feature of the first data d1. The calculation of the pseudo feature related to the unique item of the second data d2 is performed by referring to a model that estimates a unique feature of the second data d2 based on the common feature.

[0074] The second conversion unit 112C executes a second conversion process S112C to generate second converted features f2 from the features and pseudo features obtained from the second data d2. The pseudo features are pseudo features related to the unique items of the first data d1 calculated from the common features of the second data d2. The calculation of the pseudo features related to the unique items of the first data d1 is performed by referring to a model that estimates the unique features of the first data d1 based on the common features.

[0075] As described above, the first conversion unit 111C and the second conversion unit 112C can complement each other's pseudo-features for unique items included in only one of the first data d1 and the second data d2 based on the common features obtained from each data. This makes it possible to map the features converted by the first conversion unit 111C and the second conversion unit 112C into a common feature space.

[0076] Furthermore, as an example, both or one of the first conversion process S111C and the second conversion process S112C may be a linear conversion or a conversion using a neural network of each feature obtained from the first data d1 and the second data d2. For example, the first conversion process S111C may be an identity conversion, and the second conversion process S112C may be a linear conversion or a conversion using a neural network, and the second conversion process S112C may map the feature obtained from the second data d2 into the feature space of the feature obtained from the first data d1.

[0077] Furthermore, as an example, the first conversion process S11C and the second conversion process S12C can convert the features obtained from each data using any method and assign common features obtained from each data to the converted features.

[0078] (Specific Example of Learning Process S12C) The learning process S12C is a process of learning a loss value L ｓ In addition, the domain loss λL obtained by referring to the distribution of the first transformed feature f1 and the distribution of the second transformed feature f2 ｄ That is, the learning process S12 is a process of performing learning based on the value of the loss function L (loss function value VLF) shown below.

[0079] L=L s +λL d Domain loss λL ｄ can be the distance between the distributions. ｄThe calculation method of is not particularly limited, and can be calculated using, for example, Maximum Mean Discrepancy (MMD) and a covariance matrix. In addition, in the common feature space, the domain loss λL is calculated by adversarial learning that identifies whether the feature originates from the first data or the second data. ｄ For example, the domain loss λL ｄ can be the negative loss function of the discriminative parameters in adversarial learning.

[0080] (Modification of the Learning Device 1C) While Fig. 4 illustrates the learning device 1C processing two sets of data, namely, first data and second data, the present invention is not limited to this. For example, the learning device 1C can receive input data belonging to three or more domains. In this case, in the learning process S12C, the domain loss λLd is calculated based on the distribution of each feature converted from the feature obtained from the data belonging to the domain.

[0081] [Embodiment 4] A fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in embodiments 1 to 3 are designated by the same reference numerals, and their description will be omitted as appropriate.

[0082] <Overview of Learning Device 1D> According to this exemplary embodiment, "input data" refers to data in which any item is associated with information related to that item. According to this exemplary embodiment, "input data" includes first data and second data. The first data is labeled data that includes a correct answer label. The second data is unlabeled data that does not include an accurate label. The labeled data can be referred to as source data, and the unlabeled data can be referred to as target data.

[0083] In one example of the prior art, the features of the source data and the target data are complemented by each other to convert the features, thereby mapping the features of each data to a feature space, and then inference is performed using an inference means trained on the source data.

[0084] In this conventional technology, when there is little correlation between the common features and the unique features, the accuracy of inference using converted features may be low. Furthermore, because the conversion means and the inference means are trained separately, there is a risk that the conversion means may not be able to convert features into features that are easy to infer. Furthermore, because the inference means is not trained based on target data, the accuracy of inference for use with target data is insufficient. The learning device 1D according to this exemplary embodiment is configured to solve the problems of the conventional technology.

[0085] <Configuration of Learning Device 1D> The configuration of the learning device 1D according to this exemplary embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the configuration of the learning device 1D. As shown in Fig. 6, the learning device 1D includes a control unit 10D, a storage unit 20D, a communication unit 30, and an input / output unit 40. The control unit 10D includes a conversion unit 11D, an inference unit 13D, a learning unit 12D, and a pseudo label generation unit 14.

[0086] (Pseudo Label Generation Unit 14) The pseudo label generation unit 14 generates pseudo labels PL to be assigned to the second data d2. The pseudo labels PL are generated based on feature quantities obtained from the second data d2. Details of how the pseudo label generation unit 14 generates the pseudo labels PL will be described later.

[0087] (Learning unit 12D) The learning unit 12D trains the second transformation 112D and the inference unit 13D so that the difference between the inference result R2 by the inference unit 132 that references the second data d2 and the pseudo label PL of the second data d2 becomes small.

[0088] As described above, the learning device 1D according to this exemplary embodiment employs a configuration including a pseudo label generation unit 14. This configuration allows learning based on inference results even for conversion of unlabeled data to which no correct answer label has been assigned. Therefore, in addition to the effects of the learning device 1C according to the third exemplary embodiment, the learning device 1D according to this exemplary embodiment can convert features obtained from unlabeled data into features that are easy to infer.

[0089] Furthermore, the learning device 1D according to this exemplary embodiment can allow unlabeled data that has not been assigned a correct answer label to participate in the learning of the inference unit, thereby obtaining an inference model that can perform inference with high accuracy even on unlabeled data.

[0090] <Flow of Learning Method S1D> The flow of the learning method S1D according to the fourth exemplary embodiment will be described with reference to Fig. 7. Fig. 7 is a flow chart showing the flow of the learning method S1D.

[0091] (Pseudo label generation process S14) The pseudo label generation process S14 generates a pseudo label PL to be assigned to the second data d2. The pseudo label PL can be generated based on features obtained from the second data d2, for example, using a generative model obtained by a neural network.

[0092] (Learning process S12D) The learning process S12D trains the second transformation 112D and the inference unit 13D so that the difference between the inference result R2 by the inference unit 132 referencing the second data d2 and the pseudo label L of the second data d2 becomes small.

[0093] (Effects of Learning Method S1D) According to the learning method S1D configured as described above, similar to the effect achieved by the learning device 1D, it is possible to convert features obtained from unlabeled data into features that are easy to infer. Furthermore, according to the learning method S1D, unlabeled data that has not been assigned a correct answer label can also be included in the learning.

[0094] (Specific Example of Learning Method S1D) A specific example of learning method S1D, learning method S1D', is shown in Fig. 8. Learning method S1D' includes steps S15 and S16 in addition to the series of processes of pseudo label generation process S14, conversion process S11, learning process S12, and inference process S13.

[0095] In step S15, it is determined whether the learning process is the first time. If it is the first time, the process repeats the pseudo label generation process S14, the conversion process S11, the learning process S12, and the inference process S13. If it is the second or subsequent time, the process proceeds to step S16.

[0096] In step S16, it is determined whether the accuracy is higher than that of the model obtained as a result of the previous process. If a model with higher accuracy than that of the previous process is not obtained, the series of processes of pseudo label generation process S14, conversion process S11, learning process S12, and inference process S13 are performed again. If a model with higher accuracy than that of the previous process is obtained, the process ends. Here, in step S16 shown in FIG. 8, the criterion is that the accuracy is higher than that of the previous model, but this is not limited to this, and the process can also be ended if the accuracy is higher than an arbitrary threshold value.

[0097] (Specific Example of Pseudo Label Generation Process S14) The pseudo label generation process S14 generates pseudo labels to be assigned to the second data d2. The pseudo labels PL can be generated by inferring labels based on features obtained from the second data d2. One example of the inference process for generating the pseudo labels PL is to refer to a pseudo label inference model that infers the correct labels assigned to the first data based on common features obtained from the first data. The pseudo label inference model is obtained by learning using the first data d1. In other words, the pseudo labels PL can be generated from the common features obtained from the second data d2 by referring to the pseudo label inference model.

[0098] The pseudo label PL can be assigned as a soft label to the confidence vector itself output as a result of referencing the inference model, or, if the component with the highest value in the confidence vector output as a result of referencing the inference model is higher than an arbitrary threshold, the class to which that component belongs can be assigned as a label.

[0099] In the second or subsequent learning, the pseudo labels PL can be generated by the inference unit 13D that has been trained by the learning method S1D. In this case, the pseudo labels PL can be reassigned by the inference unit 13D that has been updated by the learning process S12D, and further learning can be performed by the learning method S1D.

[0100] (Specific example of learning process S12D) The learning process S12D updates the parameters of the model referred to by the conversion unit 11D and the inference unit 13D so as to reduce the difference between the inference result R2 based on the inference using the second data d2 and the pseudo label L of the second data d2. That is, the learning process S12D calculates the classification loss (hereinafter referred to as the "loss value L") between the inference result R2 and the pseudo label PL of the second data d2. ｔ ") is calculated, and the loss value L ｔ The process can be to minimize the following.

[0101] The learning process S12D calculates the loss value L ｓ and domain loss L ｄ In addition to the loss value L ｔ That is, the learning process S12D is a process of performing learning based on the value of the loss function L (loss function value VLF) shown below. The loss value Ls and the domain loss L ｄ The calculation method is as described above.

[0102] L=L t +L s +λL d

[0103] <Application Example> The learning device 1D can learn more effectively by, for example, transferring knowledge of labeled data (source data) from a different domain that has some common features to unlabeled data (target data) through heterogeneous domain adaptation. In particular, since the learning device 1D can be applied to unlabeled data, it can also be applied to unsupervised heterogeneous domain adaptation.

[0104] Fifth Exemplary Embodiment A fifth exemplary embodiment of the present invention will now be described in detail with reference to the drawings.

[0105] <Configuration of Inference Device 2> The configuration of the learning device 2 according to this exemplary embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of the inference device 2. As shown in Fig. 9, the inference device 2 includes an acquisition unit 21D, a conversion unit 22, and an inference unit 23.

[0106] (Acquisition Unit 21) The acquisition unit 21 is configured to acquire feature amounts from input data.

[0107] (Conversion unit 22) The conversion unit 22 is configured to generate a converted feature from a feature. The conversion unit 22 is trained by referring to an inference result by the inference unit 23, which refers to the feature after conversion by the conversion unit 22.

[0108] (Inference Unit 23) The inference unit 23 is configured to execute inference by referring to the converted feature amount.

[0109] (Effects of the inference device 2) As described above, the inference device 2 according to this exemplary embodiment employs a configuration including a trained conversion unit 22. This configuration enables conversion into feature quantities that are easy to infer. Therefore, the inference device 2 according to this exemplary embodiment can perform inference with high accuracy.

[0110] <Flow of inference method S2> The flow of the inference method S2 according to the fifth exemplary embodiment will be described with reference to Fig. 10. Fig. 10 is a flow chart showing the flow of the learning method S2.

[0111] (Acquisition Process S21) In the acquisition process S21, feature amounts are acquired from input data.

[0112] (Conversion Process S22) The conversion process S22 generates converted features from the features. The conversion process S22 is executed by the conversion unit 22 that has been trained by referring to the inference result of the inference process S23 that references the converted features.

[0113] (Inference Process S23) In the inference process S23, inference is performed by referring to the converted feature amount.

[0114] (Effects of the inference device 2) According to the inference method S2 configured as described above, similar to the effects achieved by the inference device 2, inference can be performed with high accuracy.

[0115] [Example 6] A sixth example embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in Example 2 are given the same reference numerals, and their description will be omitted as appropriate.

[0116] <Configuration of Learning System 100A> The configuration of learning system 100A according to this exemplary embodiment will be described with reference to FIG. 11. FIG. 11 is a block diagram showing the configuration of learning system 100A. As shown in FIG. 11, learning system 100A includes a first learning device 1E, a second learning device 1F, and an integrated device 3A. Also, as shown in FIG. 11, learning device 1A, learning device 1B, and integrated device 3A are configured to be able to communicate with each other via network N. Here, the specific configuration of network N does not limit this exemplary embodiment, but examples include a wireless local area network (LAN), a wired LAN, a wide area network (WAN), a public line network, a mobile data communication network, or a combination of these networks.

[0117] (First Learning Device 1E) The configuration of a learning device 1E according to this exemplary embodiment will be described with reference to FIG.

[0118] 11, the learning device 1E includes a control unit 10E and a communication unit 30E. Although not shown, the learning device 1E may also include a storage unit.

[0119] The first learning device 1E is configured to convert feature quantities obtained from input first data, perform inference based on the converted feature quantities, and perform learning by referring to the inference results. The first conversion unit, first inference unit, and first learning unit included in the first learning device 1E can have the same configurations as the conversion unit, inference unit, and learning unit included in the above-mentioned learning device, respectively.

[0120] The parameter update unit 15E is configured to update the parameters referred to by the inference unit 13E to the parameters generated by the integration device 3A.

[0121] The communication unit 30E communicates with devices external to the first learning device 1E. As an example, the communication unit 30E communicates with the communication unit 30A of the integrating device 3A. The communication unit 30E transmits data supplied from the control unit 10E to the communication unit 30A of the integrating device 3A, and supplies data received from the communication unit 30A of the integrating device 3A to the control unit 10E.

[0122] The communication unit 30E can transmit parameters referenced by the inference unit 13E to the integration device 3A after learning by the learning device 1E has been performed any number of times, or after the learning has been repeated until the accuracy of the obtained inference model reaches any threshold value.

[0123] (Second learning device 1F) The second learning device 1F is configured to convert feature quantities obtained from input second data, perform inference based on the converted feature quantities, and perform learning by referring to the inference results. Each unit of the second learning device 1F has a configuration having the same function as each unit of the first learning device 1E.

[0124] (Integration Device 3A) The configuration of the integration device 3A according to this exemplary embodiment will be described with reference to FIG.

[0125] As shown in FIG. 11, the integrating device 3A includes a communication unit 30A and a control unit 31.

[0126] The communication unit 30A communicates with devices external to the integrating device 3A. As an example, the communication unit 30A communicates with the first learning device 1E and the second learning device 1F. The communication unit 30A transmits data supplied from the control unit 31 to the communication unit 30A of the integrating device 3A, and supplies data received from the communication units 30E and 30F of the first learning device 1E and the second learning device 1F to the control unit 31.

[0127] The control unit 31 includes a parameter generation unit 311. The parameter generation unit 311 is configured to generate new parameters based on parameters referenced by the inference unit 13E of the first learning device 1E and the inference unit 13F of the second learning device 1F.

[0128] <Specific Example of System 100A> In one example of system 100A, the first learning device 1E and the second learning device 1F are devices used by a data holder who performs inference, and the integrating device 3A is a server. With this configuration, the first learning device 1E and the second learning device 1F can perform federated learning only by communicating with the integrating device 3A (server) without directly communicating with each other.

[0129] (Other Configurations) When the input data is unlabeled data, the first learning device 1E and the second learning device 1F can be provided with a pseudo-label generation unit. As an example, the pseudo-label generation model can refer to a pseudo-label generation model based on common features provided by another learning device included in the learning system 100A. Also, as an example, the pseudo-label generation model can be provided by the integration device 3A.

[0130] The first learning unit 12E and the second learning unit 12F can train the first conversion unit 11E and the second conversion unit 11F, respectively, so as to reduce the difference between the distribution of the first converted feature quantity converted by the first conversion unit 11E and the distribution of the second converted feature quantity converted by the second conversion unit 11F. The integration device 3A can generate new parameters referenced by each conversion unit so as to reduce the difference between the distribution of the first converted feature quantity and the distribution of the second converted feature quantity. That is, the integration device 3A can calculate domain loss and generate new parameters referenced by each conversion unit. For example, the first learning device 1E and the second learning device 1F can transmit covariance matrices to the integration device 3A. Calculating the domain loss based on the covariance matrix allows the data owner to calculate the domain loss without transmitting raw data externally. Training each conversion unit based on the domain loss can realize a learning unit that can convert features into ones that are easy to infer.

[0131] Although the learning system 100A includes a first learning device 1E and a second learning device 1F, the learning system 100A is not limited to this and may include three or more learning devices. In this case, the parameter generation unit 311 of the integration device 3A can generate new parameters based on the parameters transmitted from all the learning devices.

[0132] [Application Example] The learning system 100A can be used for federated learning among multiple data holders. The multiple data holders may be, for example, companies in different industries. By using such a learning system 100A, it is possible to create an inference model with high inference accuracy that references data belonging to many domains.

[0133] [Example of Software Implementation] Some or all of the functions of the learning devices 1, 1A, 1B, 1C, 1D, 1E, and 1F may be implemented by hardware such as an integrated circuit (IC chip), or by software.

[0134] In the latter case, the learning devices 1, 1A, 1B, 1C, 1D, 1E, and 1F are realized, for example, by a computer that executes instructions in a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 12. Computer C includes at least one processor C1 and at least one memory C2. Memory C2 stores a program P for operating computer C as learning devices 1, 1A, 1B, 1C, 1D, 1E, and 1F. In computer C, processor C1 reads and executes program P from memory C2, thereby realizing each function of learning devices 1, 1A, 1B, 1C, 1D, 1E, and 1F.

[0135] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0136] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0137] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communications network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0138] [Appendix 1] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0139] [Additional Note 2] Part or all of the above-described embodiment can also be described as follows: However, the present invention is not limited to the following described aspects.

[0140] (Supplementary Note 1) A learning device comprising: a conversion means for generating converted features from features obtained from input data including first data belonging to a first domain and second data belonging to a second domain; and a learning means for training the conversion means by referring to an inference result by an inference means that references the features converted by the conversion means, wherein the conversion means comprises: a first conversion means for generating first converted features from features obtained from the first data; and a second conversion means for generating second converted features from features obtained from the second data, and the learning means trains the conversion means so that a difference between the distribution of the first converted features and the distribution of the second converted features becomes small.

[0141] (Supplementary Note 2) The learning device according to Supplementary Note 1, wherein the learning means trains the conversion means so as to reduce a difference between an inference result by the inference means that refers to the converted feature and a correct label of the inference.

[0142] (Supplementary Note 3) The learning device according to Supplementary Note 1 or 2, further comprising the inference means.

[0143] (Supplementary Note 4) The learning device according to Supplementary Note 3, wherein the first data is labeled data including a correct label, and the second data is unlabeled data not including a correct label, and the learning device further comprises a pseudo label generation means for generating pseudo labels to be assigned to the second data, and the learning means trains the conversion means and the inference means so as to reduce a difference between an inference result by the inference means referring to the first data and a correct label of the first data, and a difference between an inference result by the inference means referring to the second data and a pseudo label of the second data.

[0144] (Supplementary Note 5) The learning device according to any one of Supplementary Notes 1 to 4, wherein the features obtained from the first data and the features obtained from the second data each include common features related to common items that are common to both data and unique features related to unique items that are not common to both data; the first conversion means refers to a model that estimates the unique features of the second data based on the common features, calculates pseudo features related to the unique items of the second data from the common features of the first data, and generates first converted features from the features obtained from the first data and the pseudo features; and the second conversion means refers to a model that estimates the unique features of the first data based on the common features, calculates pseudo features related to the unique items of the first data from the common features of the second data, and generates second converted features from the features obtained from the second data and the pseudo features.

[0145] (Supplementary Note 6) An inference device comprising: an acquisition means for acquiring features from input data; a conversion means for generating features converted from the features, the conversion means being trained by referring to an inference result by an inference means that references the features converted by the conversion means; and an inference means for performing inference by referring to the converted features.

[0146] (Supplementary Note 7) A learning system comprising: a first learning device comprising: first conversion means for generating first converted features from features obtained from input first data; first inference means for making inference by referring to the first converted features; and first learning means for training the first conversion means by referring to a result of the inference by the first inference means; a second learning device comprising: second conversion means for generating second converted features from features obtained from input second data; second inference means for making inference by referring to the second converted features; and second learning means for training the conversion means by referring to a result of the inference by the second inference means; and an integrating device for generating new parameters based on parameters referred to by the first inference means and the second inference means, respectively;

[0147] (Supplementary Note 8) The learning system described in Supplementary Note 7, wherein the first learning means and the second learning means respectively train the first conversion means and the second conversion means so that a difference between a distribution of the first converted feature and a distribution of the second converted feature becomes small.

[0148] (Supplementary Note 9) A learning method including: a transformation step of generating transformed features from features obtained from input data including first data belonging to a first domain and second data belonging to a second domain; and a learning step of training a transformation means that executes the transformation process by referring to an inference result based on the transformed features, wherein the transformation process comprises: a first transformation process of generating first transformed features from features obtained from the first data; and a second transformation process of generating second transformed features from features obtained from the second data, and the learning process trains the transformation means that executes the transformation process so that a difference between a distribution of the first transformed features and a distribution of the second transformed features becomes small.

[0149] (Supplementary Note 10) A non-transitory recording medium storing a program for causing a computer to function as a learning device, the program causing the computer to execute: a conversion process for generating converted features from features obtained from input data including first data belonging to a first domain and second data belonging to a second domain; and a learning process for training conversion means that executes the generation of the converted features by referring to an inference result obtained by inference that references the converted features, the conversion process including: a first conversion process for generating first converted features from features obtained from the first data; and a second conversion process for generating second converted features from features obtained from the second data, and the learning process trains the conversion means that executes the conversion process so as to reduce a difference between a distribution of the first converted features and a distribution of the second converted features.

[0150] The learning device and learning system described above may further include a memory that stores a program for causing the processor to execute the conversion process, the learning process, and the inference process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium.

[0151] DESCRIPTION OF SYMBOLS 1, 1A, 1B, 1C, 1D, 1E, 1F... Learning device (first learning device, second learning device) 11, 11A, 11B, 11C, 11D, 11E, 11F... Conversion unit (first conversion unit, second conversion unit) 111C, 111D... First conversion unit 112C, 112D... Second conversion unit 12, 12A, 12B, 12C, 12D, 12E, 12F... Learning unit (first learning unit, second learning unit) 13, 13A, 13B, 13C, 13D, 13E, 13F... Inference unit (first inference unit, second inference unit) 131... First inference unit 132... Second inference unit 14... Pseudo label generation unit 2... Inference device 21... Acquisition unit 22: Conversion unit 23: Inference unit 3, 3A: Integration device

Claims

1. A learning device comprising: a conversion means for generating converted features from features obtained from input data including first data belonging to a first domain and second data belonging to a second domain; and a learning means for training the conversion means by referring to an inference result by an inference means which references the features converted by the conversion means, wherein the conversion means comprises: a first conversion means for generating first converted features from features obtained from the first data; and a second conversion means for generating second converted features from features obtained from the second data, and the learning means trains the conversion means so that the difference between the distribution of the first converted features and the distribution of the second converted features becomes small.

2. The learning device according to claim 1, wherein the learning means trains the conversion means so as to reduce the difference between an inference result by the inference means that references the converted feature amount and a correct label of the inference.

3. The learning device according to claim 1 or 2, further comprising the inference means.

4. The learning device according to claim 3, wherein the first data is labeled data including an accurate label, and the second data is unlabeled data not including an accurate label, the learning device further comprises a pseudo label generation means for generating a pseudo label to be assigned to the second data, and the learning means trains the conversion means and the inference means so as to reduce a difference between an inference result by the inference means referring to the first data and a correct label of the first data, and a difference between an inference result by the inference means referring to the second data and a pseudo label of the second data.

5. The learning device according to any one of claims 1 to 4, wherein the features obtained from the first data and the features obtained from the second data each include a common feature related to a common item common to both data and a unique feature related to a unique item not common to both data, wherein the first conversion means refers to a model that estimates the unique features of the second data based on the common features, calculates pseudo features related to the unique items of the second data from the common features of the first data, and generates first converted features from the features obtained from the first data and the pseudo features, and wherein the second conversion means refers to a model that estimates the unique features of the first data based on the common features, calculates pseudo features related to the unique items of the first data from the common features of the second data, and generates second converted features from the features obtained from the second data and the pseudo features.

6. An inference device comprising: an acquisition means for acquiring features from input data; a conversion means for generating features converted from the features, the conversion means being trained by referring to an inference result by an inference means which references the features converted by the conversion means; and an inference means for performing inference by referring to the features converted by the conversion means.

7. A learning system comprising: a first learning device comprising: a first conversion means for generating a first converted feature from a feature obtained from input first data; a first inference means for making an inference by referring to the first converted feature; and a first learning means for training the first conversion means by referring to a result of the inference by the first inference means; a second learning device comprising: a second conversion means for generating a second converted feature from a feature obtained from input second data; a second inference means for making an inference by referring to the second converted feature; and a second learning means for training the conversion means by referring to a result of the inference by the second inference means; and an integration device for generating new parameters based on each parameter referred to by the first inference means and the second inference means, each of which updates each of the parameters to the new parameters created by the integration device.

8. The learning system described in claim 7, wherein the first learning means and the second learning means each train the first conversion means and the second conversion means so that the difference between the distribution of the first converted feature and the distribution of the second converted feature becomes small.

9. A learning method comprising: a conversion process for generating converted features from features obtained from input data including first data belonging to a first domain and second data belonging to a second domain; and a learning process for training a conversion means that executes the conversion process by referring to an inference result based on the converted features, wherein the conversion process comprises: a first conversion process for generating first converted features from features obtained from the first data; and a second conversion process for generating second converted features from features obtained from the second data, and the learning process trains the conversion means that executes the conversion process so that a difference between a distribution of the first converted features and a distribution of the second converted features becomes small.

10. A non-transitory recording medium storing a program for causing a computer to function as a learning device, the program causing the computer to execute a conversion process for generating converted features from features obtained from input data including first data belonging to a first domain and second data belonging to a second domain, and a learning process for training a conversion means that executes the generation of the converted features by referring to an inference result based on inference referencing the converted features, the conversion process including a first conversion process for generating first converted features from features obtained from the first data, and a second conversion process for generating second converted features from features obtained from the second data, and the learning process for training the conversion means that executes the conversion process so as to reduce a difference between a distribution of the first converted features and a distribution of the second converted features.