Learning device

Abstract

A learning device 500 includes a common model learning unit 521 that performs learning of a common model used by other learning devices as well, using, out of features that the own device has, common features that are features held in common with the other learning devices, and a unique model learning unit 522 that performs learning of a unique model that is a model unique to the own device, using, out of features that the own device has, unique features that are features not held in common with the other learning devices, on the basis of the common model learned by the common model learning unit 521.

Description

Technical Field

[0001] The present invention relates to a learning device, a learning method, a recording medium, and a learning system.

Background Art

[0002] Techniques used when performing learning using learning data are known.

[0003] For example, Patent Document 1 describes a learning device including a data generator and an additional learner that performs additional learning of a learned model using additional training data generated by the data generator. Note that, as described in Patent Document 1, additional learning may also be referred to as continuous learning.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] There is a technique called federated learning in which multiple clients cooperate to train a machine learning model without directly exchanging the feature quantities, which are learning data, that each client has. In such federated learning, in the learning called horizontal federated learning, learning is performed using the feature quantities that each client commonly has. Therefore, when performing horizontal federated learning, there has been a problem that feature quantities that are not common among clients among the feature quantities held by the clients cannot be utilized.

[0006] Therefore, an object of the present invention is to provide a learning device, a learning method, a recording medium, and a learning system capable of solving the problem that there are feature quantities that cannot be utilized when performing horizontal federated learning.

Means for Solving the Problems

[0007] To achieve such an object, a learning device according to one embodiment of the present disclosure includes a common model learning unit that learns a common model used by other learning devices by using a common feature amount that is a feature amount common to other learning devices among the feature amounts of the own device; a unique model learning unit that learns a unique model that is a model unique to the own device by using a unique feature amount that is a feature amount not common to other learning devices among the feature amounts of the own device, based on the common model learned by the common model learning unit; and has the following configuration.

[0008] In addition, a learning method according to another embodiment of the present disclosure is configured such that an information processing device learns a common model used by other learning devices by using a common feature amount that is a feature amount common to other learning devices among the feature amounts of the own device, and learns a unique model that is a model unique to the own device by using a unique feature amount that is a feature amount not common to other learning devices among the feature amounts of the own device, based on the learned common model. has the following configuration.

[0009] In addition, a recording medium according to another embodiment of the present disclosure is a computer-readable recording medium that records a program for causing an information processing device to learn a common model used by other learning devices by using a common feature amount that is a feature amount common to other learning devices among the feature amounts of the own device, and learn a unique model that is a model unique to the own device by using a unique feature amount that is a feature amount not common to other learning devices among the feature amounts of the own device, based on the learned common model.

[0010] In addition, a learning system according to another embodiment of the present disclosure A learning device includes a common model learning unit that learns a common model used by other learning devices using common feature amounts that are feature amounts common to other learning devices among the feature amounts possessed by the own device, and a unique model learning unit that learns a unique model that is a model unique to the own device using unique feature amounts that are feature amounts not common to other learning devices among the feature amounts possessed by the own device based on the common model learned by the common model learning unit. A server device includes an integration unit that integrates the common models learned by each learning device by communicating with a plurality of learning devices. It has such a configuration.

Advantages of the Invention

[0011] According to each of the above-described configurations, it is possible to provide a learning device, a learning method, a recording medium, and a learning system that can improve the inference accuracy by using feature amounts that cannot be utilized during horizontal collaborative learning during learning or inference.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0013] [First Embodiment] The first embodiment of the present disclosure will be described with reference to FIGS. 1 to 9. FIG. 1 is a diagram for explaining the outline of the present invention. FIG. 2 is a diagram showing an overall configuration example of the learning system 100. FIG. 3 is a block diagram showing an example of the configuration of the learning device 200. FIG. 4 is a diagram showing an example of the learning data 221. FIG. 5 is a diagram showing an example of the combination of the common model and the specific model which are learning models. FIG. 6 is a diagram showing an example of the configuration of the server device 300. FIGS. 7 and 8 are flowcharts showing an operation example of the learning device 200. FIG. 9 is a flowchart showing another operation example of the learning device 200.

[0014] In the first embodiment of the present disclosure, a learning system 100 having a learning device 200 capable of performing horizontal federated learning using a common feature amount which is a feature amount commonly possessed by a plurality of clients, and also performing learning, inference, etc. that utilize a specific feature amount which is a feature amount unique to the own device will be described. For example, in the case illustrated in FIG. 1, the learning device 200 has the feature amount of the client 1, and the other learning device 400 within the learning system 100 has the feature amount of the client 2. In such a configuration, for example, the learning device 200 performs horizontal federated learning using the feature amount of the attribute C which is the common feature amount among the feature amounts of the attributes A, B, and C which are the feature amounts possessed by the own device.

[0015] In addition, the learning device 200 performs learning of the proprietary model using at least the proprietary feature amounts. For example, in the case of FIG. 1, the learning device 200 has the feature amounts of attribute A and attribute B, while the other learning device 400 does not. Therefore, the learning device 200 performs learning of the proprietary model using at least the feature amounts of attribute A and attribute B, which are the proprietary feature amounts. Also, in the case of the present embodiment, when performing learning of the proprietary model, the learning device 200 performs learning of the proprietary model so as not to forget the results learned by horizontal federated learning. For example, the learning device 200 applies a technique of Continual Learning, such as a method of fixing the model parameters learned for task 1 and inputting the output of the intermediate layer of task 1 to each layer to learn the model for task 2, to perform learning of the proprietary model so as not to forget the results of horizontal federated learning. Thereby, the learning device 200 creates a proprietary model that inherits the knowledge of the common model learned by horizontal federated learning.

[0016] FIG. 2 shows a configuration example of the entire learning system 100. Referring to FIG. 2, the learning system 100 includes a learning device 200, a server device 300, and at least one or more other learning devices 400. As shown in FIG. 2, the learning device 200 and the server device 300 are connected so as to be able to communicate with each other via a network or the like. Also, the server device 300 and the other learning devices 400 are connected so as to be able to communicate with each other via a network or the like.

[0017] The learning device 200 is an information processing device that generates a common model in cooperation with the other learning devices 400 by horizontal federated learning using common feature amounts, and generates a proprietary model by learning using at least proprietary feature amounts. FIG. 3 shows a configuration example of the learning device 200. Referring to FIG. 3, the learning device 200 mainly includes, for example, a communication I / F unit 210, a storage unit 220, and an arithmetic processing unit 230.

[0018] Note that in FIG. 3, the case of realizing the function as the learning device 200 using one information processing device is illustrated. However, the learning device 200 may be realized using a plurality of information processing devices, for example, being realized on the cloud. Further, the learning device 200 may have a configuration other than those exemplified above, such as an operation input unit such as a keyboard and a mouse, and a screen display unit.

[0019] The communication I / F unit 210 is composed of a data communication circuit and the like. The communication I / F unit 210 performs data communication with an external device such as a server device 300 connected via a communication line.

[0020] The storage unit 220 is a storage device such as a hard disk or a memory. The storage unit 220 stores processing information and a program 225 necessary for various processes in the arithmetic processing unit 230. The program 225 is read into and executed by the arithmetic processing unit 230 to realize various processing units. The program 225 is pre-read from an external device or a recording medium via a data input / output function such as the communication I / F unit 210 and stored in the storage unit 220. The main information stored in the storage unit 220 includes, for example, learning data 221, common model information 222, coupling parameter information 223, and unique model information 224.

[0021] The learning data 221 includes feature amounts such as common feature amounts and unique feature amounts, which are data used when learning a common model or a unique model. As an example, the learning data 221 is data in a table format or the like, but may be a feature amount other than those exemplified. For example, the learning data 221 is pre-acquired from an external device or the like via the communication I / F unit 210 or the like, or is pre-input using an operation input unit such as a keyboard and a mouse included in the learning device 200, and is stored in the storage unit 220.

[0022] FIG. 4 shows an example of the learning data 221. Referring to FIG. 4, the learning data 221 includes a common feature amount that is a feature amount common to the other learning device 400, and a unique feature amount that is a feature amount unique to the own device and not possessed by the other learning device 400. For example, in the case illustrated in FIG. 4, the learning data 221 includes the feature amounts of attribute A and attribute B that are unique feature amounts, and the feature amount of attribute C that is a shared feature amount.

[0023] The common model information 222 includes a model obtained by performing machine learning processing using the common feature amount. For example, the common model information 222 is updated in response to receiving a common model, parameter update information, etc. from the server device 300 via the transmission / reception unit 231, or when the common model learning unit 232 performs learning based on the common feature amount included in the learning data 221.

[0024] As described above, in this embodiment, horizontal federated learning is performed. Therefore, the common model included in the common model information 222 is a model learned using also common feature amounts (for example, the attribute C feature amount of sample ID4 in FIG. 1) that the learning device 200 does not have, as a result of horizontal federated learning that learns in cooperation with the other learning device 400.

[0025] The combination parameter information 223 includes combination parameters such as a matrix used when combining the common model and the unique model. For example, as will be described later, the combination / learning unit 233 multiplies the output of the i-th layer of the common model by a matrix W i and combines it with the j-th layer of the unique model. The combination parameter information 223 includes the above matrix W i etc. as combination parameters. In other words, the combination parameter information 223 includes, for example, combination parameters for each combination location. For example, the combination parameter information 223 is updated in response to receiving combination parameters, parameter update information, etc. from the server device 300 via the transmission / reception unit 231, or when the combination / learning unit 233 performs learning based on the common feature amount, unique feature amount, etc. included in the learning data 221.

[0026] Note that, as will be described later, in this embodiment, the coupling parameters are also subject to horizontal federated learning. Therefore, the coupling parameters included in the coupling parameter information 223 reflect the results learned by other learning devices 400 other than the learning device 200 as a result of horizontal federated learning that cooperates with the other learning devices 400.

[0027] The proprietary model information 224 includes a model obtained by performing machine learning processing using at least the proprietary feature amounts. For example, the proprietary model information 224 is updated according to the results of learning performed by the proprietary model learning unit 234 based at least on the proprietary feature amounts included in the learning data 221.

[0028] Note that, as described above, in this embodiment, the proprietary model learning unit 234 learns the proprietary model so as not to forget the results of horizontal federated learning. Therefore, the proprietary model included in the proprietary model information 224 inherits the knowledge of the common model learned by horizontal federated learning.

[0029] The arithmetic processing unit 230 includes an arithmetic device such as a CPU (Central Processing Unit) and its peripheral circuits. The arithmetic processing unit 230 reads and executes the program 225 from the storage unit 220, thereby causing the hardware and the program 225 to cooperate to realize various processing units. The main processing units realized by the arithmetic processing unit 230 include, for example, a transmission / reception unit 231, a common model learning unit 232, a coupling / learning unit 233, a proprietary model learning unit 234, an inference unit 235, and the like.

[0030] The transmission / reception unit 231 transmits and receives data necessary for performing horizontal federated learning to and from the server device 300.

[0031] For example, the transmission / reception unit 231 receives a common model, parameter update information of the common model, and the like from the server device 300. Then, the transmission / reception unit 231 stores the received common model and the like in the storage unit 220 as the common model information 222.

[0032] In addition, the transmission / reception unit 231 transmits parameter update information of the common model or the updated common model to the server device 300. For example, when the common model learning unit 232 updates the common model included in the common model information 222 using the common feature amounts included in the learning data 221, the transmission / reception unit 231 transmits parameter update information indicating the parameters updated by learning or the updated common model to the server device 300.

[0033] In addition, the transmission / reception unit 231 receives the coupling parameter or parameter update information of the coupling parameter from the server device. Then, the transmission / reception unit 231 stores the received coupling parameter or the like in the storage unit 220 as the coupling parameter information 223.

[0034] In addition, the transmission / reception unit 231 transmits parameter update information of the coupling parameter or the updated coupling parameter to the server device. For example, when the coupling / learning unit 233 updates the coupling parameter included in the coupling parameter information 223 using the common feature amounts, unique feature amounts, etc. included in the learning data 221, the transmission / reception unit 231 transmits parameter update information indicating the parameters updated by learning or the updated coupling parameter to the server device 300.

[0035] For example, as described above, the transmission / reception unit 231 transmits and receives information necessary for performing horizontal federated learning. For example, the transmission / reception unit 231 transmits to or receives from the server device 300 the common model, parameter update information of the common model, the coupling parameter, parameter update information of the coupling parameter, etc.

[0036] The common model learning unit 232 generates a common model using the common feature amounts included in the learning data 221. In the case of the present embodiment, the common model learning unit 232 performs horizontal federated learning in which it cooperates with other learning devices 400 to generate a common model, thereby generating a common model based on common feature amounts that the learning device 200 does not have.

[0037] For example, the common model learning unit 232 receives a common model from the server device 300 via the transceiver unit 231. Also, the common model learning unit 232 generates a new common model by updating the common model using the common feature amounts included in the learning data 221. Then, the common model learning unit 232 stores the common model updated and generated by learning in the storage unit 220 as common model information 222. Further, the common model learning unit 232 transmits update parameter information indicating the parameters updated by learning and the like to the server device 300 via the transceiver unit 231. Note that the common model learning unit 232 may repeat the process of receiving the common model and then transmitting update parameter information and the like to the server device 300 until the learning is completed.

[0038] The combination / learning unit 233 combines the common model included in the common model information 222 and the specific model included in the specific model information 224 using the combination parameter indicated by the combination parameter information 223. For example, the combination / learning unit 233 multiplies the output of the i-th layer of the common model by a predetermined value such as the matrix W i which is a combination parameter and combines it with the j-th layer of the specific model.

[0039] As an example, a hyperparameter η i indicating the strength of the combination is predetermined. Also, the matrix W i which is a combination parameter is received from the server device 300 via the transceiver unit 231. For example, the combination / learning unit 233 combines the common model and the specific model using the hyperparameter η i and the matrix W i . For example, the combination / learning unit 233 multiplies the output of the i-th layer of the common model by the matrix W i and η iMultiply the product and add it to the output before passing through the activation function of the j-th layer of the specific model, thereby combining the common model and the specific model. Note that the combination / learning unit 233 can, for example, directly add the output of the common model to the output of the specific model. Also, the combination / learning unit 233 may add several layers after the output. Further, for example, the hyperparameter η is an arbitrary value between 0 and 1 (inclusive).

[0040] Also, the combination / learning unit 233 learns the combination parameters. For example, the combination / learning unit 233 learns the combination parameters using the common features and specific features included in the training data 221. As an example, the combination / learning unit 233 learns the combination parameters by performing horizontal association learning in the same way as the common model. For example, the combination / learning unit 233 receives the combination parameters from the server device 300 via the transceiver unit 231. Also, after combining the common model and the specific model, the combination / learning unit 233 generates new combination parameters by updating the combination parameters using the common features and specific features included in the training data 221. Then, the combination / learning unit 233 stores the combination parameters updated and generated by learning in the storage unit 220 as combination parameter information 223. Also, the combination / learning unit 233 transmits update parameter information indicating the parameters updated by learning to the server device 300 via the transceiver unit 231. The combination / learning unit 233 may repeat the above process until the learning is completed.

[0041] The specific model learning unit 234 generates a specific model using at least the specific features included in the training data 221. Also, in the case of this embodiment, the specific model learning unit 234 can perform the learning and update of the specific model so as not to forget the results learned by horizontal association learning.

[0042] For example, the specific model learning unit 234 learns a specific model using specific feature quantities. Also, the specific model learning unit 234 inputs the output of the intermediate layer of the common model learned by horizontal collaborative learning into each layer of the specific model and then performs learning based at least on the specific feature quantities, so as to learn the specific model without forgetting the results learned by horizontal collaborative learning. In other words, the specific model learning unit 234 performs learning using the common model and the specific model in the combined state combined by the combination and learning unit 233, using the common model updated by horizontal collaborative learning and the coupling parameters. As an example, the specific model learning unit 234 inputs common feature quantities into the common model, inputs specific feature quantities into the specific model, performs learning, and updates the specific model to generate a new specific model. Then, the specific model learning unit 234 stores the updated and generated specific model in the storage unit 220 as specific model information 224. Note that the specific model included in the specific model information 224 is specific to the learning device 200. Therefore, it is not necessary to transmit the specific model to the server device 300 or the like.

[0043] Note that the feature quantities used when the specific model learning unit 234 generates a specific model may include those other than the specific feature quantities. For example, the specific model learning unit 234 may learn a specific model using both the specific feature quantities and the common feature quantities. The specific model learning unit 234 may learn a specific model using the specific feature quantities and some predetermined common feature quantities. Whether to use other than the specific feature quantities when the specific model learning unit 234 learns a specific model may be determined by any means.

[0044] The inference unit 235 performs inference using the results of learning. For example, the inference unit 235 inputs common feature quantities into the common model and inputs specific feature quantities into the specific model to perform inference. For example, the inference unit 235 can use the output of the specific model as the final inference result.

[0045] The above is a configuration example of the learning device 200.

[0046] Here, summarizing the relationships among the common model, the specific model, and the coupling parameters learned by the learning device 200, for example, it becomes as illustrated in FIG. 5. In FIG. 5, the case where the value of the hyperparameter η is 1 and the layer number j = i + 1 is illustrated. As shown in FIG. 5, as an example, the common model and the specific model are coupled by multiplying the output of the intermediate layer of the common model by the product of the matrix and the hyperparameter and adding it to the output before passing through the activation function of the intermediate layer of the specific model. Here, when expressing the output of layer i of the specific model by a mathematical formula, for example, it becomes as shown in the following Equation 1.

Equation

[0047] Also, in the case illustrated in FIG. 5, the final output h out becomes, for example, as shown in the following Equation 2.

Equation

[0048] The above is an example of the relationships among the common model, the specific model, and the coupling parameters learned by the learning device 200. Note that when the value of the hyperparameter η is set to 0, the intermediate layers of the common model and the specific model are not coupled, and only the outputs of the common model and the specific model are added. Also, by changing the a part of the layer number j = i + a to a value other than 1, the way of coupling the common model and the specific model can be changed.

[0049] The server device 300 is an information processing device that receives information to be horizontally federated learned, such as a common model, combined parameters, and parameter update information, from the learning device 200, other learning devices 400, etc., and performs integration processing such as averaging. Further, the server device 300 transmits the integrated common model and combined parameters (or parameter update information, etc.) to the learning device 200 and other learning devices 400.

[0050] FIG. 6 shows a configuration example of the server device 300. Referring to FIG. 6, the server device 300 includes a transmission / reception unit 310 and an integration unit 320. For example, the server device 300 includes an arithmetic device such as a CPU and a storage device, and the arithmetic device executes a program stored in the storage device to realize each of the above processing units. Note that the server device 300 may have general functions other than those exemplified above.

[0051] The transmission / reception unit 310 receives a common model, parameter update information of the common model, combined parameters, parameter update information of the combined parameters, etc. from the learning device 200, other learning devices 400, etc. Further, the transmission / reception unit 310 can transmit a common model, parameter update information of the common model, combined parameters, parameter update information of the combined parameters, etc. to the learning device 200, other learning devices 400, etc.

[0052] The integration unit 320 proceeds with the federated learning process by integrating a plurality of common models, combined parameters, etc. received from the learning device 200, other learning devices 400, etc.

[0053] For example, the integration unit 320 generates a common model as an integrated model by integrating the common models received from the learning device 200 and other learning devices 400, such as by taking the average of the plurality of common models received from the learning device 200 and other learning devices 400. Further, the integration unit 320 can transmit the integrated common model or the parameter update information of the integrated common model to the learning device 200 and other learning devices 400 via the transmission / reception unit 310.

[0054] Further, for example, the integration unit 320 generates a coupling parameter as an integrated parameter obtained by integrating the coupling parameters received from the learning device 200 and other learning devices 400, such as by taking the average of a plurality of coupling parameters received from the learning device 200 and other learning devices 400. Further, the integration unit 320 can transmit the integrated coupling parameter or the parameter update information of the integrated coupling parameter to the learning device 200 and other learning devices 400 via the transmission / reception unit 310.

[0055] For example, as described above, the server device 300 has a configuration for realizing general horizontal federated learning. Further, in the case of the present embodiment, the server device 300 is configured to be able to perform horizontal federated learning not only on the common model but also on the coupling parameters.

[0056] Note that the server device 300 may be configured to determine the first common model and the coupling parameters by any method. For example, the server device 300 may be configured to learn the first common model using only the shared feature amounts on the cloud and transmit the learned common model to the learning device 200 and other learning devices 400.

[0057] The other learning device 400 is an information processing device having at least a function for performing horizontal federated learning on the above-described common model. Further, at least a part of the other learning devices 400 has a function for learning the above-described specific model in addition to the function for performing horizontal federated learning on the common model. In other words, at least a part of the other learning devices 400 included in the learning system 100 can have the same configuration as the configuration of the above-described learning device 200. Since the configuration of the learning device 200 has already been described, the description of the specific configuration of the other learning device 400 is omitted.

[0058] The above is a configuration example of the learning system 100. Subsequently, with reference to FIGS. 7 and 8, an operation example of the learning device 200 will be described.

[0059] FIG. 7 is a flowchart showing an operation example of the learning device 200. Referring to FIG. 7, the learning device 200 determines a hyperparameter η using any means (step S110). The hyperparameter η may be determined in advance.

[0060] Also, the learning device 200 determines a feature amount to be input to the specific model using any means (step S120). For example, the learning device 200 can determine to input only specific feature amounts to the specific model. The types of feature amounts to be input to the specific model may be determined in advance.

[0061] The common model learning unit 232 communicates with the server device 300 via the transmission / reception unit 231, and updates the parameters of the common model using the horizontal federated learning method by performing learning using common feature amounts (step S130). Details of the process in step S130 will be described later.

[0062] The combination / learning unit 233 combines the common model and the specific model using the combination parameter W and the hyperparameter η (step S140). For example, the combination / learning unit 233 multiplies the output of the i-th layer of the common model by the matrix W i and η i and adds the result to the output before passing through the activation function of the j-th layer of the specific model, thereby combining the common model and the specific model.

[0063] Also, the combination / learning unit 233 updates the combination parameter W using the horizontal federated learning method (step S150). The update of the combination parameter using horizontal federated learning may be performed using the same method as in step S130.

[0064] The intrinsic model learning unit 234 learns and updates the intrinsic model so as not to forget the results learned by horizontal federated learning (step S160). For example, the intrinsic model learning unit 234 performs learning using the common model and the intrinsic model in the combined state combined by the combination and learning unit 233, using the common model updated by horizontal federated learning and the combination parameters. Thereby, the intrinsic model learning unit 234 updates the parameters of the intrinsic model.

[0065] The learning device 200 repeats the processing from step S130 to step S160 until the learning is completed (step S170). When the learning is completed (step S170, Yes), the learning device 200 ends the processing.

[0066] FIG. 8 is a flowchart showing a detailed example of the processing of step S130. Referring to FIG. 8, the transmission / reception unit 231 receives a common model from the server device 300 (step S1331). For example, the transmission / reception unit 231 may receive the common model by requesting the server device 300 to transmit the common model, or may receive the common model from the server device 300 in advance.

[0067] The common model learning unit 232 generates a new common model by updating the common model using the common feature amounts included in the learning data 221 (step S132). In addition, the common model learning unit 232 transmits update parameter information indicating the parameters updated by the learning to the server device 300 via the transmission / reception unit 231 (step S133).

[0068] The transmission / reception unit 231 and the common model learning unit 232 can repeat the processing from step S131 to step S133 until the learning is completed (step S134). When the learning is completed (step S134, Yes), the transmission / reception unit 231 and the common model learning unit 232 end the processing of step S130.

[0069] In this way, the learning device 200 has a common model learning unit 232 and a specific model learning unit 234. With such a configuration, the specific model learning unit 234 can learn a specific model so as not to forget the results of learning through horizontal federated learning using the common model learning unit 232. As a result, it is possible to perform learning using not only shared feature amounts but also specific feature amounts that cannot be utilized when performing horizontal federated learning. Thereby, for example, the inference accuracy can be further improved.

[0070] Note that in this embodiment, the case where the learning of the common model and the learning of the specific model using horizontal federated learning are alternately performed has been described. However, as illustrated in FIG. 9, the learning device 200 may be configured to perform the learning of the common model and the learning of the specific model simultaneously. When performing the learning of the common model and the learning of the specific model simultaneously, for example, the learning device 200 determines a hyperparameter γ in addition to the hyperparameter η using any means (step S210). For example, the hyperparameter γ is an arbitrary value of 0 or more. Also, the learning device 200 determines the feature amounts input to the specific model using any means (step S220). Subsequently, the combination / learning unit 233 of the learning device 200 multiplies the output of the i-th layer of the common model by the matrix W i and η i and adds it to the output before passing through the activation function of the j-th layer of the specific model, and combines the common model and the specific model in the same manner as described in this embodiment (step S230). Thereafter, the learning device 200 sets the loss function regarding the final output of the specific model as L, the loss function regarding the output of the common model as L1, and simultaneously updates the specific model, the common model, and the combination parameter W i by minimizing L + γL1 (step S240). At this time, the common model, the combination parameter W iIt is updated by horizontal federated learning. Then, the learning device 200 repeats the processes of combination and update until the learning is completed (step S250). For example, by such a method, the learning device 200 may be configured to perform the learning of the common model and the learning of the specific model simultaneously. Generally, however, higher accuracy is achieved when the learning of the common model and the learning of the specific model are performed alternately.

[0071] Also, in the present embodiment, as one method for performing the learning of the specific model so as not to forget the results learned by horizontal federated learning, the case where the intermediate layer of the common model and the intermediate layer of the specific model are combined and learned using a combination parameter or the like is exemplified. However, the learning device 200 may be configured to perform the learning of the specific model so as not to forget the results learned by horizontal federated learning using a general continuous learning method other than that exemplified in the present embodiment. Further, the learning device 200 may be configured to update only the common model by horizontal federated learning.

[0072] Also, in the present embodiment, the case where the learning system 100 has the server device 300 and horizontal federated learning is performed using the server device 300 is exemplified. However, the learning system 100 does not necessarily have to have the server device 300. When the learning system 100 does not have the server device 300, the learning device 200 performs horizontal federated learning by directly transmitting and receiving the common model, combination parameters, parameter update information, etc. to and from other learning devices 400.

[0073] [Second Embodiment] Next, a second embodiment of the present invention will be described with reference to FIGS. 10 and 11. FIGS. 10 and 11 show a configuration example of the learning device 500.

[0074] FIG. 10 shows a hardware configuration example of the learning device 500 which is an information processing device. Referring to FIG. 10, the learning device 500 has the following hardware configuration as an example. ·CPU (Central Processing Unit) 501 (arithmetic unit) · ROM (Read Only Memory) 502 (memory device) · RAM (Random Access Memory) 503 (memory device) · Program group 504 loaded into RAM 503 · Memory device 505 that stores program group 504 · Drive device 506 that reads and writes to recording medium 510 outside the information processing device · Communication interface 507 connected to communication network 511 outside the information processing device · Input / output interface 508 that performs data input / output · Bus 509 that connects each component

[0075] Also, the learning device 500 can realize the functions as the common model learning unit 521 and the specific model learning unit 522 shown in FIG. 11 by the CPU 501 acquiring the program group 504 and the CPU 501 executing it. The program group 504 is stored in the memory device 505 or ROM 502 in advance, for example, and is loaded into the RAM 503 or the like by the CPU 501 and executed as needed. Also, the program group 504 may be supplied to the CPU 501 via the communication network 511, or may be stored in the recording medium 510 in advance, and the drive device 506 may read the program and supply it to the CPU 501.

[0076] Note that FIG. 10 shows an example of the hardware configuration of the learning device 500. The hardware configuration of the learning device 500 is not limited to the above-described case. For example, the learning device 500 may be configured from a part of the above-described configuration, such as not having the drive device 506.

[0077] The common model learning unit 521 learns a common model in cooperation with other learning devices using common feature amounts that are feature amounts common to other learning devices among the feature amounts possessed by the own device. That is, the common model learning unit 521 learns a common model by collaborative learning.

[0078] Based on the common model learned by the common model learning unit 521, the proprietary model learning unit 522 learns a proprietary model that is a model specific to the device itself, using proprietary feature quantities that are feature quantities of the device itself and not common to other learning devices, so as not to forget the results of the learning by the common model learning unit 621 in cooperation with other learning devices.

[0079] In this way, the learning device 500 has a proprietary model learning unit 522. According to such a configuration, the proprietary model learning unit 522 can learn a proprietary model using proprietary feature quantities so as not to forget the results of the learning by the common model learning unit 621 in cooperation with other learning devices. As a result, it is possible to perform learning using not only shared feature quantities but also proprietary feature quantities that are feature quantities that cannot be utilized when performing collaborative learning. Thereby, for example, the inference accuracy can be further improved.

[0080] Note that the above-described learning device 500 can be realized by incorporating a predetermined program into an information processing device such as the learning device 500. Specifically, a program according to another aspect of the present invention causes an information processing device such as the learning device 500 to learn a common model in cooperation with other learning devices using common feature quantities that are feature quantities of the device itself and common to other learning devices, and based on the learned common model, to learn a proprietary model that is a model specific to the device itself, using proprietary feature quantities that are feature quantities of the device itself and not common to other learning devices, so as not to forget the results of the learning in cooperation with other learning devices.

[0081] Also, a learning method executed by an information processing device such as the above-described learning device 500 is a method in which an information processing device such as the learning device 500 learns a common model in cooperation with other learning devices using common feature quantities that are feature quantities of the device itself and common to other learning devices, and based on the learned common model, learns a proprietary model that is a model specific to the device itself, using proprietary feature quantities that are feature quantities of the device itself and not common to other learning devices, so as not to forget the results of the learning in cooperation with other learning devices.

[0082] Even an invention of a program, a computer-readable recording medium recording the program, or a learning method having the above-described configuration can achieve the object of the present invention described above because it has the same operations and effects as the above-described learning device 500.

[0083] <Supplementary Note> Some or all of the above embodiments may also be described as follows. Hereinafter, an outline of a learning device and the like in the present invention will be described. However, the present invention is not limited to the following configuration.

[0084] (Supplementary Note 1) A common model learning unit that learns a common model used by other learning devices using a common feature amount that is a feature amount common to other learning devices among the feature amounts possessed by the own device, A unique model learning unit that learns a unique model that is a model unique to the own device using unique feature amounts that are feature amounts not common to other learning devices among the feature amounts possessed by the own device based on the common model learned by the common model learning unit, having a learning device. (Supplementary Note 2) The learning device according to Supplementary Note 1, wherein the unique model learning unit learns the unique model by performing learning using a continuous learning method. a learning device. (Supplementary Note 3) The learning device according to Supplementary Note 1 or Supplementary Note 2, wherein the unique model learning unit learns the unique model by inputting the output of the intermediate layer of the common model learned by the common model learning unit into each layer of the unique model and then performing learning using unique feature amounts. a learning device. (Supplementary Note 4) The learning device according to any one of Supplementary Notes 1 to 3, having a coupling unit that couples the common model and the unique model using predetermined coupling parameters, The specific model learning unit performs learning using the common model and the specific model combined by the combining unit. Learning device. (Appendix 5) The learning device according to Appendix 4, The combining unit combines the common model and the specific model by multiplying the output of the intermediate layer of the common model by a value based on the combining parameter and adding it to the output before passing through the activation function of the intermediate layer of the specific model. Learning device. (Appendix 6) The learning device according to Appendix 4 or Appendix 5, The combining unit learns the combining parameter in cooperation with another learning device. Learning device. (Appendix 7) The learning device according to any one of Appendices 1 to 6, having an inference unit that inputs the common feature amount to the common model and inputs the specific feature amount to the specific model for inference. Learning device. (Appendix 8) An information processing device learns a common model used by other learning devices using a common feature amount that is a feature amount common to other learning devices among the feature amounts possessed by the device itself, and learns a specific model that is a model specific to the device itself using a specific feature amount that is a feature amount not common to other learning devices among the feature amounts possessed by the device itself based on the learned common model. Learning method. (Appendix 9) In an information processing device learns a common model used by other learning devices using a common feature amount that is a feature amount common to other learning devices among the feature amounts possessed by the device itself, and learns a specific model that is a model specific to the device itself using a specific feature amount that is a feature amount not common to other learning devices among the feature amounts possessed by the device itself based on the learned common model. A computer-readable recording medium recording a program for realizing the process. (Appendix 10) A common model learning unit that learns a common model used by other learning devices using common feature amounts that are feature amounts common to other learning devices among the feature amounts possessed by the own device, and based on the common model learned by the common model learning unit, a unique model learning unit that learns a unique model that is a model unique to the own device using unique feature amounts that are feature amounts not common to other learning devices among the feature amounts possessed by the own device, and a learning device having the same; A server device having an integration unit that integrates the common models learned by each learning device by communicating with a plurality of learning devices; A learning system having the same.

[0085] The invention of the present application has been described above with reference to each of the above embodiments. However, the invention of the present application is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the invention of the present application within the scope of the invention of the present application.

Explanation of Signs

[0086] 100 Learning system 200 Learning device 210 Communication I / F unit 220 Storage unit 221 Learning data 222 Common model information 223 Combination parameter information 224 Unique model information 225 Program 230 Arithmetic processing unit 231 Transmission / reception unit 232 Common model learning unit 233 Combination / learning unit 234 Unique model learning unit 300 Server device 310 Transmission / reception unit 320 Integration unit 400 Other learning devices 500 Learning device 501 CPU 502 ROM 503 RAM 504 Program group 505 Memory device 506 Drive device 507 Communication interface 508 Input / output interface 509 Bus 510 Recording medium 511 Communication network 521 Common model learning unit 522 Specific model learning unit

Claims

1. A common model learning unit that learns a common model used by other learning devices using common feature amounts that are feature amounts common to other learning devices among the feature amounts possessed by the own device; A unique model learning unit that learns a unique model that is a model unique to the own device using unique feature amounts that are feature amounts not common to other learning devices among the feature amounts possessed by the own device, based on the common model learned by the common model learning unit; It has, The unique model learning unit learns the unique model by inputting the output of the intermediate layer of the common model learned by the common model learning unit into each layer of the unique model and then performing learning using the unique feature amounts. Learning device.

2. The learning device according to Claim 1, The unique model learning unit learns the unique model by performing learning using a continuous learning method. Learning device.

3. The learning device according to Claim 1, It has a coupling unit that couples the common model and the unique model using predetermined coupling parameters, The unique model learning unit performs learning using the common model and the unique model coupled by the coupling unit. Learning device.

4. The learning device according to Claim 3, The coupling unit couples the common model and the unique model by multiplying the output of the intermediate layer of the common model by a value based on the coupling parameters and adding it to the output before passing through the activation function of the intermediate layer of the unique model. Learning device.

5. The learning device according to Claim 3, The coupling unit learns the coupling parameters in cooperation with other learning devices. Learning device.

6. The learning device according to Claim 1, It has an inference unit that inputs the common feature amounts into the common model and inputs the unique feature amounts into the unique model to perform inference. Learning device.

7. An information processing device, Learns a common model used by other learning devices using common feature amounts that are feature amounts common to other learning devices among the feature amounts possessed by the own device, Based on the learned common model, learns a unique model that is a model unique to the own device using unique feature amounts that are feature amounts not common to other learning devices among the feature amounts possessed by the own device, When learning the unique model, the unique model is learned by inputting the output of the intermediate layer of the learned common model into each layer of the unique model and then performing learning using the unique feature amounts. Learning method.

8. In an information processing device, Using a common feature amount that is a feature amount common to other learning devices among the feature amounts possessed by the own device, learn a common model that is also used by other learning devices, Based on the learned common model, learn a unique model that is a model unique to the own device using unique feature amounts that are feature amounts not common to other learning devices among the feature amounts possessed by the own device to realize the process, When learning the unique model, a program for learning the unique model is performed by inputting the output of the intermediate layer of the learned common model into each layer of the unique model and then performing learning using the unique feature amounts.

9. A common model learning unit that learns a common model that is also used by other learning devices using a common feature amount that is a feature amount common to other learning devices among the feature amounts possessed by the own device, and based on the common model learned by the common model learning unit, a unique model learning unit that learns a unique model that is a model unique to the own device using unique feature amounts that are feature amounts not common to other learning devices among the feature amounts possessed by the own device, and a learning device having the same, A server device having an integration unit that integrates the common models learned by each learning device by communicating with a plurality of learning devices, having, The unique model learning unit learns the unique model by inputting the output of the intermediate layer of the common model learned by the common model learning unit into each layer of the unique model and then performing learning using the unique feature amounts. Learning system.

10. A common model learning unit that learns a common model that is also used by other learning devices using a common feature amount that is a feature amount common to other learning devices among the feature amounts possessed by the own device, A unique model learning unit that learns a unique model that is a model unique to the own device using unique feature amounts that are feature amounts not common to other learning devices among the feature amounts possessed by the own device based on the common model learned by the common model learning unit, having, A coupling unit that couples the common model and the unique model using predetermined coupling parameters, The unique model learning unit performs learning using the common model and the unique model coupled by the coupling unit. Learning device.

Images