Model training method, model training program, model training system, recognition device and recognition program

Abstract

To perform efficient federated learning on arbitrary training data.SOLUTION: A model training method includes a classification unit 251 that classifies, for each of a plurality of training data groups 420 each having one or more pieces of training data, at least some of the training data included in the training data group 420 into a plurality of groups using a classification algorithm 511. The model training method also includes a partial learning unit 252 that uses training data to train a learning model 540 corresponding to a group into which the training data of a plurality of groups has been classified, thereby generating an updated learning model 541. The model training method also includes an aggregating unit 161 that aggregates a plurality of updated learning models 541 trained using each of the plurality of training data groups 420 to generate an aggregated learning model 542.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a model learning method, a model learning program, a model learning system, a recognition device, and a recognition program. [Background technology]

[0002] Artificial intelligence is generally trained using a large amount of data. There are two methods for training artificial intelligence: the conventional method, in which all data is collected and training is performed, and federated learning (FL), in which training is performed on devices that hold each piece of data and the trained learning model is aggregated. However, when federated learning is performed using non-independent and identically distributed (Non-IID) data, the heterogeneity of the data affects the accuracy and efficiency of the learning model, making efficient training difficult.

[0003] For example, Non-Patent Document 1 discloses a technique for clustering data in the intermediate layer using Deep Embedded Clustering (DEC) and integrating the cluster loss into the federated learning loss to generate a unified global model.

[0004] Furthermore, Non-Patent Document 2 discloses a technique for efficiently performing federated learning using multiple nodes with non-independent identically distributed training data. This technique efficiently performs learning by clustering each node using the weights of the learning model trained at each node. As a result, one global model is generated for one cluster. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Afra Mashhadi and 2 others, “Deep embedded clustering of urban communities using federated learning.”, 2021 International Joint Conference on Neural Networks (IJCNN), IEEE, 2021 [Non-patent document 2] Xiaomin Ouyang and 4 others, “ClusterFL: A Clustering-based Federated Learning System for Human Activity Recognition.”, ACM Transactions on Sensor Networks, vol.19, No.1, pp.1-32, December 2022 Summary of the Invention [Problem to be solved by the invention]

[0006] In the technology described in Non-Patent Document 1, one global model is generated, which may result in a loss of diversity in each piece of data. In addition, in the technology described in Non-Patent Document 2, a global model is generated for each cluster, which may result in a loss of diversity in the data held by each node.

[0007] In view of the above circumstances, an object of the present disclosure is to provide a model training method, a model training system, a model training program, and a recognition device and a recognition program suitable for the training method, which perform efficient federated learning on arbitrary data. In particular, federated learning is performed efficiently when the distribution of training data at each node is biased, for example, when the data are in a non-independent and identically distributed relationship. Further objects and the object of the present disclosure will become apparent from the description of this specification and the accompanying drawings. [Means for solving the problem]

[0008] The following describes the means for solving the problems using the numbers and symbols used in the description of the invention. These numbers and symbols are added in parentheses for reference purposes to show an example of the correspondence between the claims and the description of the invention. Therefore, the claims should not be interpreted as being limited by the parenthetical descriptions.

[0009] According to one embodiment, the model training method includes a classification unit (251) classifying, for each of a plurality of training data groups (420), each having one or more pieces of training data, at least a portion of the training data included in the training data group (420) into a plurality of groups using a classification algorithm (511). The model training method also includes a partial training unit (252) using the training data to train a training model (540) corresponding to a group into which the training data has been classified among the plurality of groups, thereby generating an updated training model (541). The model training method also includes an aggregation unit (161) aggregating a plurality of updated training models (541) trained using each of the plurality of training data groups (420), thereby generating an aggregated training model (542).

[0010] According to one embodiment, the model learning system (1000) includes a classification unit (251), a partial learning unit (252), and an aggregation unit (161). The classification unit (251) classifies at least a portion of the learning data included in each of a plurality of learning data groups (420), each having one or more pieces of learning data, into a plurality of groups using a classification algorithm (511). The partial learning unit (252) uses the learning data to train a learning model (540) corresponding to a group into which the learning data has been classified among the plurality of groups, thereby generating an updated learning model (541). The aggregation unit (161) aggregates a plurality of updated learning models (541) trained using each of the plurality of learning data groups (420), thereby generating an aggregated learning model (542).

[0011] According to one embodiment, the model learning program (410) causes the calculation device (120, 220) to classify, for each of a plurality of training data groups (420), each having one or more training data, at least a portion of the training data included in the training data group (420) into a plurality of groups using a classification algorithm (511). The model learning program (410) also causes the calculation device (120, 220) to use the training data, via the partial learning unit (252), to train a training model (540) corresponding to a group into which the training data has been classified among the plurality of groups, thereby generating an updated training model (541). The model learning program (410) also causes the calculation device (120, 220) to aggregate the plurality of updated training models (541) trained using each of the plurality of training data groups (420), thereby generating an aggregated training model (542).

[0012] According to one embodiment, the recognition device includes a classification determination unit (510) and a recognition unit (530). The classification determination unit (510) determines, from a plurality of groups, a group to which input data belongs using a classification algorithm (511). The recognition unit (530) has a plurality of learning models (540) trained to recognize the input data for the plurality of groups, and recognizes the input data using the learning model (540) corresponding to the group to which the input data belongs among the plurality of groups.

[0013] According to one embodiment, the recognition program (440) causes the computing device (120, 220) to determine, from among a plurality of groups, a group to which input data belongs using a classification algorithm (511). The recognition program (440) also has a plurality of learning models (540) trained to recognize the input data for the plurality of groups, and causes the computing device (120, 220) to recognize the input data using the learning model (540) corresponding to the group to which the input data belongs among the plurality of groups. [Effects of the Invention]

[0014] According to one embodiment, federated learning can be performed efficiently on any learning data of each node. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram of a model learning system according to one embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of a recognizer according to an embodiment. [Figure 3] FIG. 3 is a block diagram illustrating a configuration of an aggregation device according to an embodiment. [Figure 4] FIG. 4 is a diagram showing functional blocks executed by a model learning system according to one embodiment. [Figure 5] FIG. 5 is a block diagram showing the configuration of a learning device according to an embodiment. [Figure 6] FIG. 6 is a flowchart showing the process of training a recognizer in a model training system according to one embodiment. [Figure 7] FIG. 7 is a diagram for explaining the process in which the model learning system according to one embodiment distributes a learning model and a classification algorithm. [Figure 8] FIG. 8 is a diagram for explaining the process of learning a learning model in a model learning system according to an embodiment. [Figure 9] FIG. 9 is a diagram for explaining the process of distributing an aggregated learning model in a model learning system according to an embodiment. [Figure 10] FIG. 10 is a block diagram showing the configuration of an aggregation device that realizes a recognizer according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] (Embodiment) A model learning system 1000 according to this embodiment of the present invention will be described with reference to the drawings. In this embodiment, the model learning system 1000 includes an aggregation device 100 and a plurality of learning devices 200 (e.g., a first learning device 200-1, a second learning device 200-2, a third learning device 200-3, etc.), as shown in Fig. 1. The aggregation device 100 is communicatively connected to the plurality of learning devices 200 via a network 20, e.g., the Internet.

[0017] The model learning system 1000 uses learning data stored in each learning device 200 to learn a learning model, for example, artificial intelligence such as a neural network, in each learning device 200. Since multiple learning models are generated for each learning device 200, the aggregation device 100 aggregates the multiple generated learning models and distributes the aggregated learning model to the learning devices 200. Learning of the learning model progresses by repeating the learning process again in each learning device 200.

[0018] The model learning system 1000 according to this embodiment trains a recognizer 500 shown in FIG. 2 . The recognizer 500 can be used for any process of recognizing input data. For example, the recognizer 500 may recognize areas of objects, such as signs, roads, people, and cars, from an image captured by a camera mounted on an automobile. The recognizer 500 includes a classification determination unit 510, a selection unit 520, and multiple recognition units 530. Input data to be recognized is input to the classification determination unit 510 and the selection unit 520.

[0019] The classification determination unit 510 uses a classification algorithm 511 to determine which of a plurality of groups the input data belongs to. The classification determination unit 510 determines the group to which the input data belongs, for example, based on the environment in which the image represented by the input data was captured, such as daytime, nighttime, or rainy. For example, the classification determination unit 510 determines the group to which the input data belongs, such as a group including data representing images captured in the daytime, a group including data representing images captured at night, or a group including data representing images captured in the rain. The classification determination unit 510 transmits the group to which the input data belongs to the selection unit 520. The classification algorithm 511 represents, for example, deep embedded clustering (DEC). The number of groups classified by the classification algorithm 511 may be any number equal to or greater than two.

[0020] The selection unit 520 transfers the input data to the recognition unit 530 corresponding to the group to which the input data belongs. For example, when the classification determination unit 510 determines that the input data belongs to the group corresponding to the first recognition unit 530-1, the selection unit 520 transfers the input data to the first recognition unit 530-1. Similarly, when the selection unit 520 determines that the input data belongs to the group corresponding to the second recognition unit 530-2, the selection unit 520 transfers the input data to the second recognition unit 530-2. When the selection unit 520 determines that the input data belongs to the group corresponding to the third recognition unit 530-3, the selection unit 520 transfers the input data to the third recognition unit 530-3.

[0021] The recognition units 530 to which input data is transmitted recognize the input data using the learning model 540 possessed by each recognition unit 530. Each recognition unit 530 has a different learning model 540. Each learning model 540 is trained to recognize input data determined by the classification determination unit 510 to belong to each group. For example, the first learning model 540-1 possessed by the first recognition unit 530-1 is trained to recognize input data determined by the classification determination unit 510 to belong to the first group. Similarly, the second learning model 540-2 possessed by the second recognition unit 530-2 is trained to recognize input data determined by the classification determination unit 510 to belong to the second group. The third learning model 540-3 possessed by the third recognition unit 530-3 is trained to recognize input data determined by the classification determination unit 510 to belong to the third group.

[0022] For example, the first recognition unit 530-1 is trained to recognize images captured during the day, the second recognition unit 530-2 is trained to recognize images captured at night, and the third recognition unit 530-3 is trained to recognize images captured in the rain. Therefore, when the classification determination unit 510 determines that the input data belongs to a group including data representing images captured during the day, the selection unit 520 transmits the input data to the first recognition unit 530-1. The first recognition unit 530-1 recognizes areas of objects, such as signs, roads, people, and cars, from the image represented in the transmitted input data. Similarly, when the input data is determined to belong to a group including data representing images captured at night, the input data is transmitted to the second recognition unit 530-2 and recognized by the second recognition unit 530-2. When the input data is determined to belong to a group including data representing images captured in the rain, the input data is transmitted to the third recognition unit 530-3 and recognized by the third recognition unit 530-3.

[0023] The model training system 1000 is configured to train such a recognizer 500 using associative learning.

[0024] (Model learning system configuration) The configuration of the aggregation device 100 included in the model learning system 1000 will be described. As shown in FIG. 3, the aggregation device 100 includes an input / output device 110, a calculation device 120, a communication device 130, and a storage device 140. The aggregation device 100 includes a computer such as a cloud server, for example. Information used by the calculation device 120 to execute processing is input to the input / output device 110. The input / output device 110 also outputs the results of processing executed by the calculation device 120. The input / output device 110 includes various input devices and output devices, and may include, for example, a keyboard, a mouse, a microphone, a display, a speaker, a touch panel, etc. The input / output device 110 may be omitted.

[0025] The communication device 130 is communicatively connected to the network 20 and communicates with each learning device 200 via the network 20. The communication device 130 transfers signals acquired from the learning device 200 to the calculation device 120. The communication device 130 also transfers signals generated by the calculation device 120 to the learning device 200. The communication device 130 may acquire information from another device without going through the network 20. For example, the communication device 130 may acquire information from another device, such as the learning device 200, via any storage medium, such as a memory card or a Universal Serial Bus (USB) memory. The communication device 130 may also acquire information from another device directly connected via a USB or the like. The communication device 130 may include various interfaces, such as a Network Interface Card (NIC), a USB, or a communication terminal.

[0026] The storage device 140 stores various data for training the recognizer 500, such as the classification training data group 400, and the model training program 410. The storage device 140 is used as a non-transitory tangible storage medium for storing the model training program 410. The model training program 410 may be provided as a computer program product recorded on a computer-readable storage medium 1, or may be provided as a computer program product downloadable from a server.

[0027] The classification training data group 400 includes multiple classification training data used to generate the classification algorithm 511. For example, the classification training data represents images captured by a camera mounted on an automobile. The classification training data does not need to include information representing the group to which it belongs. The distribution of the classification training data represents, for example, the distribution of all training data stored in each learning device 200. Here, the distribution of the classification training data representing the distribution of all training data means that the distribution of the classification training data is similar to the distribution of the entire training data. For example, the classification training data and the entire training data have an independent and identical distribution relationship. For example, the classification image data included in the classification training data group 400 includes data representing images captured in various environments, such as daytime, nighttime, and rainy, and has approximately the same number of data representing images captured in each environment.

[0028] The arithmetic device 120 reads out the model learning program 410 from the storage device 140 and executes it to perform various data processing for learning the recognizer 500. For example, the arithmetic device 120 includes a central processing unit (CPU) and the like.

[0029] The calculation device 120 reads and executes the model learning program 410, thereby implementing a classification algorithm generation unit 150 and a model aggregation node 160 in cooperation with the storage device 140, as shown in FIG. 4. The classification algorithm generation unit 150 generates a classification algorithm 511 using multiple classification learning data included in the classification learning data group 400. The model aggregation node 160 includes an aggregation unit 161. The aggregation unit 161 aggregates multiple learning models 540 trained in each learning device 200, such as multiple first learning models 540-1, multiple second learning models 540-2, and multiple third learning models 540-3, to generate multiple aggregated learning models. The aggregation unit 161 generates one aggregated learning model by aggregating multiple corresponding learning models 540. For example, a first aggregated learning model is generated by aggregating multiple first learning models 540-1. A second aggregated learning model is generated by aggregating multiple second learning models 540-2. The third aggregated learning model is generated by aggregating multiple third learning models 540-3.

[0030] Next, the configuration of the learning device 200 shown in FIG. 1 will be described. As shown in FIG. 5, each learning device 200 includes an input / output device 210, a calculation device 220, a communication device 230, and a storage device 240. The learning device 200 includes, for example, a computer, a tablet, a mobile phone, a smartphone, a mobile terminal, etc. Information used by the calculation device 220 to execute processing is input to the input / output device 210. The input / output device 210 also outputs the results of processing executed by the calculation device 220. The input / output device 210 includes various input devices and output devices, and may include, for example, a keyboard, a mouse, a microphone, a display, a speaker, a touch panel, etc. The input / output device 210 may be omitted.

[0031] The communication device 230 is communicatively connected to the network 20 and communicates with the aggregation device 100 via the network 20. The communication device 230, for example, transfers a signal acquired from the aggregation device 100 to the calculation device 220. The communication device 230 also transfers a signal generated by the calculation device 220 to the aggregation device 100. For example, the communication device 230 may acquire information from another device, such as the aggregation device 100, via any storage medium, such as a memory card or a Universal Serial Bus (USB) memory. The communication device 130 may also acquire information from another device directly connected via a USB or the like. The communication device 130 may include various interfaces, such as a transceiver used for wireless communication, such as a wireless local area network (LAN) or a cellular network, a network interface card (NIC), a USB, or a communication terminal.

[0032] The storage device 240 stores various data for training the recognizer 500, particularly the multiple training models 540, such as the training data group 420 and the partial training program 430. The storage device 240 is used as a non-transitory tangible storage medium for storing the partial training program 430. The partial training program 430 may be provided as a computer program product recorded on a computer-readable storage medium 2, or may be provided as a computer program product downloadable from a server. Alternatively, the partial training program 430 may be provided from the aggregation device 100.

[0033] The training data group 420 includes one or more training data used for training multiple learning models 540. For example, the training data represents images captured by a camera mounted on a vehicle and includes information representing object regions within the images, such as sign regions, road regions, human regions, and car regions. The training data may have a non-independent identical distribution relationship with the training data of the training data group 420 stored in other learning devices 200. For example, the training data group 420 held by the first learning device 200-1 may include more images captured in the rain than other images, and the training data group 420 held by the second learning device 200-2 may include more images captured in the daytime than other images. Furthermore, the training data group 420 held by the third learning device 200-3 may include approximately the same number of images captured in the daytime, at night, and in the rain. The number of training data held by each learning device 200 may be the same or different.

[0034] The arithmetic device 220 reads and executes the partial learning program 430 from the storage device 240 to perform various data processing for learning the recognizer 500, particularly the multiple learning models 540. For example, the arithmetic device 220 includes a central processing unit (CPU) and the like.

[0035] The arithmetic device 220 reads and executes the partial learning program 430, thereby realizing the model learning node 250 in cooperation with the storage device 240, as shown in FIG. 4. The model learning node 250 has a classification unit 251 and a partial learning unit 252, and learns multiple learning models 540. The classification unit 251 classifies one or more pieces of learning data into multiple groups using a classification algorithm 511. The partial learning unit 252 uses the learning data belonging to each group to learn a learning model 540 corresponding to each group. For example, the partial learning unit 252 uses the learning data belonging to the first group to learn a first learning model 540-1 corresponding to the first group, and uses the learning data belonging to the second group to learn a second learning model 540-2 corresponding to the second group. Furthermore, the partial learning unit 252 uses the learning data belonging to the third group to learn a third learning model 540-3 corresponding to the third group.

[0036] (Model learning system operation) The operation of model learning system 1000 will now be described. For example, a user starts aggregation device 100 and inputs an operation to input / output device 110 of aggregation device 100 to train recognizer 500. When the user's operation is input, calculation device 120 of aggregation device 100 reads and executes model learning program 410 from storage device 140. When model learning program 410 is executed, calculation device 120 starts the processing shown in FIG. 6, which is part of a model learning method.

[0037] In step S110, the classification algorithm generation unit 150 realized by the calculation device 120 generates the classification algorithm 511 using the classification training data of the classification training data group 400. For example, if the classification algorithm 511 classifies input data using artificial intelligence such as deep embedded clustering, the classification algorithm generation unit 150 uses the classification training data to train the classification algorithm 511 and generate the classification algorithm 511.

[0038] In step S120, the aggregating unit 161 distributes information representing the generated classification algorithm 511 and an initial model representing the learning model 540 before learning to each learning device 200. The initial model represents, for example, a learning model 540 in which weights connecting nodes are set randomly. As shown in FIG. 7 , the aggregating unit 161 generates as many initial models as the number of groups into which input data is classified by the classification algorithm 511, and distributes information on the learning models 540 representing the generated multiple initial models to each learning device 200. For example, the aggregating unit 161 distributes information representing the node structure of the learning model 540 and information representing the weights connecting each node to each learning device 200. At this time, the aggregating unit 161 outputs information representing the groups corresponding to each of the multiple learning models 540 to each learning device 200.

[0039] 6, the classification unit 251 of the learning device 200 classifies the learning data of the learning data group 420 into a plurality of groups using the classification algorithm 511 acquired from the aggregation device 100. For example, the classification unit 251 classifies the learning data into a first group including images captured during the day, a second group including images captured at night, a third group including images captured in the rain, and so on. Note that the processes from step S130 to step S150 are executed in the learning device 200, for example, in each of the model learning nodes 250.

[0040] In step S140, the partial learning unit 252 uses the learning data classified into multiple groups to train a learning model 540 corresponding to the classified group, thereby generating an updated learning model. As shown in FIG. 8 , the partial learning unit 252 trains the multiple learning models 540 acquired from the aggregation device 100 using the learning data classified into the groups corresponding to the learning models 540, thereby generating multiple updated learning models 541. For example, the partial learning unit 252 uses the training data to train the corresponding learning model 540, thereby generating an updated learning model 541 in which the weights connecting the nodes of the learning model 540 are updated. For example, the partial learning unit 252 uses one or more training data classified into a first group, for example, training data representing images captured during the daytime, to train a first learning model 540-1 corresponding to the first group, for example, a learning model 540 that recognizes images captured during the daytime. The partial learning unit 252 trains the first learning model 540-1 to generate a first updated learning model 541-1. Similarly, the partial learning unit 252 uses one or more pieces of learning data classified into the second group, for example, learning data representing images captured at night, to train a second learning model 540-2 corresponding to the second group, for example, a learning model 540 that recognizes images captured at night. The partial learning unit 252 generates a second updated learning model 541-2 by training the second learning model 540-2. The partial learning unit 252 uses one or more pieces of learning data classified into the third group, for example, learning data representing images captured in the rain, to train a third learning model 540-3 corresponding to the third group, for example, a learning model 540 that recognizes images captured in the rain. The partial learning unit 252 trains the third learning model 540-3 to generate a third updated learning model 541-3. Therefore, multiple updated learning models 541 are generated in the learning device 200, for example, in each model learning node 250.

[0041] 6, the partial learning unit 252 outputs information on the generated plurality of updated learning models 541 to the aggregation device 100. For example, as shown in FIG. 8, the partial learning unit 252 outputs information representing weights connecting nodes in the plurality of updated learning models 541 to the aggregation device 100.

[0042] In step S160 shown in FIG. 6, the aggregation unit 161 of the aggregation device 100 aggregates, for each group, the multiple updated learning models 541 corresponding to each group acquired from the learning device 200. As shown in FIG. 9, the aggregation unit 161 aggregates the multiple updated learning models 541 corresponding to each group to generate multiple aggregated learning models 542 corresponding to each group. For example, the aggregation unit 161 acquires a first updated learning model 541-1 corresponding to the first group from each of the learning devices 200. The aggregation unit 161 aggregates the acquired multiple first updated learning models 541-1 to generate one first aggregated learning model 542-1 corresponding to the first group. The aggregation unit 161 aggregates the acquired multiple second updated learning models 541-2 to generate one second aggregated learning model 542-2 corresponding to the second group. The aggregation unit 161 aggregates the acquired third updated learning models 541-3 to generate one third aggregated learning model 542-3 corresponding to the third group.

[0043] The aggregating unit 161 generates the aggregated learning model 542 from the multiple updated learning models 541 by any method. For example, the aggregating unit 161 generates the aggregated learning model 542 by using FedAvg (Federated Average), which calculates the average value of each weight connecting nodes of the multiple updated learning models 541 as the weight connecting nodes of the aggregated learning model 542.

[0044] In step S170 shown in FIG. 6, the aggregating unit 161 determines whether the multiple aggregated learning models 542 satisfy a termination condition. The termination condition may be any condition indicating that learning has ended. For example, the termination condition may indicate that the recognition rate of the aggregated learning model 542 exceeds a threshold. For example, the termination condition may indicate that the recognition rates of all of the aggregated learning models 542 exceed a threshold. Alternatively, the termination condition may indicate that the number of repetitions reaches a threshold, for example, that the processes from step S130 to step S160 are repeated a threshold number of times. For example, the aggregating unit 161 terminates learning of the recognizer 500 when all of the multiple aggregated learning models 542 satisfy the termination condition (YES). When even one of the multiple aggregated learning models 542 does not satisfy the termination condition (NO), the aggregating unit 161 executes the process of step S180. Alternatively, the aggregating unit 161 may terminate learning of the recognizer 500 when a predetermined number or more of the multiple aggregated learning models 542 satisfy the termination condition (YES). When the number of aggregated amount models that satisfy the termination condition among the plurality of aggregated learning models 542 is less than a predetermined number (NO), the aggregation unit 161 may execute the process of step S180.

[0045] In step S180, the aggregation unit 161 distributes the multiple aggregated learning models 542 as learning models 540 to each learning device 200. As shown in FIG. 9 , each learning device 200 repeats the process from step S130 using the acquired aggregated learning model 542 as the learning model 540, thereby repeating learning of the aggregated learning model 542.

[0046] In this way, the model learning system 1000 learns the recognizer 500. The model learning system 1000 learns the learning model 540 according to the group of the learning data, and therefore can efficiently learn a variety of learning data. For example, even if the groups to which the learning data belong in each learning device 200 are biased, the learning of each learning model 540 can be efficiently progressed by updating the learning model 540 with the learning data corresponding to each group.

[0047] Furthermore, the information about the learning model 540 that the model learning system 1000 acquires from the learning device 200 is information about the learning model 540 for each group. Therefore, in the information that the model learning system 1000 acquires from the learning device 200, the bias toward which of the groups the learning data acquired by each learning device 200 is likely to be reflected in the weights connecting the nodes of each learning model 540. For example, the information output from the learning device 200 is unlikely to include information indicating whether the learning data acquired by the learning device 200 contains a large number of images captured during the day or a large number of images captured at night. Therefore, the tendency of the learning data held by the learning device 200 is easily protected, and therefore, even if highly private learning data is used, information about privacy is unlikely to leak from the learning device 200 to the aggregation device 100.

[0048] (Using recognizer 500) The recognizer 500 trained by the model training system 1000 may be executed in the aggregation device 100, and the aggregation device 100 may be realized as a recognition device. In this case, the storage device 140 of the aggregation device 100 stores a recognition program 440 for realizing the recognizer 500, as shown in FIG. 10 . The storage device 140 is used as a non-transitory tangible storage medium for storing the recognition program 440. The recognition program 440 may be provided as a computer program product recorded on a computer-readable storage medium 3, or may be provided as a computer program product downloadable from a server.

[0049] The arithmetic unit 120 of the aggregation device 100 reads and executes the recognition program 440 from the storage device 140 to implement each part of the recognizer 500 shown in FIG. 2. For example, the aggregation device 100 accepts input data. For example, the aggregation device 100 accepts input data, such as data representing an image captured by a camera mounted on an automobile, from another terminal, such as a terminal external to the model learning system 1000, or the learning device 200, via the network 20. Upon receiving the input data, the arithmetic unit 120 of the aggregation device 100 executes the recognition program 440 to start processing part of the recognition method.

[0050] For example, upon receiving input data, the classification determination unit 510 implemented by the computing device 120 uses a classification algorithm 511 to determine the group to which the input data, e.g., the input image, belongs. The selection unit 520 transmits the input data to one of the multiple recognition units 530 that corresponds to the group to which the input data belongs, e.g., a group including images captured during the day. The recognition unit 530 to which the input data is transmitted recognizes the input data. For example, the recognition unit 530 recognizes object regions, such as sign regions, road regions, human regions, and car regions, from the input image. Information representing the recognition results is output to a terminal that output the input data, such as a terminal external to the model learning system 1000 or the learning device 200, and is used by the terminal that output the input data. For example, the terminal extracts the region of a sign identified from the image and recognizes the characters displayed on the sign. The terminal may also identify the location of nearby cars from the region of a car identified from the image.

[0051] In this way, input data is recognized using the trained recognizer 500. Note that the recognizer 500 may be realized in a terminal external to the model learning system 1000. For example, a computing device in a terminal external to the model learning system 1000 may realize the recognizer 500 by executing the recognition program 440.

[0052] (Variation) The above-described embodiments and modifications are merely examples, and the configurations described in each embodiment and modification may be modified as desired without impairing functionality. For example, the classification training data in the classification training data group 400 stored in the aggregation device 100 may include information indicating the group to which it belongs. In this case, in step S110 shown in FIG. 6, the classification algorithm generation unit 150 of the aggregation device 100 trains the classification algorithm 511 so that the classification training data can be classified into the group to which each piece of data belongs.

[0053] Furthermore, the distribution of the categorization training data included in the categorization training data group 400 does not have to represent the distribution of all the training data stored in each learning device 200. For example, the categorization training data may have any distribution that includes the entire distribution of the training data included in all the training data groups 420. For example, the categorization image data included in the categorization training data group 400 may include data representing images captured in various environments, such as daytime, nighttime, and rainy, while the number of data representing images captured in each environment included in the categorization training data group 400 may be uneven.

[0054] 6, the aggregating unit 161 of the aggregating device 100 may generate one initial model instead of generating as many initial models as the number of groups into which the input data is classified by the classification algorithm 511. For example, the aggregating unit 161 may generate one initial model and distribute the generated initial model to each learning device 200. In this case, each learning device 200 may duplicate the initial model the number of times equal to the number of groups into which the input data is classified by the classification algorithm 511, and learn by associating the duplicated initial models with the groups. Furthermore, the aggregating unit 161 may generate initial models in numbers less than the number of groups and distribute them to each learning device 200. In this case, each learning device 200 may duplicate the initial model by the number that is less than the number of groups.

[0055] Furthermore, the classification algorithm 511 is generated once by the classification algorithm generation unit 150 of the aggregation device 100 and then used without change by the classification unit 251 of the learning device 200 to classify training data into multiple groups. However, this is not limiting. For example, the classification algorithm 511 may be updated using training data of the training data group 420 stored in the learning device 200. For example, the partial training unit 252 of the learning device 200 may train the classification algorithm 511 and the training model 540 using the training data of the training data group 420 by any method. For example, the partial training unit 252 may perform a process of training the training model 540 while fixing the classification algorithm 511, and a process of training the classification algorithm 511 while fixing the training model 540. In this case, the partial training unit 252 outputs, to the aggregation device 100, an updated classification algorithm trained using the classification algorithm 511 and an updated training model 541 trained using the training model 540. The aggregation unit 161 of the aggregation device 100 aggregates multiple updated classification algorithms acquired from each of the learning devices 200 to generate one aggregated classification algorithm, and generates multiple aggregated learning models 542 from multiple updated learning models 541. Similar to the aggregated learning model 542, the aggregation unit 161 outputs the aggregated classification algorithm to each learning device 200 as a classification algorithm 511, and the classification algorithm 511 is trained.

[0056] 6, the partial learning unit 252 of the learning device 200 may use a portion of the learning data included in the learning data group 420 to learn the learning model 540. For example, the learning data used by the partial learning unit 252 when learning the learning model 540 for the first time may be different from the learning data used when learning the learning model 540 for the second time.

[0057] 6, the aggregating unit 161 of the aggregating device 100 may aggregate the multiple learning models 540 into one learning model 540 after the multiple learning models 540 satisfy a termination condition. In this case, the recognizer 500 trained by the model training system 1000 has one recognition unit 530 and recognizes input data using the single learning model 540. In addition, the recognizer 500 may have multiple recognition units 530 having the same learning model 540 as a result of training by the partial training unit 252 of the training device 200 in the configuration shown in FIG.

[0058] The recognizer 500 is not limited to performing semantic segmentation of images, but may recognize any input data for any purpose.

[0059] The above-described embodiments and modifications are merely examples. The configurations described in each embodiment and modification may be arbitrarily modified and / or combined as long as the functionality is not impaired. Furthermore, some of the functions described in the embodiments and modifications may be omitted as long as the necessary functionality is achieved. For example, some or all of the processing of the aggregating device 100 may be performed by the learning device 200. For example, the arithmetic unit 220 of one learning device 200 may implement both the model aggregation node 160 and the model learning node 250. The arithmetic unit 220 of one learning device 200 may implement multiple model learning nodes 250. Furthermore, the arithmetic unit 120 of the aggregating device 100 may implement both the model aggregation node 160 and one or more model learning nodes 250. The aggregating device 100 and the learning device 200 may be implemented by multiple computers. In one learning device 200, the model learning program 410 may include a partial learning program 430.

[0060] Furthermore, the classification algorithm 511 may be generated by a device external to the model learning system 1000. In this case, in step S110 shown in Fig. 6, the aggregation device 100 may acquire the classification algorithm 511 from the external device. Furthermore, each of the learning devices 200 may acquire the classification algorithm 511 from the external device.

[0061] Furthermore, the model learning system 1000 may cause an external terminal having learning data to learn the learning model 540. For example, the aggregation device 100 distributes the partial learning program 430, the classification algorithm 511, and the learning model 540 to the external terminal. The external terminal executes the partial learning program 430, classifies the learning data using the classification algorithm 511, and learns the learning model 540 using the classified learning data. Furthermore, the model learning system 1000 may cause a different learning device 200 to learn the learning model 540 during repeated learning of the learning model 540. For example, during repeated learning of the learning model 540, a learning device 200 that learned the learning model 540 the first time may not learn the learning model 540 the second time. Furthermore, a learning device 200 that did not learn the learning model 540 the first time may learn the learning model 540 the second time. [Explanation of symbols]

[0062] 1, 2, 3: Storage medium 20: Network 100: Aggregation device 110: Input / output device 120: Arithmetic device 130: Communication equipment 140: Storage device 150: Classification algorithm generation unit 160: Model Aggregation Node 161: Aggregation section 200: Learning device 210: Input / output device 220: Arithmetic device 230:Communication equipment 240: Storage device 250: Model learning node 251: Classification section 252: Partial learning section 400: Classification training data set 410: Model Learning Program 420: Training data set 430: Partial Learning Program 440: Recognition program 500: Recognizer 510: Classification judgment section 511: Classification Algorithms 520: Selection section 530: Recognition section 540: Learning Model 541: Renewal learning model 542: Aggregate Learning Model 1000: Model Learning System

Claims

1. classifying, by a classification unit, at least some of the learning data included in each of a plurality of learning data groups, each of which has one or more learning data, into a plurality of groups using a classification algorithm; generating an updated learning model by using the learning data to learn a learning model corresponding to the group into which the learning data is classified among the plurality of groups, using a partial learning unit; aggregating, by an aggregating unit, the updated learning models trained using the respective learning data groups to generate an aggregated learning model; Model learning methods including:

2. generating the updated training model includes: training the aggregated learning model as the learning model to generate a new updated learning model; The model training method of claim 1 , comprising:

3. A classification algorithm generation unit generates the classification algorithm based on the classification learning data. The model training method of claim 1 further comprising:

4. The classification into the plurality of groups includes: classifying the training data into the plurality of groups using, without modification, the classification algorithm generated based on the classification training data; The model training method of claim 3 , comprising:

5. The distribution of the classification training data represents the distribution of all the training data in the plurality of training data groups. The model learning method according to claim 3 or 4.

6. The training data included in a first training data group and the training data included in a second training data group among the plurality of training data groups are in a non-independent and identically distributed relationship with each other. The model learning method according to any one of claims 1 to 4.

7. the plurality of training data groups are stored in a plurality of model training nodes, respectively; the classification unit and the partial learning unit are provided in at least a first model learning node among the plurality of model learning nodes; The aggregation unit is provided in a model aggregation node. The model learning method according to any one of claims 1 to 4.

8. The classification algorithm represents classification by deep convolutional clustering. The model learning method according to any one of claims 1 to 4.

9. classifying, by a classification unit, at least some of the learning data included in each of a plurality of learning data groups, each of which has one or more learning data, into a plurality of groups using a classification algorithm; generating an updated learning model by using the learning data to learn a learning model corresponding to the group into which the learning data is classified among the plurality of groups, using a partial learning unit; aggregating, by an aggregating unit, the updated learning models trained using the respective learning data groups to generate an aggregated learning model; A model learning program that causes a computing device to execute the above.

10. a classification unit that classifies at least a portion of the training data included in each of a plurality of training data groups, each of the training data groups having one or more training data, into a plurality of groups using a classification algorithm; a partial learning unit that uses the learning data to learn a learning model corresponding to a group in which the learning data among the plurality of groups is classified, and generates an updated learning model; an aggregating unit that aggregates the updated learning models trained using the plurality of learning data groups to generate an aggregated learning model; A model learning system comprising:

11. a classification determination unit that determines a group to which input data belongs from among a plurality of groups using a classification algorithm; a recognition unit that has a plurality of learning models trained to recognize the input data for the plurality of groups, and recognizes the input data using a learning model corresponding to a group to which the input data belongs among the plurality of groups; A recognition device comprising:

12. determining a group to which input data belongs from a plurality of groups using a classification algorithm; a plurality of learning models for the plurality of groups that are trained to recognize the input data, and recognizing the input data using a learning model corresponding to a group to which the input data belongs among the plurality of groups; A recognition program that causes a computing device to execute the above.

Images