Information processing apparatus, information processing method, and computer-readable recording medium

Abstract

An information processing apparatus includes: a network building means for building a machine learning model network arranged with a number of candidate operations; a network splitting means for splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and a learning processing means for performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network. After the machine learning, the network splitting means updates the arrangement of the candidate operations in the captain network and the follower network using the results of the knowledge distillation.

Description

INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND COMPUTER-READABLE RECORDING MEDIUM

[0001] The present disclosure relates to an information processing apparatus and an information processing method for constructing a machine learning model, and a computer-readable storage medium storing a program for realizing these.

[0002] In the modern era, the convolution neural network (CNN) has received a lot of popularity because of its capability to achieve high accuracy on computer vision tasks. However, designing CNN is a time-consuming task and requires expertise. Also, with a change in the target task, re-designing is mandatory. A neural architecture search methodology, also known as NAS (Neural Architecture Search), has been developed to solve the problem. Existing NAS cannot only automatically design the CNN of high accuracy but also can also design CNN of high speed.

[0003] There are several variants of the NAS available, for example, but not limited to Bayesian optimization-based NAS, Reinforcement learning-based NAS, Differentiable NAS, and so on. Different variants have a distinct advantages in terms of speed and accuracy. Regarding speed, Differentiable NAS is one of the fastest CNN producing NAS methods.

[0004] Differentiable NAS begins with building an extensive network structure consisting of several candidate operations at each layer in the network. The extensive network is termed SuperNet. An example of the SuperNet is illustrated in fig. 2 below. After building SuperNet, training is performed on the training dataset. In training, only a single operation in each layer participates during one forward propagation and backpropagation step.

[0005] The selection of the particular operation in each layer in each training step can be based on any probability distribution, like uniform, random, and so on. Finally, after training completion, using any evaluation metrics, one or many operation(s) in each stage of the SuperNet is selected, and the new small network structure is built, also termed as Subnet.

[0006] The evaluation metric examples are but not limited to validation loss, special architecture parameter value, validation accuracy, and so on. The differentiable NAS is very fast compared to other NAS methods, and hence the popularity of differentiable NAS has recently stable.

[0007] However, along with several benefits of the differentiable NAS, there exist several concerns about differentiable NAS as well. One of the vital concern is rank disorder. The ranking of the subnet evaluated within the SuperNet is inconsistent with the ranking one would obtain from training the respective subnet independently. Inconsistency existing in the ranking of the differentiable NAS is called a Rank disorder.

[0008] In other words, when the ranking order done of all the subnet within the SuperNet, based on an evaluation metric, does not match with the ranking order based on actual accuracy obtained from training the respective subnet independently, it is termed rank disorder.

[0009] Several reasons exist for the rank disorder concern in differentiable NAS, of which the primary reasons are feature shift and parameter shift. The feature shift occurs during the forward propagation, and the parameter shift occurs during the backpropagation.

[0010] The feature shift occurs basically when for the same input image, during the forward propagation of different training step, any particular candidate operation receive different and uncertain feature maps as input. The reason for the different and uncertain feature maps as input is the different forward propagation paths taken in different training steps. Such different and uncertain feature map distribution is termed a feature shift.

[0011] The parameter shift occurs basically when for the same input image, during the backpropagation of different training step, any particular candidate operation receive different and uncertain weight parameter gradients as input. The reason for the different and uncertain weight parameter gradients as input is the different backpropagation paths taken in different training steps. Such different and uncertain weight parameter gradient distribution is termed a parameter shift.

[0012] The occurrence of feature shift and parameter shift in the SuperNet during the training leads to improper training of the SuperNet. The improper training of the SuperNet yields an inaccurate subnet. Hence, the inaccurate subnet, when ranked based on any evaluation metric, does not yield an accurate ranking. Thereby, rank disorder concern leads to inefficient differentiable NAS.

[0013] The solution to such a problem has been proposed by the prior art as mentioned in non-patent literature 1. The Non-patent literature 1 employs a fixed trained teacher network as the supervision to guide the training of the SuperNet. The knowledge distillation from each layer of the teacher network to all candidate operations in each respective layer of the SuperNet is performed.

[0014] The method effectively provides the same input feature map during forward propagation and the same input weight parameter gradient path during backpropagation, avoiding any feature shift and parameter shift, respectively. However, Non-patent literature has several limitations. The primary limitation is the need for a huge and trained teacher network, which might introduce the need for additional computational and storage resources.

[0015] Also, using a fixed teacher network based knowledge distillation method inherit the primary problem of biasness. The candidate operations in the SuperNet, which have structure similar or same to the teacher network, would enjoy the biasness and may get properly trained, while the other candidates, with a non-similar structure, might not get properly trained in same epoch. The biasness in the knowledge distillation can again introduce improper training of the SuperNet and yield the inaccurate subnet and rank disorder concern.

[0016] [NPL1] Blockwisely Supervised Neural Architecture Search with Knowledge Distillation; Changlin Li1, Jiefeng Peng1, Liuchun Yuan, Guangrun Wang, Xiaodan Liang, Liang Lin, Xiaojun Chang; https: / / arxiv.org / pdf / 1911.13053.pdf

[0017] In this way, the primary problem of the differential NAS disclosed in non-patent literature 1 is biasness and thereby yielding rank disorder, when a fixed teacher network-based knowledge distillation is performed.

[0018] The reason for the primary problem is a fixed structure of the teacher network, leading to the proper training of candidate operation with a structure similar or same to the teacher network and improper training of the candidate operations with a structure not similar or same to the teacher network.

[0019] One example of an object of the present disclosure is to reduce biasness among candidate operations and thereby reduce rank disorder concern.

[0020] In order to achieve the foregoing object, an information processing apparatus according to one aspect of the present discloser includes: a network building means for building a machine learning model network arranged with a number of candidate operations; a network splitting means for splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and a learning processing means for performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network; after the machine learning, the network splitting means uses the results of the knowledge distillation to update the arrangement of the candidate operations in the captain network and the follower network.

[0021] In order to achieve the foregoing object, an information processing method according to another aspect of the present discloser includes: building a machine learning model network arranged with a number of candidate operations; splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network; after the machine learning, using the results of the knowledge distillation to update the arrangement of the candidate operations in the captain network and the follower network.

[0022] In order to achieve the foregoing object, a computer-readable recording medium according to still another aspect of the present discloser has recorded therein a program, and the program includes an instruction to cause the computer to execute: building a machine learning model network arranged with a number of candidate operations; splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network; after the machine learning, updating the arrangement of the candidate operations in the captain network and the follower network using the results of the knowledge distillation.

[0023] As described above, according to the present discloser, it is possible to reduce biasness among candidate operations and thereby reduce rank disorder concern.

[0024] FIG. 1 is a block diagram illustrating a configuration of example of the information apparatus.FIG.2 is a diagram illustrating an example of a SuperNet.FIG. 3 is a diagram illustrating an example of a SuperNet splitted into a captain network and a follower network.FIG.4 is a diagram illustrating an example of a SuperNet during training.FIG. 5 is a diagram illustrating an example of training by knowledge distillation.FIG.6 is a diagram illustrating an example of architecture parameter in the SuperNet.FIG. 7 is a diagram illustrating an example of training after updating architecture parameters.FIG. 8 is a diagram illustrating an example of subnet.FIG. 9 is a flowchart illustrating an example of the operation of the information processing apparatus.FIG.10 is a block diagram illustrating an example of a computer that realizes the information processing apparatus.

[0025] (Example embodiment) Example embodiment of the present discloser is described in detail below referring to the accompanying drawings.

[0026] Configuration of Apparatus First, a configuration of an information processing apparatus 10 according to the example embodiment will be described using FIG. 1. FIG. 1 is a block diagram illustrating a configuration of example of the information apparatus.

[0027] As illustrated in FIG. 1, the information processing apparatus 10 includes a network building unit 30, a network splitting unit 40, and a learning processing unit 50. The network building unit 30 builds a machine learning model network arranged with a number of candidate operations. The network splitting unit 40 splits the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged.

[0028] The learning processing unit 50 performs machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network. After the machine learning by the learning processing unit 50, the network splitting unit 40 updates the arrangement of the candidate operations in the captain network and the follower network using the results of the knowledge distillation.

[0029] Thus, in the information processing apparatus 10, the arrangement of candidate operations in the captain network and the follower network is updated using the knowledge distillation results, and the teacher network in knowledge distillation is not fixed. Therefore, according to the information processing apparatus 10, the biasness among the candidate operations is reduced, and as a result, the rank disorder is suppressed.

[0030] Next, the configuration and function of the information processing apparatus 10 according to the example embodiment will be described in detail using FIGS. 2 to 8 in addition to FIG. 1.

[0031] As illustrated in FIG. 1, in the example embodiment, the information processing apparatus 10 is connected to external data bases 21 and 61 via a network or the like. The data base 21 stores a data set 20 as training data. The data base 61 stores subnets 60 output by the information processing apparatus 10. Data set 20 comprises of image data and labels. Image data are input to a trained CNN model, and the labels are intended to be output from the trained CNN model.

[0032] At the first, the network building unit 30 builds a machine learning model network arranged with a number of candidate operations. In the example embodiment, the machine learning model network is a SuperNet. FIG.2 is a diagram illustrating an example of a SuperNet.

[0033] As illustrated in FIG. 2, the SuperNet 100 comprises multiple layers in which candidate operations are arranged in parallel. In the example of FIG. 2, SuperNet 100 comprises three layers. The first layer consists of candidate operations 101, 102, 103. The second layer consists of candidate operations 111, 112, 113. The last layer consists of candidate operations 121,122, 123. Each candidate operation is labeled as "Cand1", "Cand2", and "Cand3" according to the type of operation. The number of candidate operations and the choice of candidate operations in each layer may vary.

[0034] Furthe, as illustrated in FIG. 2, the output of each candidate operation in all layers is assigned a parameter called architecture parameter α in the SuperNet 100. SuperNet 100 build such a large connecting model. The sample example of the SuperNet 100 builds with several candidate operations. Architecture parameters assigned to each candidate operation is illustrated in fig. 2.

[0035] Then, the network splitting unit 40 according to the example embodiment performs the splitting of the SuperNet 100 into two parts, namely the captain network 200 and the follower network 300, as illustrated in fig. 3. FIG. 3 is a diagram illustrating an example of a SuperNet splitted into a captain network and a follower network.

[0036] In the case of first time splitting of SuperNet 100, the network splitting unit 40 can splite the SuperNet 100 either heuristically or randomly. The network splitting unit 40 select one candidate from each layer in SuperNet 100 and builds an individual network which is termed as captain network 200 using the selected candidate. In this case, the unselected portion of SuperNet 100 is kept and becomes the follower network 300.

[0037] In the sample example illustrated in fig. 3, candidate operations 101, 111, and 121 are selected for the first, second, and last layer of the captain network 200, respectively. As mentioned earlier, during the first time splitting of SuperNet 100, the network splitting unit 40 can select the candidate in the captain network 200 heuristically or randomly. Note that during the training of SuperNet 100, the network splitting unit 40 selects the candidate operations for the captain network 200 based on knowledge distillation loss, as will be described later.

[0038] Once the SuperNet 100 is split into the captain network 200 and follower network 300, the learning processing unit 50 performs the training (machine learning) of the SuperNet 100 as illustrated in FIG. 4. FIG.4 is a diagram illustrating an example of a SuperNet during training.

[0039] In the SuperNet 100, the learning processing unit 50 trains the captain network 200 with the usual objective function to reduce object detection task loss using the data set 20. Whereas the learning processing unit 50 trains the follower network 300 by layer-wise knowledge distillation using the captain network 200 as the teacher network, as illustrated in fig. 5. FIG. 5 is a diagram illustrating an example of training by knowledge distillation.

[0040] In order to perform the knowledge distillation for the follower network 300, an additional loss layer is employed at the output of all candidate operations in the follower network 300 in illustrated in FIG.5. The candidate operation in a layer of the captain network 200 acts as a teacher layer, and all candidate operations in the follower network 300 in the respective layer act as the student layer. All the student layer in the follower network 300 receive the same input as of the teacher layer.

[0041] All the student layers are trained using knowledge distillation from the teacher layer with an objective function to reduce the respective knowledge distillation loss. The learning processing unit 50 trains the captain network 200 with object detection task loss and trains the follower network 300 with knowledge distillation as illustrated in FIG. 5. The training phase where the candidate operations of SuperNet 100 weight parameters are updated is also termed as the weight parameter update phase.

[0042] All the architecture parameters α of the SuperNet 100 are trained using the usual differential NAS way as illustrated in FIG.6. The phase of training the architecture parameter is termed as the architecture parameter update phase. During the architecture update phase, the weight parameter of the candidate operations in the SuperNet 100 are usually frozen. FIG.6 is a diagram illustrating an example of architecture parameter in the SuperNet. The weight parameter update phase and architecture parameter update phase are performed alternatingly.

[0043] After completion of one or several epoch of training, the network splitting unit 40 performs tasks of making a new captain network 200, as illustrated in FIG. 6. In each layer of the follower network 300, the candidate operation with the least distillation loss is selected as the winner and becomes the teacher layer of the new captain network 200 at the respective layer. In other words, the network splitting unit 40 arranges the candidate operation with the smallest distillation loss in the captain network 200 in each layer, and arranges the other candidate operations so that they are closer to the captain network 200 in ascending order of distillation loss. This updates the architecture parameter α.

[0044] In the example of FIG.6, all the candidate operations in the captain network 200 before updating architecture parameter α become part of the follower network 300 at the respective layer. In the sample example of FIG.6, the candidate operations 103, 112, and 123 are selected in the first, second, and last layers of the captain network 200, respectively.

[0045] After the updating of the architecture parameter α, the learning processing unit 50 performs the training (machine learning) of the SuperNet 100 for one or several epochs as illustrated in FIG. 7. Specifically, as illustrated in FIG.7, the learning processing unit 50 trains the newly captain network 200 with the usual objective function to reduce object detection task loss using the newly data set 20. Whereas the learning processing unit 50 trains the newly follower network 300 by layer-wise knowledge distillation using the newly captain network 200 as the teacher network. FIG. 7 is a diagram illustrating an example of training after updating architecture parameters.

[0046] In this way, in SuperNet 100, weight parameter updates and architecture parameter updates are alternately and repeatedly executed over several epochs. And finally, for each layer, one or more candidate operations with high architectural parameter values are selected as winners to build a new small network. Such a small network build of winner candidate operation is a subnet 60. The subnet 60 is the output of the information processing apparatus 10. The sample example of the subnet 60 is as illustrated in FIG. 8. FIG. 8 is a diagram illustrating an example of subnet.

[0047] Operation of Apparatus Next, operation of the information processing apparatus 10 according to the example embodiment will be described using FIG. 9. FIG. 9 is a flowchart illustrating an example of the operation of the information processing apparatus. In the following description, FIGS. 1 to 8 are referred to as appropriate. In the example embodiment, the information processing method is implemented by operating the information processing apparatus 10. Accordingly, the description of the information processing method according to the example embodiment is replaced with the following description of the operations of the information processing apparatus 10.

[0048] First, as illustrated in FIG. 9, the network building unit 30 builds the SuperNet (the machine learning model network) 100 arranged with a number of candidate operations (Step A1). The sample of the SuperNett 100 is illustrated in FIG. 2.

[0049] Next, the network splitting unit 40 performs the splitting of the SuperNet 100 into two parts, namely the captain network 200 and the follower network 300 (Step A2). The sample of the captain network 200 and the sample of follower network 300 are illustrated in FIG. 3.

[0050] Next, the learning processing unit 50 trains the captain network 200 with the usual objective function to reduce object detection task loss using the data set 20, further, trains the follower network 300 by layer-wise knowledge distillation (KD) using the captain network 200 as the teacher network (Step A3). The state of training of captain network 200 and follower network 300 are illustrated in FIGS. 4 and 5.

[0051] Next, the learning processing unit 50 determines whether or not one epoch has been completed (Step A4). As a result of the determination in step A4, if not completed, the learning processing unit 50 executes step A3 again.

[0052] On the other hand, as a result of the determination in step A4, if it is completed, the learning processing unit 50 instructs the network splitting unit 40 to update the architecture parameter α in order to construct a new captain network 200 (step A5).

[0053] Next, the network splitting unit 40 constructs a new captain network 200 by arranging the candidate operations with the smallest distillation loss in each layer in the captain network 200 (step A6). In addition, the network splitting unit 40 arranges the other candidate operations so that they are closer to the captain network 200 in descending order of distillation loss. This updates the architecture parameter α (see FIG. 6).

[0054] Furthermore, the network splitting unit 40 arranges all candidate operations in the captain network 200 before updating the architecture parameter α in the follower network 300 in each layer (step A7).

[0055] Next, the learning processing unit 50 determines whether all epochs have been completed (step A8). As a result of the determination in step A8, if not completed, the learning processing unit 50 executes step A3 again (see FIG. 7).

[0056] On the other hand, if the result of determination in step A8 is that it has been completed, the learning processing unit 50 selects one or more candidate operations with high architecture parameter values for each layer, and constructs the subnet 60 (step A9). An example of subnet 60 is illustrated in FIG. 8. After execution of step A9, the processing of the information processing apparatus 10 ends.

[0057] Effect of the example embodiment As described above, in the example embodiment, during the training of the SuperNet 100, each candidate operation in respective layers of the follower network 300 receives input from the same previous layer of the captain network 200. Hence, the feature shift problem does not occur in the follower network 300.

[0058] The output path of each candidate operation in respective layers is the same in the follower network 300, i.e., loss layers. Hence parameter shift problems do not occur in the follower network 300. In the case of the captain network 200, because of a single path during forward propagation and backpropagation, the problem of feature shift and parameter shift does not occur. Hence, the SuperNet 100 training methodology in the information processing apparatus 10 perfectly avoids the feature shift and parameter shift.

[0059] Also, in the example embodiment, no additional or external teacher network is employed apart from the existing SuperNet, hence no overhead in terms of the need for additional computational and storage resources.

[0060] Furthermore, due to no fixed teacher network and changing of teacher network in terms of captain network 200, every or after a few epochs during the entire SuperNet training, the problem of biasness of SuperNet 100 to a particular teacher network is successfully avoided.

[0061] By tackling feature shift, parameter shift, and biasness problems, the information processing apparatus 10 thus can avoid rank disorder problems and generate optimal subnet 60 as output.

[0062] Modification In the information processing apparatus10, instead of using architecture parameters for selecting the subnet 60 after the completion of the SuperNet 100, validation loss of all the candidate subnets in the SuperNet 100 can be employed. In the case where validation loss is used to select the subnet 60, the architecture parameter in the SuperNet 100 is not required, and thereby no need for the architecture parameter update phase in the training of SuperNet 100.

[0063] Program It suffices for a program in the example embodiment to be a program that causes a computer to carry out steps A1 to A9 shown in FIG. 9. Also, by this program being installed and executed in the computer, the information processing apparatus10 and the information processing method according to the example embodiment can be realized. In this case, a processor of the computer functions and performs processing as the network building unit 30, the network splitting unit 40, and the learning processing unit 50. The computer includes general-purpose PC, smartphone and tablet-type terminal device.

[0064] Furthermore, the program according to the example embodiment may be executed by a computer system constructed with a plurality of computers. In this case, for example, each computer may function as one of the network building unit 30, the network splitting unit 40, and the learning processing unit 50.

[0065] Physical configuration Using FIG. 10, the following describes a computer that realizes the information processing apparatus10 by executing the program according to the example embodiment. FIG.10 is a block diagram illustrating an example of a computer that realizes the information processing apparatus.

[0066] As shown in FIG. 10, a computer 410 includes a CPU (Central Processing Unit) 411, a main memory 412, a storage device 413, an input interface 414, a display controller 415, a data reader / writer 416, and a communication interface 417. These components are connected in such a manner that they can perform data communication with one another via a bus 421.

[0067] The computer 410 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to the CPU 411, or in place of the CPU 411. In this case, the GPU or the FPGA can execute the programs according to the example embodiment.

[0068] The CPU 411 deploys the program according to the example embodiment, which is composed of a code group stored in the storage device 413 to the main memory 412, and carries out various types of calculation by executing the codes in a predetermined order. The main memory 412 is typically a volatile storage device, such as a DRAM (dynamic random-access memory).

[0069] Also, the program according to the example embodiment is provided in a state where it is stored in a computer-readable recording medium 120. Note that the program according to the present example embodiment may be distributed over the Internet connected via the communication interface 417.

[0070] Also, specific examples of the storage device 413 include a hard disk drive and a semiconductor storage device, such as a flash memory. The input interface 414 mediates data transmission between the CPU 411 and an input device 418, such as a keyboard and a mouse. The display controller 415 is connected to a display device 419, and controls display on the display device 419.

[0071] The data reader / writer 416 mediates data transmission between the CPU 411 and the recording medium 420, reads out the program from the recording medium 420, and writes the result of processing in the computer 410 to the recording medium 420. The communication interface 417 mediates data transmission between the CPU 411 and another computer.

[0072] Specific examples of the recording medium 420 include: a general-purpose semiconductor storage device, such as CF (CompactFlash (Registered Trademark) and SD (Secure Digital); a magnetic recording medium, such as a flexible disk; and an optical recording medium, such as a CD-ROM (Compact Disk Read Only Memory).

[0073] Note that the information processing apparatus 10 can also be realized by using items of hardware that respectively correspond to the components, such as an electronic circuit, rather than the computer in which the program is installed. Furthermore, a part of the information processing apparatus 10 may be realized by the program, and the remaining part of the information processing apparatus 10 may be realized by hardware. The computer is not limited to the computer illustrated in FIG. 10.

[0074] A part or an entirety of the above-described example embodiment can be represented by (Supplementary Note 1) to (Supplementary Note 12) described below but is not limited to the description below.

[0075] (Supplementary Note 1) An information processing apparatus comprising: a network building means for building a machine learning model network arranged with a number of candidate operations; a network splitting means for splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and a learning processing means for performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network; after the machine learning, the network splitting means updates the arrangement of the candidate operations in the captain network and the follower network using the results of the knowledge distillation.

[0076] (Supplementary Note 2) The information processing apparatus according to Supplementary Note 1, wherein the machine learning model network is a SuperNet, the learning processing means outputs the captain network with the updated arrangement of the candidate operations as a subnet of the super net.

[0077] (Supplementary Note 3) The information processing apparatus according to Supplementary Note 1, wherein the captain network has a multi-layered structure in which a single candidate operation is arranged in each layer, the follower network has a multi-layer structure in which a single or multiple candidate operations are arranged in each layer.

[0078] (Supplementary Note 4) The information processing apparatus according to Supplementary Note 1, After updating the arrangement of the candidate operations, the learning processing means performs new machine learning on the updated captain network using new training data, and further performs new machine learning on the updated follower network by knowledge distillation using the newly machine-learned captain network, After the new machine learning, the network splitting means uses the result of knowledge distillation used the newly machine-learned captain network to newly update the arrangement of the candidate operations in the captain network and the follower network.

[0079] (Supplementary Note 5) An information processing method comprising: building a machine learning model network arranged with a number of candidate operations; splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network; after the machine learning, updating the arrangement of the candidate operations in the captain network and the follower network using the results of the knowledge distillation.

[0080] (Supplementary Note 6) The information processing method according to Supplementary Note 5, wherein the machine learning model network is a SuperNet, outputting the captain network with the updated arrangement of the candidate operations as a subnet of the super net.

[0081] (Supplementary Note 7) The information processing method according to Supplementary Note 5, wherein the captain network has a multi-layered structure in which a single candidate operation is arranged in each layer, the follower network has a multi-layer structure in which a single or multiple candidate operations are arranged in each layer.

[0082] (Supplementary Note 8) The information processing method according to Supplementary Note 5, After updating the arrangement of the candidate operations, performing new machine learning on the updated captain network using new training data, and further performing new machine learning on the updated follower network by knowledge distillation using the newly machine-learned captain network, After the new machine learning, using the result of knowledge distillation used the newly machine-learned captain network to newly update the arrangement of the candidate operations in the captain network and the follower network.

[0083] (Supplementary Note 9) A computer-readable recording medium that includes a program recorded thereon, the program including instructions that cause a computer to carry out: building a machine learning model network arranged with a number of candidate operations; splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network; after the machine learning, updating the arrangement of the candidate operations in the captain network and the follower network using the results of the knowledge distillation.

[0084] (Supplementary Note 10) The computer-readable recording medium according to Supplementary Note 9, wherein the machine learning model network is a SuperNet, outputting the captain network with the updated arrangement of the candidate operations as a subnet of the super net.

[0085] (Supplementary Note 11) The computer-readable recording medium according to Supplementary Note 9, wherein the captain network has a multi-layered structure in which a single candidate operation is arranged in each layer, the follower network has a multi-layer structure in which a single or multiple candidate operations are arranged in each layer.

[0086] (Supplementary Note 12) The computer-readable recording medium according to Supplementary Note 9, After updating the arrangement of the candidate operations, performing new machine learning on the updated captain network using new training data, and further performing new machine learning on the updated follower network by knowledge distillation using the newly machine-learned captain network, After the new machine learning, using the result of knowledge distillation used the newly machine-learned captain network to newly update the arrangement of the candidate operations in the captain network and the follower network.

[0087] Although the invention of the present application has been described above with reference to the example embodiment, the invention of the present application is not limited to the above-described example embodiment. Various changes that can be understood by a person skilled in the art within the scope of the invention of the present application can be made to the configuration and the details of the invention of the present application.

[0088] As described above, according to the present discloser, it is possible to reduce biasness among candidate operations and thereby reduce rank disorder concern. The present discloser is useful in machine learning systems, and the like.

[0089] 10 Information processing apparatus 20 Data set 21 Data base 30 Network building unit 40 Network splitting unit 50 Learning processing unit 60 Subnet 61 Data base 100 SuperNet 101, 102, 103 Candidate operation 111, 112, 113 Candidate operation 121, 122, 123 Candidate operation 200 Captain network 300 Follower network 410 Computer 411 CPU 412 Main memory 413 Storage device 414 Input interface 415 Display controller 416 Data reader / writer 417 Communication interface 418 Input device 419 Display device 420 Recording medium 421 Bus

Claims

1. An information processing apparatus comprising: a network building means for building a machine learning model network arranged with a number of candidate operations; a network splitting means for splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and a learning processing means for performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network; after the machine learning, the network splitting means updates the arrangement of the candidate operations in the captain network and the follower network using the results of the knowledge distillation.

2. The information processing apparatus according to claim 1, wherein the machine learning model network is a SuperNet, the learning processing means outputs the captain network with the updated arrangement of the candidate operations as a subnet of the super net.

3. The information processing apparatus according to claim 1, wherein the captain network has a multi-layered structure in which a single candidate operation is arranged in each layer, the follower network has a multi-layer structure in which a single or multiple candidate operations are arranged in each layer.

4. The information processing apparatus according to claim 1, After updating the arrangement of the candidate operations, the learning processing means performs new machine learning on the updated captain network using new training data, and further performs new machine learning on the updated follower network by knowledge distillation using the newly machine-learned captain network, After the new machine learning, the network splitting means uses the result of knowledge distillation used the newly machine-learned captain network to newly update the arrangement of the candidate operations in the captain network and the follower network.

5. An information processing method comprising: building a machine learning model network arranged with a number of candidate operations; splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network; after the machine learning, updating the arrangement of the candidate operations in the captain network and the follower network using the results of the knowledge distillation.

6. A computer-readable recording medium that includes a program recorded thereon, the program including instructions that cause a computer to carry out: building a machine learning model network arranged with a number of candidate operations; splitting the machine learning model network into a captain network and a follower network in which the different candidate operations are arranged; and performing machine learning on the captain network using training data, and further performing machine learning on the follower network by knowledge distillation using the machine-learned captain network; after the machine learning, updating the arrangement of the candidate operations in the captain network and the follower network using the results of the knowledge distillation.

Images