Information processing device and information processing method

Abstract

To provide a technique to make feature vectors substantially the same to the same input, the feature vectors being a feature vector obtained from one hierarchical structure and a feature vector obtained from the other hierarchical structure.SOLUTION: An information processing apparatus divides a hierarchical structure of a teacher model for determining a feature vector of input data into a plurality of sub structures, and performs learning of a feature extraction unit corresponding to the sub structures based on input data input to the sub structures and output data output from the sub structures based on the input data. The information processing apparatus creates, by using the learned feature extraction unit, a student model having a hierarchical structure for determining the feature vector of the input data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to learning techniques. [Background technology]

[0002] In recent years, many feature extraction techniques have been proposed to extract useful information from images of objects captured in photographs by performing advanced processing on the images. In particular, feature extraction techniques that use multi-layered neural networks called deep nets (also known as deep neural networks, deep learning, or deep learning) to extract feature vectors of objects in images have been actively researched.

[0003] While the popularity of feature extraction techniques using deep networks is well known, a deep network learning technique called distillation, as described in Patent Document 1, has recently attracted attention. Distillation is a technique for using a trained deep network model (called a teacher model) to train a deep network with a different network architecture (called a student model). Learning by distillation is generally performed with the goal of reducing the weight of the teacher model, and therefore the student model is often provided with a lighter network architecture than the teacher model. In distillation, the student model is trained using the feature vectors output by the teacher model instead of ground truth labels. Therefore, learning by distillation does not require the large number of labeled training images required for conventional learning. It is known that this distillation technique can propagate the knowledge of the teacher model to the student model.

[0004] A student model trained using distillation technology can output feature vectors that are nearly identical to those of a teacher model. Therefore, even though the network architectures and associated parameters of the student and teacher models are different, when the same image is input to both models, they will output nearly identical feature vectors.

[0005] Furthermore, Non-Patent Document 1 discloses a technique for improving the ease of learning through distillation by learning so that the intermediate feature vectors generated in the process of calculating feature vectors by the teacher model and the intermediate feature vectors generated in the process of calculating feature vectors by the student model are approximately identical, and research and development of this technique is being actively conducted. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] US10289962 [Non-patent literature]

[0007] [Non-Patent Document 1] FITNETS: HINTS FOR THIN DEEP NETS Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta & Yoshua Bengio Summary of the Invention [Problem to be solved by the invention]

[0008] However, when the distillation becomes more difficult, for example, when distilling a student model in which the number of neural network parameters is significantly reduced from that of the teacher model, conventional methods may not result in the feature vectors of the student model and the teacher model being approximately identical.The present invention provides a technology for making the feature vectors obtained from one hierarchical structure and the feature vectors obtained from the other hierarchical structure for the same input approximately identical. [Means for solving the problem]

[0009] In one aspect of the present invention, a hierarchical structure of a teacher model for obtaining a feature vector of input data is divided into a plurality of sub-structures, and the hierarchical structure OfA replacement hierarchical structure in which, for each substructure of interest, the substructure of interest is replaced with a feature extraction unit corresponding to the substructure of interest. the replacement hierarchical structure using the feature extraction unit and a substructure other than the target substructure in the hierarchical structure of the teacher model; and a learning means for learning a feature extraction unit corresponding to each target substructure based on an output of the final layer of the replacement hierarchical structure for input data and an output of the final layer of the teacher model for the input data. a generation means for generating a student model having a hierarchical structure for obtaining a feature vector of input data using the feature extraction unit trained by the learning means; The present invention is characterized by comprising: [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a technique for making the feature vector obtained from one hierarchical structure and the feature vector obtained from another hierarchical structure for the same input substantially identical. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing an example of the hardware configuration of an information processing device 100. [Figure 2] FIG. 1 is a block diagram showing an example of the functional configuration of an information processing device 100. [Figure 3] FIG. 2 is a block diagram showing the flow of data in the information processing device 100. [Figure 4] 4A and 4B are diagrams showing examples of division by the division unit 204. [Figure 5] FIG. 2 shows an example of the configuration of an extraction unit 211. [Figure 6] 3 is a flowchart of processing performed by the information processing device 100. [Figure 7] 10 is a flowchart showing details of the process in step S606. [Figure 8] FIG. 10 is a diagram illustrating a first modified example. [Figure 9] FIG. 1 is a block diagram showing an example of the functional configuration of an information processing device 100. [Figure 10] FIG. 9 is a block diagram showing an example of the functional configuration of a relearning unit 901. [Figure 11]FIG. 2 is a block diagram showing the flow of data in the information processing device 100. [Figure 12] FIG. 9 is a diagram for explaining the operation of a relearning unit 901. [Figure 13] FIG. 2 is a simplified explanatory diagram illustrating the operation of the image processing device 100. [Figure 14A] FIG. 2 is a simplified explanatory diagram illustrating the operation of the image processing device 100. [Figure 14B] FIG. 2 is a simplified explanatory diagram illustrating the operation of the image processing device 100. [Figure 14C] FIG. 2 is a simplified explanatory diagram illustrating the operation of the image processing device 100. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] [First embodiment] In this embodiment, the hierarchical structure of the teacher model for obtaining the feature vector of input data is divided into a plurality of sub-structures, and the feature vector corresponding to the sub-structure is obtained based on the input data input to the sub-structure and the output data output from the sub-structure based on the input data. Feature Extraction An example of an information processing device that trains a feature extraction unit and uses the trained feature extraction unit to generate a student model having a hierarchical structure for obtaining a feature vector of input data will be described.

[0014] First, an example of the hardware configuration of an information processing device 100 according to this embodiment will be described using the block diagram of Fig. 1. The information processing device 100 according to this embodiment can be applied to computer devices such as PCs (personal computers), smartphones, and tablet terminal devices.

[0015] The CPU 101 executes various processes using computer programs and data stored in the ROM 102 and the RAM 103. As a result, the CPU 101 controls the overall operation of the information processing device 100 and executes or controls various processes that will be described as being performed by the information processing device 100.

[0016] The ROM 102 stores setting data for the information processing device 100, computer programs and data related to the startup of the information processing device 100, computer programs and data related to the basic operation of the information processing device 100, and the like.

[0017] The RAM 103 has an area for storing computer programs and data loaded from the ROM 102 or the external storage device 104, and a work area used when the CPU 101 executes various processes. In this way, the RAM 103 can provide various areas as needed.

[0018] The external storage device 104 is a storage device such as a hard disk drive, a memory card, an optical disk such as a flexible disk (FD) or compact disc (CD) that can be attached to or detached from the information processing device 100, a magnetic or optical card, an IC card, or a memory card. The external storage device 104 stores an OS (operating system), computer programs and data for causing the CPU 101 to execute or control various processes that will be described as being performed by the information processing device 100, and the like. The computer programs and data stored in the external storage device 104 are loaded into the RAM 103 as appropriate under the control of the CPU 101, and become the subject of processing by the CPU 101.

[0019] The CPU 101, ROM 102, RAM 103, and external storage device 104 are all connected to a system bus 108. An input I / F (interface) 105 and an output I / F 106 are also connected to the system bus 108.

[0020] An input unit 109 is connected to the input I / F 105. The input unit 109 is a user interface such as a keyboard, a mouse, or a touch panel, and can input various instructions to the CPU 101 by a user operating the input unit 109.

[0021] A monitor 110 is connected to the output I / F 106. The monitor 110 has a liquid crystal screen or a touch panel screen, and displays the processing results of the CPU 101 as images, text, etc. Note that instead of or in addition to the monitor 110, a projection device such as a projector that projects images and text may be connected to the output I / F 106.

[0022] An example of the functional configuration of such an information processing device 100 is shown in the block diagram of Figure 2. In the following, each functional unit shown in Figure 2 will be described as the subject of processing, but in reality, the function of the functional unit is realized by the CPU 101 executing a computer program that causes the CPU 101 to execute or control the function of the functional unit. Note that one or more of the functional units shown in Figure 2 may be implemented in hardware. The data flow in the configuration shown in Figure 2 is shown in the block diagram of Figure 3.

[0023] The acquisition unit 201 acquires the learning image 301 from the external storage device 104. Note that the image 301 does not necessarily have to be acquired from the external storage device 104, and may instead be received from an external device via communication, for example.

[0024] The extraction unit 202 is a convolutional neural network (CNN). r The feature vector of the image 301 is obtained using a deep network (layered neural network) such as a neural network (NNN). The extraction unit 202 will be described as a teacher model. The parameters of the deep network (weight values, etc.) are parameters that have already been acquired through learning, and these parameters are not changed during the learning process of this embodiment.

[0025] The learning unit 203 includes a dividing unit 204 and N (N is an integer equal to or greater than 2) learning units (learning unit 205-1, ..., learning unit 205-N). The dividing unit 204 divides the layers constituting the deep net used by the extraction unit 202 into N groups. For example, if the deep net used by the extraction unit 202 is composed of 10 layers and the 10 layers are to be divided into four groups, the dividing unit 204 divides the layers into a 2-layer group, a 3-layer group, a 3-layer group, and a 2-layer group, starting from the input layer. The division may be based on any criteria. For example, the dividing unit 204 may divide the layers so that the number of parameters in each group is equal. In this way, the dividing unit 204 divides the hierarchical structure of the deep net used by the extraction unit 202 into multiple sub-structures.

[0026] In this embodiment, the dividing unit 204 divides the layers constituting the deep net used by the extraction unit 202 into N groups, as shown in FIG. 4, and the i-th group (1≦i≦N) is designated as the feature extraction unit 400-i. 301 as input data and outputs intermediate feature vector 401-1 as output data. Feature extraction unit 400-j (2≦j≦N−1) operates using intermediate feature vector 401-(j−1) as input data and outputs intermediate feature vector 401-j as output data. Feature extraction unit 400-N operates using intermediate feature vector 401-(N−1) as input data and outputs feature vector 402 as output data.

[0027] The dividing unit 204 then inputs the data used as input data by the feature extraction unit 400-i as input data 302-i and the data output as output data by the feature extraction unit 400-i as feature vector 303-i to the learning unit 205-i.

[0028] That is, the dividing unit 204 inputs the image 301 used as input data by the feature extraction unit 400-1 as input data 302-1 and the intermediate feature vector 401-1 output by the feature extraction unit 400-1 as output data as feature vector 303-1 to the learning unit 205-1. The dividing unit 204 also inputs the intermediate feature vector 401-1 used as input data by the feature extraction unit 400-2 as input data 302-2 and the intermediate feature vector 401-2 output by the feature extraction unit 400-2 as output data as feature vector 303-2 to the learning unit 205-2. The dividing unit 204 also inputs the intermediate feature vector 401-(N-1) used as input data by the feature extraction unit 400-N as input data 302-N and the feature vector 402 output by the feature extraction unit 400-N as output data as feature vector 303-N to the learning unit 205-N.

[0029] The configuration and operation of learning unit 205-1 will be described below, but the configurations and operations of the other learning units (learning units 205-2 to 205-N) are similar. Feature extraction unit 206-1 is an extraction unit (hierarchical neural network) that operates in the same manner as corresponding feature extraction unit 400-1, but has a configuration that requires less computational effort and parameter data than feature extraction unit 400-1. Feature extraction unit 206-1 obtains feature vector 304-1 from input data 302-1. Note that the number of dimensions of feature vector 303-1 is the same as the number of dimensions of feature vector 304-1; for example, if feature vector 303-1 is a 512-dimensional vector, feature vector 304-1 will also be a 512-dimensional vector.

[0030] The acquisition unit 207-1 obtains a difference value 305-1 between the feature vector 303-1 and the feature vector 304-1. The method for obtaining the difference value 305-1 is not limited to a specific method. For example, the acquisition unit 207-1 obtains a squared value Dk=(Ak-Bk) of the difference between an element Ak (k=1 to K: K is the number of dimensions of the feature vector 303-1) of the feature vector 303-1 and an element Bk of the feature vector 304-1. 2 The acquisition unit 207-1 then divides the sum of the squared values (D1+D2+, . . . , +DK) by K to obtain the value (mean square error) as the difference value 305-1.

[0031] The update unit 208-1 performs error backpropagation to change the parameter 306-1 of the feature extraction unit 206-1 so that the difference value 305-1 becomes smaller. Sowing The parameters of the feature extraction unit 206-1 are calculated by the 306 In other words, the update unit 208-1 performs learning on the feature extraction unit 206-1 so that the difference value 305-1 becomes smaller.

[0032] Determination unit 209-1 determines whether or not a termination condition for learning of feature extraction unit 206-1 by update unit 208-1 has been satisfied. The "termination condition for learning of feature extraction unit 206-1" is not limited to a specific condition, and may be one or more. For example, it may be "difference value 305-1 is equal to or less than a threshold," "amount of change in difference value 305-1 is equal to or less than a threshold," "number of learning times is equal to or greater than a threshold," "elapsed time since start of learning is equal to or greater than a threshold," etc., and one or more of these may be satisfied.

[0033] As described above, the other learning units also perform similar operations to learn the feature extraction unit 206-i. That is, in the learning unit 205-i, the feature extraction unit 206-i obtains the feature vector 304-i from the input data 302-i in the same manner as the corresponding feature extraction unit 400-i. The acquisition unit 207-i obtains the difference value 305-i between the feature vector 303-i and the feature vector 304-i. The update unit 208-i performs error backpropagation to adjust the parameter 306-i of the feature extraction unit 206-i so that the difference value 305-i becomes smaller. Sowing The parameters of the feature extraction unit 206-i are calculated by the 306 -i. That is, the update unit 208-i performs learning of the feature extraction unit 206-i so that the difference value 305-i becomes smaller. The determination unit 209-i determines whether or not the condition for ending learning of the feature extraction unit 206-i by the update unit 208-i has been satisfied.

[0034] When it is determined that the learning termination condition is satisfied in all the determination units (determination units 209-1 to 209-N), integrating unit 210 integrates feature extraction unit 206-1, feature extraction unit 206-2, ..., feature extraction unit 206-N to generate extraction unit 211. For example, as shown in Fig. 5, integrating unit 210 generates, as extraction unit 211, a hierarchical neural network having a configuration in which feature extraction unit 206-1, feature extraction unit 206-2, ..., feature extraction unit 206-N are linked in this order from the input layer side.

[0035] When an image 301 is input to the extraction unit 211, the feature extraction unit 206-1 obtains and outputs an intermediate feature vector from the image 301. The feature extraction unit 206-i (2≦i≦N−1) obtains and outputs another intermediate feature vector from the intermediate feature vector output from the feature extraction unit 206-(i−1). The feature extraction unit 206-N obtains and outputs a feature vector 502 for the image 301 from the intermediate feature vector output from the feature extraction unit 206-(N−1).

[0036] The integration unit 210 may store the extraction unit 211 generated in this manner in the external storage device 104, or may transmit it to an external device via communication, and the output destination of the extraction unit 211 is not limited to a specific output destination.

[0037] Because feature extraction unit 206-i learns to output feature vectors close to the intermediate feature vectors output by corresponding feature extraction unit 400-i, the finally obtained feature vector 502 also has a value close to feature vector 402. Furthermore, feature extraction unit 206-i constituting extraction unit 211 has a smaller amount of calculation and parameter data than the corresponding feature extraction unit 400-i. Therefore, the amount of calculation and parameter data of extraction unit 211 is smaller than that of extraction unit 202.

[0038] In learning the parameters of the extraction unit, the difficulty of learning generally increases as the number of parameters to be learned increases, but in this embodiment, the above configuration can reduce the difficulty of learning. As a result, it becomes possible to easily learn distillation of highly difficult configurations that could not be achieved with conventional methods.

[0039] FIG. 13 is a simplified explanatory diagram of the operation of the information processing device 100 according to this embodiment, and the operation of the information processing device 100 described above will be described again using FIG. 13. FIG. 13 shows a case where the extraction unit 202 has four extraction units (feature extraction units 400-1, 400-2, 400-3, 400-4). The feature extraction units 400-1, 400-2, 400-3, 400-4 have learned parameters, so the parameters are not updated. The feature extraction units 206-1, 206-2, 206-3, 206-4 correspond to the intermediate feature vectors 401-1, 401-2, 401-3, 401-4 (synonymous with 303-1, 303-2, 303-3, 303-4) that are the outputs of the feature extraction units 400-1, 400-2, 400-3, 400-4, respectively. 2 , 206-3, and 206-4, which are the outputs of the feature vectors 304-1, 304-2, 304-3, and 304-4, respectively. 2 , 206-3, and 206-4 update their respective parameters for learning. 2 When all learning in the feature extraction units 206-1, 206-2, and 206-3 is completed, 2 , 206-3, and 206-4 are integrated to obtain the extraction unit 211. 2 , 206-3, 206-4 have a smaller amount of calculation and parameter data than feature extraction units 400-1, 400-2, 400-3, 400-4, so the amount of calculation and parameter data of extraction unit 211 is smaller than extraction unit 202.

[0040] 14A to 14C are diagrams illustrating the explanations of FIGS. 2 to 5 using a neural network configuration. FIG. 14 shows a case where extraction unit 202 has N extraction units (feature extraction units 400-1 to 400-N). FIG. 14A illustrates an attempt to simplify the neural network of feature extraction unit 400-1 to feature extraction unit 206-1. In other words, an attempt is made to partially simplify extraction unit 202. Update unit 208-1 updates the parameters of feature extraction unit 206-1 so as to reduce difference value 305-1 between intermediate feature vector 401-1 (synonymous with 303-1), which is the output of feature extraction unit 400-1, and the corresponding feature vector 304-1, which is the output of feature extraction unit 206-1. Learning is terminated when determination unit 209-1 determines that difference value 305-1 has become smaller than a predetermined value. FIG. 14B shows how the same processing as in FIG. 14A is performed on feature extraction unit 400-i (each group divided by division unit 204). When learning is completed for an extraction unit to be replaced with a student model among extraction units 202 as a teacher model, feature extraction units 206-1 to 206-N are integrated to obtain extraction unit 211. All of feature extraction units 400-1 to 400-N can be replaced with feature extraction units 206-1 to 206-N, or they can be partially replaced. FIG. 14C shows how feature extraction units 206-1 to 206-N are integrated.

[0041] The process performed by the information processing device 100 to generate the extraction unit 211 will be described with reference to the flowchart of Fig. 6. In step S601, the acquisition unit 201 acquires the image 301 for learning.

[0042] In step S602, integration unit 210 determines whether all determination units (determination units 209-1 to 209-N) have determined that the learning termination condition has been satisfied. If the result of this determination is that all determination units (determination units 209-1 to 209-N) have determined that the learning termination condition has been satisfied, the process proceeds to step S608. On the other hand, if there is one or more determination units among determination units 209-1 to 209-N that have not determined that the learning termination condition has been satisfied, the process proceeds to step S603.

[0043] In step S603, the extraction unit 202 obtains a feature vector 402 of the image 301 using a deep network such as a CNN. The division unit 204 divides the multiple layers constituting the deep network into N groups (feature extraction units 400-1 to 400-N). Then, for i = 1 to N, the division unit 204 acquires, as input data 302-i, the data used as input data by the feature extraction unit 400-i, and as feature vector 303-i, the data output as output data by the feature extraction unit 400-i, and inputs the input data 302-i and the feature vector 303-i to the learning unit 205-i. Then, the processes of steps S604 to S607 are performed for i = 1 to N.

[0044] In step S605, the determination unit 209-i determines whether the learning termination condition of the feature extraction unit 206-i is satisfied. If the result of this determination is that the learning termination condition of the feature extraction unit 206-i is satisfied, the process proceeds to step S607, and if the learning termination condition of the feature extraction unit 206-i is not satisfied, the process proceeds to step S606.

[0045] In step S606, the learning unit 205-i learns the feature extraction unit 206-i. Details of the processing in step S606 will be described with reference to the flowchart in Fig. 7. In step S701, the feature extraction unit 206-i acquires the input data 302-i input from the division unit 204. In step S702, the acquisition unit 207-i acquires the feature vector 303-i input from the division unit 204.

[0046] In step S703, the feature extraction unit 206-i operates in the same manner as the corresponding feature extraction unit 400-i to obtain (acquire) the feature vector 304-i from the input data 302-i. In step S704, the acquisition unit 207-i obtains the difference value 305-i between the feature vector 303-i and the feature vector 304-i.

[0047] In step S705, the update unit 208-i performs error backpropagation to update the parameter 306-i of the feature extraction unit 206-i so that the difference value 305-i becomes smaller. SowingThe parameters of the feature extraction unit 206-i are calculated by the 306 In step S706, the determination unit 209-1 determines whether or not the condition for ending the learning of the feature extraction unit 206-1 by the update unit 208-1 has been satisfied.

[0048] Returning to FIG. 6, in step S608, the integrating unit 210 generates the extraction unit 211 by linking (integrating) the feature extraction units 206-1, 206-2, . . . , 206-N in this order.

[0049] As described above, according to this embodiment, the extraction unit 202 as the teacher model is divided and distilled, and finally integrated to generate the extraction unit 211 as the student model. This makes it possible to make the feature vectors of the student model and the teacher model nearly identical, even in the case of highly difficult distillation that could not be achieved with conventional methods.

[0050] <Variation 1> In the first embodiment, feature extraction unit 206-i is trained based on difference value 305-i between feature vector 303-i and feature vector 304-i, but the training method of feature extraction unit 206-i is not limited to this. For example, as shown in Fig. 8, learning unit 205-i configures an extraction unit (replaced hierarchical structure) in which feature extraction unit 400-i (substructure of interest) is replaced with feature extraction unit 206-i, and inputs image 301 to the configured extraction unit to obtain feature vector 801-i. Learning unit 205-i then calculates the difference value between feature vector 801-i and feature vector 402 in the same manner as difference value 305-i, and trains feature extraction unit 206-i so that difference value 801-i becomes smaller.

[0051] <Variation 2> In the first embodiment, all of feature extraction units 206-1 to 206-N are generated by learning to configure extraction unit 211. However, some (single or multiple) feature extraction units 206-j (1≦j≦N) of feature extraction units 206-1 to 206-N do not need to be generated by learning. In this case, integration unit 210 configures extraction unit 211 by using feature extraction unit 400-j as the integration target instead of feature extraction unit 206-j.

[0052] In this way, in extraction unit 211 generated by this modification, the amount of calculations and the amount of parameter data for feature extraction unit 400-i are the same as those for extraction unit 202, but the amount of calculations and the amount of parameter data for feature extraction unit 206-i are less than those for extraction unit 202. Therefore, ultimately, the amount of calculations and the amount of parameter data for extraction unit 211 are less than those for extraction unit 202, and it becomes possible to make the feature vectors of the student model and the teacher model approximately identical even in highly difficult distillation that was difficult to achieve with conventional methods.

[0053] [Second embodiment] In this embodiment, differences from the first embodiment will be described, and unless otherwise specified below, it is assumed to be the same as the first embodiment. In this embodiment, the extraction unit 211 is retrained using the feature vector 402 output from the extraction unit 202 as a teacher model and the feature vector 502 output from the extraction unit 211 as a student model.

[0054] An example of the functional configuration of an information processing device 100 according to this embodiment is shown in the block diagram of FIG. 9. In the following, each functional unit shown in FIG. 9 will be described as the subject of processing, but in reality, the function of the functional unit is realized by the CPU 101 executing a computer program that causes the CPU 101 to execute or control the function of the functional unit. Note that one or more of the functional units shown in FIG. 9 may be implemented in hardware. The configuration shown in FIG. 9 is the configuration shown in FIG. 2 to which a relearning unit 901 has been added. An example of the functional configuration of the relearning unit 901 is shown in the block diagram of FIG. 10. The data flow in the configurations shown in FIGS. 9 and 10 is shown in the block diagram of FIG. 11.

[0055] The re-learning unit 901 uses the feature vector 402 output from the extraction unit 202 and the feature vector 502 output from the extraction unit 211 to calculate the parameter 1101 of the extraction unit 211 by backpropagation. Sowing Then, the re-learning unit 901 changes the parameters of the extraction unit 211 to the obtained parameters 1101 (learning (re-learning) of the extraction unit 211 is performed).

[0056] The operation of the relearning unit 901 will be described in more detail. Fig. 12 is a diagram illustrating the operation of the relearning unit 901. As described above, the acquisition unit 201 acquires the image 301, the extraction unit 202 finds and outputs the feature vector 402 from the image 301, and the extraction unit 211 finds and outputs the feature vector 502 from the image 301.

[0057] The acquisition unit 1001 acquires the feature vector 402 output from the extraction unit 202 and the feature vector 502 output from the extraction unit 211. The acquisition unit 1001 then calculates a difference value 1201 between the feature vector 402 and the feature vector 502 (for example, the mean square error between the feature vector 402 and the feature vector 502).

[0058] The update unit 1002 updates the parameters 1101 of the extraction unit 211 so that the difference value 1201 becomes smaller by backpropagation. Sowing The parameters of the extraction unit 211 are updated to the obtained parameters 1101 (that is, the extraction unit 211 is trained (re-trained)).

[0059] The determination unit 1003 determines whether or not the termination condition for the learning of the extraction unit 211 set by the update unit 1002 has been satisfied. The "termination condition for the learning of the extraction unit 211" is not limited to a specific condition, and may be one or more. For example, it may be "the difference value 1201 is equal to or less than a threshold," "the amount of change in the difference value 1201 is equal to or less than a threshold," "the number of learning times is equal to or greater than a threshold," "the elapsed time since the start of learning is equal to or greater than a threshold," etc., and it may be one of these, or two or more of these. Then, when the determination unit 1003 determines that the "termination condition for the learning of the extraction unit 211" has been satisfied, the relearning unit 901 terminates the learning (relearning) of the extraction unit 211.

[0060] The ultimate goal of distillation is to ensure that when the same data as input to the teacher model is input to the student model, the output data from the student model is similar to the output data from the teacher model. Extraction unit 211 is an integration of feature extraction unit 400-i and feature extraction unit 206-i, which outputs feature vectors similar to feature extraction unit 400-i. If feature vectors 304-1 to 304-N output by feature extraction units 206-1 to 206-N differ from feature vectors 303-1 to 303-N output by feature extraction units 400-1 to 400-N, the difference between feature vector 402 and feature vector 502 may become large. In this case, if re-learning unit 901 trains (re-learns) extraction unit 211 so that difference value 1201 becomes smaller, feature vector 402 and feature vector 502 will eventually become similar in value, thereby achieving the goal of distillation. Note that learning units 205-1 to 205-N learn in advance the parameters that configure extraction unit 211, thereby enabling re-learning unit 901 to learn. That is, because pre-learning allows re-learning unit 901 to perform only fine adjustment of the parameters, the difficulty of distillation by extraction unit 211 can be reduced.

[0061] Thus, according to this embodiment, even in highly difficult distillation that was difficult to achieve using conventional methods, it is possible to make the feature vectors of the student model and the teacher model approximately identical with even greater accuracy than in the first embodiment.

[0062] In addition, the relearning unit 901 performed learning by distillation, with the extraction unit 202 as the teacher model and the extraction unit 211 as the student model. However, the learning by the relearning unit 901 is not limited to distillation, and similar effects can be obtained by a general learning method in which learning is performed so that the feature vector 502 approaches the correct vector associated with the image 301. Therefore, the learning method by the relearning unit 901 is not limited to distillation.

[0063] Furthermore, the numerical values, processing timing, processing order, processing subject, destination / source / storage location of data (information) used in each of the above embodiments and each of the modified examples are given as examples to provide a concrete explanation, and are not intended to be limited to such examples.

[0064] Furthermore, some or all of the above-described embodiments and modifications may be used in appropriate combination, and some or all of the above-described embodiments and modifications may be used selectively.

[0065] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0066] The invention of this specification includes the following information processing device, information processing method, and computer program. (Item 1) a learning means for dividing a hierarchical structure of a teacher model for obtaining a feature vector of input data into a plurality of sub-structures, and for learning a feature extraction unit corresponding to each sub-structure based on input data input to each sub-structure and output data output from each sub-structure based on the input data; a generation means for generating a student model having a hierarchical structure for obtaining a feature vector of input data using the feature extraction unit trained by the learning means; An information processing device comprising: (Item 2) The information processing device described in item 1, characterized in that the learning means learns the feature extraction unit based on output data output from the feature extraction unit corresponding to the substructure based on input data input to the substructure, and output data output from the substructure based on the input data. (Item 3) The information processing device described in item 1 is characterized in that the learning means constructs a replaced hierarchical structure in which a substructure of interest in the hierarchical structure is replaced with a feature extraction unit corresponding to the substructure of interest, and learns the feature extraction unit corresponding to the substructure of interest based on the output of the replaced hierarchical structure for input data and the output of the hierarchical structure for the input data. (Item 4) 4. The information processing device according to any one of items 1 to 3, wherein the generation means generates the student model by linking trained feature extraction units corresponding to each of the plurality of substructures. (Item 5) The information processing device described in any one of items 1 to 3, characterized in that the generation means generates the student model by linking a feature extraction unit that has been learned by the learning means for some of the plurality of substructures and substructures of the plurality of substructures excluding the some of the substructures. (Item 6) moreover, An information processing device described in any one of items 1 to 5, characterized in that it comprises a re-learning means for re-learning the student model based on the output of the teacher model for input data and the output of the student model for the input data. (Item 7) moreover, An information processing device described in any one of items 1 to 5, characterized in that it comprises a re-learning means for re-learning the student model based on the output of the teacher model for input data and data associated with the input data. (Item 8) 8. The information processing device according to any one of items 1 to 7, wherein the feature extraction unit has a configuration in which the amount of calculation is less than that of the sub-structure corresponding to the feature extraction unit. (Item 9) 9. The information processing device according to any one of items 1 to 8, wherein the feature extraction unit has a configuration in which the amount of parameter data is smaller than that of the substructure corresponding to the feature extraction unit. (Item 10) An information processing method performed by an information processing device, a learning step in which a learning means of the information processing device divides a hierarchical structure of a teacher model for obtaining a feature vector of input data into a plurality of sub-structures, and learns a feature extraction unit corresponding to each sub-structure based on input data input to each sub-structure and output data output from each sub-structure based on the input data; a generation step in which a generation means of the information processing device generates a student model having a hierarchical structure for obtaining a feature vector of input data, using a feature extraction unit that has been trained in the learning step; An information processing method comprising: (Item 11) A computer program for causing a computer to function as each of the means of the information processing device according to any one of items 1 to 9.

[0067] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0068] 100: Information processing device 201: Acquisition unit 202: Extraction unit 203: Learning unit 204: Division unit 205-1: Learning unit 205-2: Learning unit 205-3: Learning unit 205-N: Learning unit 206-1: Extraction unit 207-1: Acquisition unit 208-1: Update unit 209-1: Determination unit 210: Integration unit 211: Extraction unit

Claims

1. a replacement hierarchical structure in which a hierarchical structure of a teacher model for determining a feature vector of input data is divided into a plurality of substructures, and for each substructure of interest in the hierarchical structure, the replacement hierarchical structure is formed by replacing the substructure of interest with a feature extraction unit corresponding to the substructure of interest, the replacement hierarchical structure being constructed using the feature extraction unit and a substructure other than the substructure of interest in the hierarchical structure of the teacher model, and a learning means for learning the feature extraction unit corresponding to each substructure of interest based on an output of a final layer of the replacement hierarchical structure for the input data and an output of a final layer of the teacher model for the input data; a generation means for generating a student model having a hierarchical structure for obtaining a feature vector of input data using the feature extraction unit trained by the learning means; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the generating means generates the student model by linking trained feature extraction units corresponding to the plurality of substructures.

3. The information processing device according to claim 1, characterized in that the generation means generates the student model by linking feature extraction units that have been trained by the learning means for some of the plurality of substructures with substructures excluding the some of the substructures among the plurality of substructures.

4. moreover, 2. The information processing apparatus according to claim 1, further comprising a re-learning means for re-learning the student model based on an output of the teacher model for input data and an output of the student model for the input data.

5. moreover, 2. The information processing apparatus according to claim 1, further comprising a re-learning means for re-learning the student model based on the output of the teacher model for input data and data associated with the input data.

6. 2. The information processing apparatus according to claim 1, wherein the feature extraction unit has a configuration in which the amount of calculation is less than that of the sub-structure corresponding to the feature extraction unit.

7. 2. The information processing apparatus according to claim 1, wherein the feature extraction unit has a configuration in which the amount of parameter data is smaller than that of the substructure corresponding to the feature extraction unit.

8. a student model having a hierarchical structure for determining a feature vector of input data using a trained feature extraction unit obtained by a learning means which divides the hierarchical structure of a teacher model for determining a feature vector of input data into a plurality of substructures, and replaces, for each substructure of interest in the hierarchical structure, the substructure of interest with a feature extraction unit corresponding to the substructure of interest, and which configures the replacement hierarchical structure using the feature extraction unit and substructures other than the substructure of interest in the hierarchical structure of the teacher model, and trains the feature extraction unit corresponding to each substructure of interest based on an output of the final layer of the replacement hierarchical structure for the input data and an output of the final layer of the teacher model for the input data; an output means for outputting a feature vector for an image input to the student model; An information processing device comprising:

9. An information processing method performed by an information processing device, a learning step in which a learning means of the information processing device divides a hierarchical structure of a teacher model for determining a feature vector of input data into a plurality of substructures, constructs a replacement hierarchical structure in which, for each substructure of interest in the hierarchical structure, the substructure of interest is replaced with a feature extraction unit corresponding to the substructure of interest, the replacement hierarchical structure using the feature extraction unit and a substructure other than the substructure of interest in the hierarchical structure of the teacher model, and learns the feature extraction unit corresponding to each substructure of interest based on an output of a final layer of the replacement hierarchical structure for the input data and an output of a final layer of the teacher model for the input data; a generation step in which a generation means of the information processing device generates a student model having a hierarchical structure for obtaining a feature vector of input data, using a feature extraction unit that has been trained in the learning step; An information processing method comprising:

10. An information processing method performed by an information processing device, an output means of the information processing device outputs a feature vector for the image input to the student model; The student model is a replacement hierarchical structure in which the hierarchical structure of a teacher model for determining a feature vector of input data is divided into a plurality of substructures, and for each substructure of interest in the hierarchical structure, the substructure of interest is replaced with a feature extraction unit corresponding to the substructure of interest, the replacement hierarchical structure being constructed using the feature extraction unit and a substructure other than the substructure of interest in the hierarchical structure of the teacher model, and the hierarchical structure is used to determine a feature vector of input data using a trained feature extraction unit obtained by a learning means which configures the replacement hierarchical structure using the feature extraction unit and a substructure other than the substructure of interest in the hierarchical structure of the teacher model, and which trains the feature extraction unit corresponding to each substructure of interest based on an output of the final layer of the replacement hierarchical structure for the input data and an output of the final layer of the teacher model for the input data. An information processing method comprising:

11. A computer program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 8.

Images