Machine learning program, machine learning method, and information processing device

Abstract

This information processing device executes first training for training a feature converter that converts a first feature amount used for prediction in a trained first machine learning model having a first number of parameters into a second feature amount, and a predictor of a second machine learning model that includes a feature generator and a predictor and has a second number of parameters smaller than the first number of parameters, the training being performed by using an output result obtained by inputting the second feature amount converted by the feature converter into the predictor. The information processing device executes second training for training the feature generator on the basis of the conversion result converted by the trained feature converter and a generation result generated by the feature generator of the second machine learning model.

Description

Machine Learning Program, Machine Learning Method, and Information Processing Device

[0001] The present invention relates to a machine learning program, a machine learning method, and an information processing device.

[0002] For example, in an environment where new detection target objects such as object detection of products in retailers such as supermarkets occur daily, an object detection model that shows high detection performance for unlearned objects outside the learning data is required. Note that as an object detection model, a neural network model composed of a feature acquisition unit (Backbone) and a prediction unit (neck / head) that predicts an object region and the class to which the object belongs is known.

[0003] Although a large-scale image recognition model with high generalization performance that can also recognize unlearned objects is known, it is not suitable for real-time processing in an environment with limited computing resources. Therefore, in recent years, techniques using knowledge distillation, feature distillation, etc. have been utilized.

[0004] For example, knowledge distillation is a learning method that transfers the "knowledge" learned by a certain learning model (teacher model) to a smaller-scale learning model (student model). By transferring the "knowledge" of a large-scale image recognition model to a lighter object detection model, it is expected to obtain a lightweight object detection model that can detect unlearned objects with high accuracy. For example, feature distillation is a method that causes the student model to imitate the feature representation of the teacher model and learn about the object detection task.

[0005] Japanese Patent Application Laid-Open No. 2023-28233

[0006] By the way, in the above distillation technology, since the feature representation suitable for prediction by the student model is different from the feature representation of the teacher model, a feature converter for filling the difference is required. Since the feature converter is learned to fill the difference in the feature representations output by the teacher model and the student model, the feature representation output by the student model is required for its learning.

[0007] Therefore, in the distillation technique described above, the feature converter and the student model are updated simultaneously, meaning that the incomplete feature converter, which is still in the learning process, affects the learning of the student model. As a result, the parameters of the student model progress in a direction different from the global optimal solution, leading to a local optimal solution, and thus degrading the detection performance of the student model.

[0008] One aspect of this project is to provide a machine learning program, a machine learning method, and an information processing device that can improve the detection accuracy of untrained objects.

[0009] In the first proposal, the machine learning program is characterized by causing a computer to perform a first training, in which a feature converter converts a first feature quantity used for prediction in a first trained machine learning model having a first number of parameters into a second feature quantity, and a predictor of the second machine learning model, which includes a feature generator and a predictor and has a second number of parameters less than the first number of parameters, is trained using the output result obtained by inputting the second feature quantity converted by the feature converter into the predictor, and a second training, in which the feature generator is trained based on the conversion result converted by the trained feature converter and the generation result generated by the feature generator of the second machine learning model.

[0010] According to one embodiment, the detection accuracy of untrained objects can be improved.

[0011] Figure 1 is a diagram illustrating the information processing device according to Example 1. Figure 2 is a diagram illustrating general feature distillation. Figure 3 is a diagram illustrating the problems of general feature distillation. Figure 4 is a diagram illustrating the processing of the information processing device according to Example 1. Figure 5 is a diagram illustrating the effects of the information processing device according to Example 1. Figure 6 is a functional block diagram showing the functional configuration of the information processing device according to Example 1. Figure 7 is a diagram illustrating the first training. Figure 8 is a diagram illustrating pattern 1 of the second training. Figure 9 is a diagram illustrating pattern 2 of the second training. Figure 10 is a diagram illustrating pattern 3 of the second training. Figure 11 is a diagram illustrating the object detection processing after training. Figure 12 is a flowchart illustrating the overall processing flow of the machine learning process. Figure 13 is a flowchart illustrating the detailed processing flow of the machine learning process. Figure 14 is a flowchart illustrating the detailed processing flow of the machine learning process. Figure 15 is a flowchart illustrating the detailed processing flow of the machine learning process according to Example 2. Figure 16 is a flowchart illustrating the detailed processing flow of the machine learning process according to Example 2. Figure 17 is a diagram illustrating an example of hardware configuration.

[0012] The following describes in detail, with reference to the drawings, embodiments of the machine learning program, machine learning method, and information processing device disclosed in this application. However, the present invention is not limited by these embodiments. Each embodiment can be combined as appropriate within a non-consistent range.

[0013] <Overall Configuration> Figure 1 is a diagram illustrating the information processing device 10 according to Embodiment 1. The information processing device 10 shown in Figure 1 is an example of a computer that acquires a lightweight object detection model capable of detecting untrained objects with high accuracy by performing knowledge distillation, which transfers the knowledge of a trained large-scale learning model with a large number of parameters to a small-scale learning model with fewer parameters than the large-scale learning model.

[0014] The large-scale learning model is an example of a neural network model used for object detection, and in this embodiment, it will be referred to as the teacher model. The small-scale learning model is also an example of a neural network model used for object detection, and in this embodiment, it will be referred to as the student model. Both the teacher model and the student model consist of a feature generator (backbone) and a predictor (neck / head) that predicts object regions and the class to which objects belong. In this embodiment, an object detection model is used as an example, but the model is not limited to this and can be applied to other models such as image recognition models.

[0015] As shown in Figure 1, the information processing device 10 includes a teacher model, a feature converter, and a student model. The teacher model is a pre-trained model and includes a feature generator that generates features from image data and a predictor that performs object detection in response to the features generated by the feature generator. The student model is a model to be trained, from which the knowledge of the teacher model is distilled, and includes a feature generator that generates features from image data and a predictor that performs object detection in response to the features generated by the feature generator. The feature converter is a linear layer or a linear transformation model that transforms features to bridge the gap between the feature representation suitable for prediction by the student model and the feature representation of the teacher model.

[0016] As described above, in the method of training a student model to mimic the feature representations of the teacher model while simultaneously learning the object detection task (feature distillation), a feature converter is needed to bridge the gap between the feature representations suitable for prediction by the student model and the feature representations of the teacher model. Since this feature converter is trained to bridge the gap between the feature representations output by the teacher model and the feature representations output by the student model, the feature representations output by the student model are necessary for its training.

[0017] Here, we will explain general feature distillation. Figure 2 is a diagram illustrating general feature distillation, and Figure 3 is a diagram illustrating the problems of general feature distillation.

[0018] As shown in Figure 2, in typical feature distillation, the feature converter is trained so that the error (distance loss) between the features generated by the teacher model's feature generator from the training data (image data) and the features converted by the feature converter (student model feature representation) are minimized, and the features generated by the student model's feature generator from the training data (student model feature representation) are minimized. At the same time, the student model's predictor is trained so that the error (object detection loss) between the output of the student model's predictor, which is input with the features generated by the student model's feature generator from the training data (student model feature representation), and the correct information from the training data is minimized.

[0019] In other words, as shown in Figure 3, the feature converter and the student model interact with each other as training progresses. As a result, the incomplete feature converter acts on the parameters of the student model, causing those parameters to be updated in a direction different from the global optimal solution. Consequently, the information processing device 10 causes the parameters of the student model to converge to a local optimal solution, and the detection accuracy of the student model decreases.

[0020] Therefore, the information processing device 10 according to Embodiment 1 trains the feature converter and the student model in stages to converge the parameters of the student model to the globally optimal solution and suppress the decrease in the detection accuracy of the student model. In other words, since the function that the feature converter should have is to project the feature representation of the teacher model to a representation suitable for the student model's predictor, the information processing device 10 trains the feature converter and the student model's feature generator independently.

[0021] Figure 4 is a diagram illustrating the processing of the information processing device 10 according to Embodiment 1, and Figure 5 is a diagram illustrating the effects of the information processing device 10 according to Embodiment 1. As shown in Figure 4, the information processing device 10 performs first training and second training to realize stepwise training of the feature converter and the student model.

[0022] For example, as the first training, the information processing device 10 uses the feature generator of the trained teacher model to train the feature converter and the predictor of the student model. Specifically, the information processing device 10 acquires images and teacher labels (ground truth information) to generate training data, and inputs the training data into the feature generator of the teacher model to generate features. Then, the information processing device 10 inputs the features into the predictor 1 of the student model and obtains the prediction result (output result) of the predictor 1.

[0023] Subsequently, the information processing device 10 calculates the object detection loss using the prediction results of the student model predictor 1 and the teacher labels, and uses the calculated loss to train the feature converter and the student model predictor 1 (update 1).

[0024] In other words, the information processing device 10 connects the student model's predictor to the feature converter and optimizes the object detection loss, thereby learning how to convert features into representations suitable for object detection by the student model's predictor, and acquiring the function of projecting the teacher model's feature representations into representations suitable for the student model's predictor.

[0025] Once the first training is complete, the information processing device 10 performs a second training, using the trained feature converter to train the student model's feature generator. At this time, the information processing device 10 can further improve the accuracy of the student model by training the student model's predictor. Specifically, the information processing device 10 inputs the training data into the teacher model's feature generator to generate features, and inputs these features into the student model's predictor 1 to obtain prediction result 1. Similarly, the information processing device 10 inputs the training data into the student model's feature generator to generate features, and inputs these features into the student model predictor 2, which has the parameters of predictor 1 generated in the first training set, to obtain prediction result 2.

[0026] Subsequently, the information processing device 10 calculates the distance loss 1 between the output (feature quantity) of the feature converter and the output (feature quantity) of the student model's feature generator. The information processing device 10 also calculates the distance loss 2 between the prediction result 1 of the student model's predictor 1 and the prediction result 2 of the student model's predictor 2. The information processing device 10 also calculates the object detection loss using the prediction result 2 of the student model's predictor 2 and the teacher label. Then, the information processing device 10 trains the student model's feature generator to minimize the distance loss 1, and trains the student model's predictor 2 to minimize the distance loss 2 and the object detection loss (update 2).

[0027] In other words, the information processing device 10 trains the student model's feature generator to mimic a feature representation suitable for the student model's predictor. At this time, since it is difficult to make the student model's feature generator perfectly mimic the converted features, the information processing device 10 also trains the predictor (student model's predictor 2) in conjunction with the feature generator.

[0028] In other words, as shown in Figure 5, the information processing device 10 performs training on the student model by applying a pre-trained feature converter. As a result, the complete feature converter acts on the parameters of the student model, causing those parameters to be updated toward the global optimal solution. Therefore, the information processing device 10 can converge the parameters of the student model toward the global optimal solution, thereby suppressing a decrease in the detection accuracy of the student model.

[0029] <Functional Configuration> Figure 6 is a functional block diagram showing the functional configuration of the information processing device 10 according to Embodiment 1. As shown in Figure 6, the information processing device 10 has a communication unit 11, an output unit 12, a storage unit 13, and a control unit 20.

[0030] The communication unit 11 is a processing unit that controls communication with other devices, and is implemented, for example, by a communication interface. For example, the communication unit 11 receives various instructions (for example, instructions to start training) from an administrator, receives image data and training data from an external terminal, and transmits training results and other information to a designated device.

[0031] The output unit 12 is a processing unit that displays and outputs various types of information, and is implemented by, for example, a display or a touch panel. For example, the output unit 12 displays and outputs the status during training, training results, etc.

[0032] The memory unit 13 is a processing unit that stores various data and programs executed by the control unit 20, and is implemented by, for example, memory or a hard disk. For example, the memory unit 13 stores training data 14, a teacher model 15, a feature converter 16, and a student model 17.

[0033] The training data 14 includes the training data for the feature converter 16, the training data for the feature generator 17a of the student model 17, and the training data for the predictor 17b of the student model 17. Further details will be described later.

[0034] The teacher model 15 is a pre-trained large-scale learning model as described in Figures 1 and 4, and has a feature generator 15a and a predictor 15b. The patent converter 16 is a feature converter as described in Figures 1 and 4, and converts the features of the teacher model 15 into the features of the student model 17. The student model 17 is a small-scale learning model that is the target of training as described in Figures 1 and 4, and has a feature generator 17a and a predictor 17b.

[0035] The control unit 20 is a processing unit that oversees the entire information processing device 10 and is implemented by, for example, a processor. This control unit 20 includes a first training unit 21, a second training unit 22, and an object detection unit 23. The first training unit 21, the second training unit 22, and the object detection unit 23 are implemented by, for example, electronic circuits of the processor or processes executed by the processor.

[0036] (Explanation of the first training) The first training unit 21 is a processing unit that uses a pre-trained teacher model 15 to train the feature converter 16 and the predictor 17b of the student model 17. Specifically, the first training unit 21 performs the first training by inputting the second feature quantities converted by the feature converter 16 into the predictor 17b and using the output results obtained. This training is performed using the feature converter 16, which converts the first feature quantities used for prediction in the teacher model 15 having a first number of parameters into second feature quantities, and the predictor 17b of the student model 17 having a second number of parameters that is less than the first number of parameters.

[0037] Figure 7 illustrates the first training. As shown in Figure 7, the first training unit 21 inputs training data, which includes image data and teacher labels containing the location and class of objects within the image data, into the feature generator 15a of the teacher model 15 to obtain "feature X," which is an example of a first feature. Next, the first training unit 21 inputs "feature X" into the feature converter 16 to obtain "feature Y1," which is an example of a second feature. Then, the first training unit 21 inputs "feature Y1" into the predictor 17b of the student model 17 to obtain "prediction result (location, class)," which is the object prediction result of the predictor 17b.

[0038] Subsequently, the first training unit 21 performs training on the feature converter 16 with feature X as the explanatory variable and the teacher label as the target variable, and performs training on the predictor 17b of the student model 7 with feature Y1 as the explanatory variable and the teacher label as the target variable. Specifically, the first training unit 21 updates the parameters of the feature converter 16 and the predictor 17b of the student model 17 so as to minimize the error (object detection loss) between the teacher label of the training data and the prediction result 1.

[0039] As described above, the first training unit 21 uses training data with teacher labels to perform supervised learning on the feature converter 16 and the predictor 17b of the student model 17 to minimize the loss of object detection.

[0040] (Explanation of the second training) The second training unit 22 is a processing unit that performs training on the student model 17 using the trained feature converter 16.

[0041] (Second training pattern 1) Specifically, the second training unit 22 executes second training pattern 1, which trains the feature generator 17a based on a feature quantity that is an example of a conversion result converted by the trained feature converter 16 and a feature quantity that is an example of a generation result generated by the feature generator 17a of the student model 17.

[0042] Figure 8 illustrates Pattern 1 of the second training. The "student model 17 predictor 17b" shown in Figure 8 may be the same predictor 17b that was trained in the first training, or it may be a different predictor with the parameters of predictor 17b set.

[0043] As shown in Figure 8, the second training unit 22 inputs the training data into the feature generator 15a of the teacher model 15 to obtain "feature X", and then inputs "feature X" into the trained feature converter 16 to obtain "feature Y2". Similarly, the second training unit 22 inputs the training data into the feature generator 17a of the student model 17 to obtain "feature Y3", and then inputs "feature Y3" into the predictor 17b of the student model 17 to obtain "prediction result 2".

[0044] Subsequently, the second training unit 22 performs training on the feature generator 17a of the student model 17, using image data as the explanatory variable and feature Y2 as the target variable. Specifically, the second training unit 22 updates the parameters of the feature generator 17b to minimize the error (distance loss) between the output result "feature Y2" of the trained feature converter 16 and the output result (feature Y3) of the feature generator 17a of the student model 17 being trained.

[0045] In other words, the second training unit 21 uses the trained feature converter 16 to perform supervised learning on the feature generator 17a to minimize distance loss.

[0046] (Pattern 2 of the second training) For example, in addition to Pattern 1 above, the second training unit 22 can also perform training on the predictor 17b of the student model 17 using object loss (Pattern 2). Specifically, the second training unit 22 updates the parameters of the predictor 17b trained in the first training using the output results obtained by inputting the feature quantities, which are the results generated by the feature generator 17a of the student model 17, into the predictor 17b of the student model 17.

[0047] Figure 9 illustrates pattern 2 of the second training. The "student model 17 predictor 17b" shown in Figure 9 may be the same predictor 17 that was trained in the first training, or it may be a different predictor with the parameters of predictor 17 set.

[0048] As shown in Figure 9, the second training unit 22 inputs the training data into the feature generator 15a of the teacher model 15 to obtain "feature X", and then inputs "feature X" into the trained feature converter 16 to obtain "feature Y2". The second training unit 22 also inputs the training data into the feature generator 17a of the student model 17 to obtain "feature Y3", and then inputs "feature Y3" into the predictor 17b of the student model 17 to obtain "prediction result 2".

[0049] Subsequently, the second training unit 22 performs training on the feature generator 17a of the student model 17, using image data as the explanatory variable and the feature Y2 as the target variable, and performs training on the predictor 17b of the student model 17, using image data as the explanatory variable and the teacher label as the target variable.

[0050] Specifically, the second training unit 22 updates the parameters of the feature generator 17b so as to minimize the error (distance loss) between the output result "feature Y2" of the trained feature converter 16 and the output result (feature Y3) of the feature generator 17a of the student model 17 being trained. The second training unit 22 also updates the parameters of the predictor 17b of the student model 17 so as to minimize the error (object detection loss) between the teacher labels of the training data and the prediction result 2.

[0051] In other words, the second training unit 21 uses the trained feature converter 16 to perform supervised learning on the feature generator 17a to minimize distance loss, and also performs supervised learning on the student model 17's predictor 17b to minimize object detection loss.

[0052] (Pattern 3 of the second training) For example, in addition to Pattern 1 or Pattern 2 above, the second training unit 22 can also perform training using distance loss on the predictor 17b of the student model 17 (Pattern 3). Specifically, the second training unit 22 inputs the feature quantities, which are the conversion results of the trained feature converter 16 trained in the first training, into the predictor 17b (predictor 1) of the trained student model 17 trained in the first training, and obtains a first output result. The second training unit 22 inputs the feature quantities, which are the generation results generated by the feature generator 17a of the student model 17, into the predictor 17b (predictor 2), which has the parameters of the trained predictor 17b set, and obtains a second output result. Then, the second training unit 22 updates the parameters of the predictor 17b (predictor 2) so that the error between the first output result and the second output result becomes small.

[0053] Figure 10 illustrates pattern 3 of the second training. The "student model 17 predictor 17b" shown in Figure 10 is another predictor with the parameters of predictor 17, which was trained in the first training, set. For explanatory purposes, the predictor 17b trained in the first training will be referred to as predictor 1, and the predictor 17b being trained in the second training will be referred to as predictor 2.

[0054] As shown in Figure 10, the second training unit 22 inputs the training data into the feature generator 15a of the teacher model 15 to obtain "feature X", and then inputs "feature X" into the trained feature converter 16 to obtain "feature Y2". Subsequently, the second training unit 22 inputs "feature Y2" into the predictor 1 of the student model 17 to obtain "prediction result 1", which is the object prediction result of the predictor 1.

[0055] Similarly, the second training unit 22 inputs the training data into the feature generator 17a of the student model 17 to obtain "feature Y3", and inputs "feature Y3" into the predictor 2 of the student model 17 to obtain "prediction result 2".

[0056] Subsequently, the second training unit 22 performs training on the feature generator 17a of the student model 17, with image data as the explanatory variable and feature Y2 as the target variable. The second training unit 22 also performs training on the predictor 2 of the student model 17, with image data as the explanatory variable and teacher labels as the target variable. Furthermore, the second training unit 22 performs training on the predictor 2 of the student model 17, with image data as the explanatory variable and prediction result 1 as the target variable.

[0057] Specifically, the second training unit 22 updates the parameters of the feature generator 17b so as to minimize the error (distance loss) between the output result "feature Y2" of the trained feature converter and the generated result "feature Y3" of the feature generator 17a of the student model 17 being trained. The second training unit 22 also updates the parameters of the predictor 17b (predictor 2) of the student model 17 so as to minimize the error (object detection loss) between the training data's teacher label and the prediction result 2. Furthermore, the second training unit 22 updates the parameters of the predictor 17b (predictor 2) of the student model 17 so as to minimize the error (distance loss) between the prediction result 1 of the student model 17's predictor 1 and the prediction result 2 of the student model 17's predictor 2.

[0058] In other words, the second training unit 21 performs supervised learning that minimizes distance loss, in addition to pattern 1 or pattern 2, and predicts the student model 17 using predictor 17b (predictor 2).

[0059] (Object Detection) Returning to Figure 6, the object detection unit 23 is a processing unit that performs object detection on the input data using the trained student model 17 trained by the first training unit 21 and the second training unit 22. Specifically, the object detection unit 23 performs object detection using the predictor 17b of the student model 17.

[0060] Figure 11 illustrates the object detection process after training. As shown in Figure 11, when the object detection unit 23 acquires image data to be detected, it inputs it into the feature generator 17a of the student model 17 to generate features. Next, the object detection unit 23 inputs the generated features into the predictor 17b and obtains the prediction result (detection result) of the predictor 17b. Then, the object detection unit 23 determines whether or not an object has been detected according to the prediction result (detection result) and outputs the determination result. Note that a determination method using a machine learning model can be employed.

[0061] <Overall Processing Flow> Figure 12 is a flowchart showing the overall processing flow of the machine learning process. As shown in Figure 12, when the information processing device 10 is instructed to start the machine learning process (S101: Yes), it obtains a trained teacher model (S102).

[0062] Next, the information processing device 10 trains the feature converter 16 using the training data (S103), and repeats this until the error converges (S104: No). Then, when the error converges and training is completed (S104: Yes), the information processing device 10 trains the student model 17 (feature generator 17a, predictor 17b) using the training data (S105), and repeats this until the error converges (S106: No). Then, when the error converges and training is completed (S106: Yes), the information processing device 10 terminates the machine learning process.

[0063] <Detailed Processing Flow> Next, we will explain the detailed flow of the machine learning process using Figures 13 and 14. Figures 13 and 14 are flowcharts showing the detailed processing flow of the machine learning process. Here, we will explain the case in which all m types of feature representations output by the student model 17 are made to mimic the feature representations of the teacher model 15.

[0064] (Definition of Symbols) First, let's explain the definition of symbols. The feature generator 17a of the student model 17 outputs m feature representations. For example, the i-th feature representation has a width w ｉ , height h ｉ , the number of dimensions d ｉ Let this be a third-order tensor.

[0065] Also, for the i-th feature representation of the student model 17, the feature representation of the Φ(i)-th teacher model 15 is distilled. For example, the Φ(i)-th output F Φ（ｉ） is a third-order tensor with width w Φ（ｉ） , height h Φ（ｉ） , and dimensionality d Φ（ｉ） . Note that the function Φ that defines the correspondence of the feature representations is a parameter, and the function Φ receives natural numbers from 1 to m as input and returns natural numbers from 1 to M.

[0066] Also, the feature transformer 16 is a set of functions for each feature representation. Here, the set of functions is represented by Equation (1). Furthermore, the predictor 17b of the student model 17 outputs n feature representations. Note that the feature representations here are synonymous with the feature quantities described in FIGS. 1 to 12.

[0067]

[0068] (Processing flow) Next, the processing flow will be described using FIGS. 13 and 14. As shown in FIG. 13, the information processing apparatus 10 reads an image (training data) (S201), applies the feature generator 15a of the teacher model 15, and obtains the feature representation "F Φ（１） , ···, F Φ（ｍ） " of the teacher model 15 (S202).

[0069] Subsequently, the information processing apparatus 10 applies the patent transformer 16 to the feature representation "F Φ（１） , ···, F Φ（ｍ） " of the teacher model 15 to obtain the feature representation "g ｉ (F<http: / / www.example.com / Φ（１） ), ···, g ｍ [http: / / www.example.com / 0000]<http: / / www.example.com / Φ（ｍ） (F Φ（ｍ） )" of the student model 17 (S203).

[0070] Then, the information processing apparatus 10 applies the predictor 17b of the student model 17 to the feature representation "g ｉ (F Φ（１） ), ···, g ｍ (F Φ（ｍ） )" of the student model 17 to obtain the prediction result of object detection (S204).

[0071] It should be noted that in the provided text, there are some placeholders like Φ（ｉ） etc. which are likely to be specific identifiers in a particular context and are left as they are in the translation. Also, the URL-like strings in the middle of the translation seem to be incorrect or incomplete in the original text. If they are meant to be something else, the translation might need to be adjusted accordingly.Subsequently, the information processing device 10 uses the parameters of the feature converter 16 and the predictor 17b of the student model 17 to determine the object detection loss "L ｏｂｊ The system updates in a direction that minimizes the loss "L" of object detection (S205). Here, the information processing device 10 updates the object detection loss "L" ｏｂｊ The process from S201 onwards is repeated until the object detection loss "L" converges (S206: No). ｏｂｊ When the process converges (S206: Yes), the process shown in Figure 14 is executed.

[0072] Specifically, as shown in Figure 14, the information processing device 10 sets Loss to "0" as an initialization (S207) and reads the image (training data) (S208).

[0073] Next, the information processing device 10 applies the feature generator 15a and feature converter 16 of the training model 15 to the image and generates a feature representation "g ｉ (F Φ（１） ), ..., g ｍ (F Φ（ｍ） The information processing device 10 acquires the feature representation "g" of the student model 17 (S209). Then, the information processing device 10 acquires the feature representation "g" of the student model 17. ｉ (F Φ（１） ), ..., g ｍ (F Φ（ｍ） The predictor 1 of student model 17 is applied to ) and the feature representation "h １ １ , ..., h ｎ １ " is obtained (S210).

[0074] In parallel, the information processing device 10 applies the feature generator 17a of the student model 17 to the image and generates a feature representation "f ｉ , ..., f ｍ The information processing device 10 acquires the feature representation "f" of the student model 17. ｉ , ..., f ｍ Apply the predictor 2 of student model 17 to " and the feature representation "h １ ２ , ..., h ｎ ２ " (S212)

[0075] Subsequently, the information processing device 10 calculates "Loss" using the acquired feature representation (S213), and the object detection loss "L" is added to the calculated "Loss". ｏｂｊ The "Loss" is updated by adding "" (S214). Note that the distance loss calculates the distance between feature representations, and the object detection loss calculates the difference between the prediction result (object position and class) that the student model ultimately outputs and the object position and class of the ground truth information. The sigma symbol in the second term on the right side of S213 corresponds to distance loss 1 (the difference between the output of the feature converter 16 and the feature generator 17a of the student model 17), and the sigma symbol in the third term on the right side corresponds to distance loss 2 (the difference between the predictor 1 of the student model 17 and the predictor 2 of the student model 17).

[0076] Subsequently, the information processing device 10 updates the parameters of the feature generator 17a and predictor 2 of the student model 17 in a direction that minimizes "Loss" (S215). Here, the information processing device 10 repeats steps S207 onward until the loss "Loss" converges (S216: No), and when "Loss" converges (S216: Yes), it executes the process shown in Figure 14.

[0077] <Effects> As described above, the information processing device 10 can learn the student model to mimic the feature representation of the teacher model while avoiding convergence to a local optimum, thereby generating a student model that can detect unlearned objects with high accuracy.

[0078] Furthermore, the information processing device 10 can further train the predictor 17b of the student model 17, which was trained together with the patent converter 16, simultaneously with the feature generator 17a of the student model 17, by using the loss from object detection, thereby further improving the detection accuracy of the student model 17.

[0079] Furthermore, when the information processing device 10 is trained simultaneously with the feature generator 17a of the student model 17, it can be trained using not only the object detection loss but also the distance loss, thereby further improving the prediction accuracy of the student model 17.

[0080] By the way, in Example 1, we described an example in which the feature generator 17a of the student model 17 is trained to imitate the feature generator 15a of the teacher model 15, but it is not limited to this. For example, the feature generator 17a of the student model 17 outputs m types of feature representations, but only some of these feature representations can be made to imitate the feature representations of the teacher model 15. Specifically, when training the feature converter 16, the information processing device 10 obtains feature representations that are not to be imitated from the feature generator of the student model 17, and when training the student model 17, it calculates only the feature representations to be imitated as distance loss.

[0081] Figures 15 and 16 are flowcharts detailing the processing flow of the machine learning process according to Example 2. The definitions of symbols and the like are the same as in Example 1. Here, during the training of the feature converter 16 (first training), the number of features generated by the feature generator 17a of the student model 17 (hereinafter referred to as feature generator 1) is "1 to j", and the number of features generated by the feature generator 17a of the teacher model 15 is "1 to m-j". Also, during the training of the student model 17 (second training), the number of features generated by the feature generator 17a of the student model 17 (hereinafter referred to as feature generator 2) is "1 to j, j+1 to m".

[0082] As shown in Figure 15, the information processing device 10 reads an image (training data) (S301), applies the feature generator 1 of the student model 17, and generates the feature representation "f" of the student model 17. ｉ １ , ..., f ｍ １ " (S302)

[0083] In parallel, the information processing device 10 applies the feature generator 15a of the teacher model 15 to the training data to generate the feature representation "F" of the teacher model 15. Φ（１） , ..., F Φ（ｍ） The information processing device 10 acquires the feature representation "F" of the teacher model 15 (S303). Φ（１） , ..., F Φ（ｍ） The patent converter 16 is applied to the feature representation of the student model 17 "g ｉ (F Φ（１） ), ..., g ｍ (F Φ（ｍ）) is acquired (S304).

[0084] Then, the information processing device 10 displays the feature representation "f" of the student model 17. ｉ １ , ..., f ｍ １ The prediction result is obtained by applying the "g" feature expression of the student model 17 to the predictor 17b, and similarly, the feature expression "g" of the student model 17 is applied to the predictor 17b of the student model 17. ｉ (F Φ（１） ), ..., g ｍ (F Φ（ｍ） The result is obtained by applying the result to the predictor 17b of the student model 17 (S305).

[0085] Subsequently, the information processing device 10 uses the parameters of the feature converter 16, the feature generator 1 and predictor 17b of the student model 17 to determine the object detection loss "L ｏｂｊ The object detection loss "L" is updated in a direction that minimizes it (S306). ｏｂｊ The loss used to train the feature converter 16 and the feature generator 1 of the student model 17, as in Example 1, and, unlike Example 1, the loss (error) between the prediction result output by the predictor 17b in response to the input of the feature generator 1 of the student model 17 and the teacher label, which is used to train the predictor 17b.

[0086] Here, the information processing device 10 has a loss of object detection "L ｏｂｊ The process from S201 onwards is repeated until the object detection loss "L" converges (S307: No). ｏｂｊ When the condition converges (S307: Yes), the process shown in Figure 16 is executed.

[0087] Specifically, as shown in Figure 16, the information processing device 10 sets Loss to "0" as an initialization (S308) and reads the image (training data) (S309).

[0088] Next, the information processing device 10 applies the feature generator 1 of the student model 17 to the image and generates the feature representation "f" of the student model 17. ｉ １ , ..., f ｍ １ The image is obtained, and the feature generator 15a and feature converter 16 of the training model 15 are applied to the image to obtain the feature representation "g ｉ(F Φ（１） ),..., g ｍ (F Φ（ｍ） ).」 is obtained (S310). Then, the information processing apparatus 10 applies the predictor 1 of the student model 17 to the feature expression 「g ｉ (F Φ（１） ),..., g ｍ (F Φ（ｍ） ).」 of the student model 17, and obtains the feature expression 「h １ １ ,..., h ｎ １ .」 (S311).

[0089] In parallel, the information processing apparatus 10 applies the feature generator 2 of the student model 17 to the image, and obtains the feature expression 「f ｉ ２ ,..., f ｍ ２ .」 of the student model 17 (S312). Then, the information processing apparatus 10 applies the predictor 2 of the student model 17 to the feature expression 「f ｉ ２ ,..., f ｍ ２ .」 of the student model 17, and obtains the feature expression 「h １ ２ ,..., h ｎ ２ .」 (S313).

[0090] After that, the information processing apparatus 10 calculates 「Loss」 using the obtained feature expressions (S314), adds the object detection loss 「L ｏｂｊ .」 to the calculated 「Loss」, and updates 「Loss」 (S315).

[0091] Then, the information processing apparatus 10 updates the parameters of the feature generator 2 and the predictor 2 of the student model 17 in the direction in which 「Loss」 is minimized (S316). Here, the information processing apparatus 10 repeats the steps after S308 until the loss 「Loss」 converges (S317: No), and when 「Loss」 converges (S317: Yes), executes the processing of FIG. 16.

[0092] As described above, the information processing device 10 can reflect only the features obtained from the teacher model 15 into the feature generator 17a of the student model 17, and then train the predictor 17b of the student model 17. Therefore, compared to Example 1, the training time of the predictor 17b can be shortened and the accuracy of the predictor 17b can be improved.

[0093] Now, although embodiments of the present invention have been described, the present invention may be implemented in various other forms besides those described above.

[0094] (Numerical values, etc.) The parameters, specific examples, numerical values, etc. used in the above embodiment are merely examples and can be changed at will. Also, the processing flow described in each flowchart can be changed as appropriate within a range that does not contradict each other. Note that the teacher model 15 is an example of the first machine learning model, and the student model 17 is an example of the second machine learning model.

[0095] (System) Unless otherwise specified, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings may be changed at will.

[0096] Furthermore, the specific forms of distribution and integration of the components of each device are not limited to those shown in the diagram. For example, the first training unit 21 and the second training unit 22 may be integrated. In other words, all or part of the components may be functionally or physically distributed or integrated in any unit depending on various loads and usage conditions. Moreover, all or any part of the processing functions of each device may be implemented by a CPU and a program that is analyzed and executed by the CPU, or by hardware using wired logic.

[0097] Furthermore, each processing function performed by each device can be implemented, in whole or in part, by a CPU and a program executed for analysis by that CPU, or by hardware using wired logic.

[0098] (Hardware) Figure 17 is a diagram illustrating an example of hardware configuration. As shown in Figure 17, the information processing device 10 includes a communication device 10a, an HDD (Hard Disk Drive) 10b, memory 10c, and a processor 10d. Furthermore, each of the parts shown in Figure 17 is interconnected by a bus or the like.

[0099] The communication device 10a is a network interface card or the like, and communicates with other devices. The HDD 10b stores programs and databases that operate the functions shown in Figure 6.

[0100] The processor 10d operates a process that performs the functions described in Figure 6 by reading a program that performs the same processing as each processing unit shown in Figure 6 from the HDD 10b or the like and loading it into memory 10c. For example, this process performs the same functions as each processing unit of the information processing device 10. Specifically, the processor 10d reads a program that has the same functions as the first training unit 21, the second training unit 22, the object detection unit 23, etc. from the HDD 10b or the like. Then, the processor 10d executes a process that performs the same processing as the first training unit 21, the second training unit 22, the object detection unit 23, etc.

[0101] Thus, the information processing device 10 operates as an information processing device that executes an information processing method by reading and executing a program. Furthermore, the information processing device 10 can also achieve the same functionality as in the above-described embodiment by reading the program from a recording medium using a media reader and executing the read program. Note that the program referred to in this other embodiment is not limited to being executed by the information processing device 10. For example, the above embodiment may also be applied similarly when another computer or server executes the program, or when they collaborate to execute the program.

[0102] This program may be distributed via a network such as the Internet. Alternatively, this program may be recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), or DVD (Digital Versatile Disc), and executed by being read from the recording medium by a computer.

[0103] 10 Information processing unit 11 Communication unit 12 Output unit 13 Storage unit 14 Training data 15 Teacher model 16 Feature converter 17 Student model 20 Control unit 21 First training unit 22 Second training unit 23 Object detection unit

Claims

1. A machine learning program characterized by causing a computer to perform the following steps:

1. A feature converter that converts first features used for prediction in a first pre-trained machine learning model having a first number of parameters into second features; 2. A predictor of the second machine learning model, which includes a feature generator and a predictor, and has a second number of parameters less than the first number of parameters, is trained using the output results obtained by inputting the second features converted by the feature converter into the predictor; 3. A second training is performed to train the feature generator based on the conversion results converted by the pre-trained feature converter and the generation results generated by the feature generator of the second machine learning model.

2. The machine learning program according to claim 1, characterized in that the process of performing the second training is to update the parameters of the predictor trained by the first training using the output result obtained by inputting the generation result by the feature generator of the second machine learning model to the predictor of the second machine learning model.

3. The machine learning program according to claim 1 or 2, characterized in that the process of performing the second training is to update the parameters of the second predictor so as to reduce the error between a first output result obtained by inputting the conversion result of the trained feature converter trained by the first training into a trained first predictor trained by the first training, and a second output result obtained by inputting the generation result generated by the feature generator into a second predictor with set parameters for the trained first predictor.

4. The machine learning program according to claim 1 or 2, characterized in that the process of performing the first training is to perform the first training by training the feature converter and the predictor of the second machine learning model, using training data which includes the first feature quantities generated from the image data and the correct information relating to the objects contained in the image data, so that the error between the output result obtained by inputting the second feature quantities converted from the first feature quantities by the feature converter into the predictor of the second machine learning model and the correct information becomes smaller.

5. The machine learning program according to claim 4, characterized in that the process of performing the second training is to perform the second training, which involves training the feature generator of the second machine learning model using training data that includes the first feature quantities generated from the image data and the ground truth information which are the second feature quantities, so that the error between the generated result generated by the feature generator of the second machine learning model from the image data and the ground truth information becomes smaller.

6. The machine learning program according to claim 5, characterized in that the process of performing the second training is to perform the second training, which involves training the predictor of the second machine learning model using training data including the image data and ground truth information relating to the objects contained in the image data, so as to reduce the error between the output result obtained by inputting the generation result generated by the feature generator of the second machine learning model from the image data to the predictor and the ground truth information.

7. The machine learning program according to 6, characterized in that the process of performing the second training is to train the second predictor such that the error between the first output result obtained by inputting the second feature quantity, which is obtained by converting the first feature quantity generated from the image data by the converter trained by the first training, into the first predictor trained by the first training, and the second output result obtained by inputting the generation result, which is generated by the feature generator of the second machine learning model from the image data, into the second predictor, which has been trained by the first training and whose parameters have been set, becomes small.

8. The machine learning program according to claim 1, characterized in that the computer is made to perform the following processes: inputting the input data to be judged into a trained feature generator trained by the second training to generate feature quantities; inputting the generated feature quantities into a trained predictor trained by the second training to perform object detection on the input data based on the output result obtained.

9. A machine learning method characterized by a computer performing a first training, in which a feature converter converts first features used for prediction in a first trained machine learning model having a first number of parameters into second features, and a predictor of the second machine learning model, which includes a feature generator and a predictor and has a second number of parameters less than the first number of parameters, by inputting the second features converted by the feature converter into the predictor and performing an output result obtained; and performing a second training, in which the feature generator is trained based on the conversion result converted by the trained feature converter and the generation result generated by the feature generator of the second machine learning model.

10. An information processing apparatus comprising: a feature converter that converts first features used for prediction in a first trained machine learning model having a first number of parameters into second features; a predictor of the second machine learning model including a feature generator and a predictor, having a second number of parameters less than the first number of parameters, and performing a first training using the output results obtained by inputting the second features converted by the feature converter into the predictor; and performing a second training to train the feature generator based on the conversion results converted by the trained feature converter and the generation results generated by the feature generator of the second machine learning model.

Images