Information processing device, information processing method, and information processing program
Patent Information
- Application Number
- JP2023506415
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-15
- Publication Date
- 2025-03-11
- Estimated Expiration
- 2041-03-15
Smart Images
Figure 0007643524000008 
Figure 0007643524000009 
Figure 0007643524000010
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]
[0002] In recent years, with the progress of the deep learning field, deep learning models with high recognition performance have appeared. In training of deep learning models, training is carried out efficiently by using a large amount of labeled data, which is data with manually assigned correct answers. For example, in the field of image recognition, when recognizing an object, at least several hundred labeled data are used for one object.
[0003] On the other hand, most of the training data provided in the environment where training data is actually acquired is unlabeled data with no correct answer, and labeled data is in small quantities. For example, when there are several hundred pieces of training data in total, there are often only a few dozen labeled data. When there is little labeled data, the deep learning model will over-fit to the training data, and its performance on untrained data will decrease. This phenomenon is called overfitting. Therefore, there is a demand for a method to train a deep learning model with high recognition performance using unlabeled data as well.
[0004] Conventionally, the following technologies have been provided for deep learning. One is a technology that acquires feature extraction capability, which is the ability to extract image features used for recognition from unlabeled data. Specifically, features are extracted from unlabeled data using a deep learning model, data is grouped together and divided into multiple clusters based on the extracted features, and a pseudo label, which is a pseudo-correct answer, is assigned to each cluster and learning is performed to acquire feature extraction capability.
[0005] The other technique is called Transfer Learning, where a deep learning model is given pre-acquired feature extraction capabilities and then trained on labeled data, with the ability to identify data based on the extracted features.
[0006] Then, by combining the two above-mentioned technologies, a method can be considered in which learning is performed with labeled data, limited to the ability to identify data based on the extracted features, based on the feature extraction ability acquired from unlabeled data. This makes it possible to obtain a deep learning model with high recognition performance even with a small amount of labeled data. [Prior art documents] [Non-patent literature]
[0007] [Non-Patent Document 1] Self-labelling via simultaneous clustering and representation learning, Yuki M. Asano, Christian Rupprecht, Andrea Vedaldi, ICLR2020, 20 August 2020. Summary of the Invention [Problem to be solved by the invention]
[0008] However, if the feature extraction ability from unlabeled data is acquired and then learning is limited to the ability to identify data based on the extracted features, the feature extraction ability and the identification ability of the deep learning model are individually learned and optimized. That is, the feature extraction ability is optimized by acquiring the feature extraction ability from unlabeled data, and the identification ability is optimized by learning limited to the ability to identify data based on the extracted features. For this reason, if each process is performed in order, it becomes difficult to tune the feature extraction ability to match the identification ability, and the system falls into a local optimum solution. As a result, the overall recognition performance of the deep learning model is reduced.
[0009] The disclosed technology has been made in consideration of the above, and aims to provide an information processing device, an information processing method, and an information processing program that improve the recognition performance of a deep learning model. [Means for solving the problem]
[0010] In one aspect of the information processing device, information processing method, and information processing program disclosed in the present application, a storage unit stores a plurality of labeled data in which a label representing a correct answer is associated with target data, a plurality of unlabeled data which is target data not associated with a correct answer and is greater in number than the labeled data, and a deep learning model. A pseudo label generation unit generates pseudo labels based on the unlabeled data and the deep learning model, and assigns the pseudo labels to the unlabeled data. A first model output unit acquires the deep learning model and stores it as a first deep learning model, inputs the unlabeled data to the first deep learning model to obtain a first output value, and a second model output unit acquires the deep learning model and stores it as a second deep learning model, inputs the labeled data to the second deep learning model to obtain a second output value. The loss calculation unit, after the pseudo label generation unit assigns the pseudo label, the first output value obtained by the first model output unit; The pseudo label and calculating a first loss using the second output value obtained by the second model output unit. The label Using and, the second loss The update unit updates the deep learning model based on the loss calculated by the loss calculation unit. Effect of the Invention
[0011] According to one aspect of the information processing device, information processing method, and information processing program disclosed in the present application, the recognition performance of a deep learning model can be improved. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a block diagram of a learning device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining the learning method according to the first embodiment. [Diagram 3] FIG. 3 is a flowchart of the entire learning process according to the first embodiment. [Figure 4] FIG. 4 is a flowchart of simultaneous learning using labeled and unlabeled data. [Diagram 5] FIG. 5 is a block diagram of the learning device according to the second embodiment. [Figure 6] FIG. 6 is a diagram for explaining a learning method according to the second embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of learning data used in the third embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a hardware configuration of the learning device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, an information processing device, an information processing method, and an information processing program disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the information processing device, the information processing method, and the information processing program disclosed in the present application are not limited to the following embodiments. EXAMPLES
[0014] Fig. 1 is a block diagram of a learning device according to a first embodiment. The learning device 1, which is an information processing device according to this embodiment, learns a deep learning model 110 that recognizes image data. Specifically, image data is data represented as a set of RGB (Red Green Blue) values for each pixel displayed on a screen. As shown in Fig. 1, the learning device 1 includes a storage unit 11, a pseudo label generation unit 12, a model output unit 13, a loss calculation unit 14, and an update unit 15.
[0015] The memory unit 11 stores a deep learning model 110, an unlabeled DB (Data Base) 111, and a labeled DB 112.
[0016] In this embodiment, the deep learning model 110 is a learning model that performs image recognition. The deep learning model 110 has a feature extraction layer that extracts features of image data, and an identification layer that identifies an object appearing in the image data from the features of the image data.
[0017] The unlabeled DB 111 is a database that stores unlabeled data 201, which is image data. The unlabeled DB 111 stores the unlabeled data 201 input by a user using an external terminal device or the like. The unlabeled data 201 is learning data that is not assigned a correct label that indicates what object is depicted in the image data.
[0018] The labeled DB 112 is a database that stores labeled data 202, which is image data. The labeled DB 112 stores the labeled data 202 input by a user using an external terminal device or the like. The labeled data 202 is learning data to which a correct answer label has been assigned.
[0019] The pseudo label generation unit 12 acquires the deep learning model 110 stored in the storage unit 11. The pseudo label generation unit 12 also reads a plurality of unlabeled data 201 from the unlabeled DB 111. At this time, it is preferable that the pseudo label generation unit 12 reads all of the unlabeled data 201. Next, the pseudo label generation unit 12 inputs each piece of unlabeled data 201 included in the read image group to the deep learning model 110 and acquires an output corresponding to each piece of unlabeled data 201.
[0020] Next, the pseudo label generation unit 12 groups each of the unlabeled data 201 included in the read image group according to the output value from the deep learning model 110, and divides the data into a predetermined number of clusters. For example, the pseudo label generation unit 12 performs cluster division using k-means clustering.
[0021] Then, the pseudo label generation unit 12 assigns a pseudo label, which is a pseudo correct answer, to each cluster. For example, if there are k classes, the pseudo label generation unit 12 assigns pseudo labels such as class #1, class #2, class #3, . . . , class #k. After that, the pseudo label generation unit 12 outputs the pseudo label assigned to each cluster together with information on the unlabeled data 201 included in each cluster to the loss calculation unit 14.
[0022] The model output unit 13 obtains outputs from the deep learning models 110 for the unlabeled data 201 and the labeled data 202. The model output unit 13 includes a first model output unit 131 and a second model output unit 132.
[0023] Furthermore, the loss calculation unit 14 compares the output value from the deep learning model 110 with the pseudo label or the label assigned to the labeled data 202, and calculates the respective losses. The loss calculation unit 14 has a first loss calculation unit 141 and a second loss calculation unit 142. The operation of the model output unit 13 and the loss calculation unit 14 will be described in detail below.
[0024] The first model output unit 131 acquires the deep learning model 110 stored in the storage unit 11. In addition, the first model output unit 131 reads from the unlabeled DB 111 a plurality of pieces of unlabeled data 201 used for learning the deep learning model 110.
[0025] Next, the first model output unit 131 inputs each unlabeled data 201 included in the read image group to a feature extraction layer of the deep learning model 110 to obtain an output from the deep learning model 110 corresponding to each unlabeled data 201. For example, u And D u Let x be the unlabeled data 201 in u In this case, the first model output unit 131 calculates y u Get the.
[0026]
number
[0027] Here, f represents the feature extraction layer of the deep learning model 110. That is, f(x u ) represents the output from the feature extraction layer. Also, h unsup represents the classification layer for unlabeled data of the deep learning model 110. That is, h unsup (f(x u )) is the output obtained by inputting the output from the feature extraction layer to the discrimination layer.
[0028] Thereafter, the first model output unit 131 outputs the output value of the deep learning model 110 for each unlabeled data 201 to the first loss calculation unit 141 of the loss calculation unit 14. For example, the first model output unit 131 outputs y u to the first loss calculation unit 141.
[0029] The first loss calculation unit 141 calculates the loss when using the unlabeled data 201. Hereinafter, the loss may be referred to as Loss.
[0030] The first loss calculation unit 141 receives an input of an output value from the deep learning model 110 for the unlabeled data 201 from the first model output unit 131. Furthermore, the first loss calculation unit 141 receives an input of a pseudo label for each cluster created by clustering the unlabeled data 201 from the pseudo label generation unit 12 together with information on the unlabeled data 201 included in each cluster.
[0031] Next, the first loss calculation unit 141 compares the acquired output value with the pseudo label, and calculates a loss when unlabeled data 201 is used, which is an error between the estimation result using the deep learning model 110 and the pseudo label, which is the correct answer here. For example, the first loss calculation unit 141 calculates a loss when y u LossL, which is the loss when using the unlabeled data 201, is calculated using the following formula (2): unsup Calculate.
[0032]
number
[0033] Here, t u are pseudo labels, and CE stands for the general cross-entropy loss.
[0034] After that, the first loss calculation unit 141 outputs the calculated loss when the unlabeled data 201 is used to the update unit 15. For example, the first loss calculation unit 141 outputs the calculated L unsupto the update unit 15.
[0035] The second model output unit 132 acquires the deep learning model 110 stored in the storage unit 11. In addition, the second model output unit 132 reads, from the labeled DB 112, labeled data 202 used for learning the deep learning model 110.
[0036] Next, the second model output unit 132 inputs each labeled data 202 included in the read image group to a feature extraction layer of the deep learning model 110 to obtain an output from the deep learning model 110 corresponding to each labeled data 202. For example, i And D i Let x be the unlabeled data 202 contained in i In this case, the second model output unit 132 calculates y i Get the.
[0037]
number
[0038] Here, f represents the feature extraction layer of the deep learning model 110. That is, f(x i ) represents the output from the feature extraction layer. Also, h sup represents the classification layer for labeled data of the deep learning model 110. That is, h sup (f(x i )) is the output obtained by inputting the output from the feature extraction layer to the discrimination layer. As described above, in the learning device 1 according to the present embodiment, the discrimination layer for the unlabeled data 201 and the discrimination layer for the unlabeled data 202 are trained separately.
[0039] Thereafter, the second model output unit 132 outputs the output value of the deep learning model 110 for each labeled data 202 to the second loss calculation unit 142. For example, the second model output unit 132 outputs y i to the second loss calculation unit 142.
[0040] The second loss calculation unit 142 receives an input of an output value from the deep learning model 110 for the labeled data 202 from the second model output unit 132. Furthermore, the second loss calculation unit 142 acquires, from the labeled DB 112, the labels assigned to each labeled data 202 read by the model output unit 13.
[0041] Next, the second loss calculation unit 142 compares the acquired output value with the label assigned to each labeled data 202, and calculates a loss when the labeled data 202 is used, which is an error between the estimation result using the deep learning model 110 and the correct label. For example, the second loss calculation unit 142 calculates a loss when the labeled data 202 is used, which is an error between the estimation result using the deep learning model 110 and the correct label. i The loss L when using the labeled data 202 is calculated using the following formula (4) for sup Calculate.
[0042]
number
[0043] Here, t i is the correct answer. Also, CE stands for the general cross entropy loss.
[0044] After that, the second loss calculation unit 142 outputs the calculated loss when the labeled data 202 is used to the update unit 15. For example, the second loss calculation unit 142 outputs the calculated L sup to the update unit 15.
[0045] The update unit 15 receives an input of the loss when the unlabeled data 201 is used from the first loss calculation unit 141. Also, the update unit 15 receives an input of the loss when the labeled data 202 is used from the second loss calculation unit 142. Then, the update unit 15 performs a predetermined weighting on the estimation result when the unlabeled data 201 is used and the estimation result when the labeled data 202 is used, and calculates the final loss. For example, the update unit 15 calculates Lunsup and L, which is the loss when using labeled data 202. sup Then, the final loss, L, is calculated using the following formula (5): total Calculate.
[0046]
number
[0047] Here, α is L sup and L unsup is a parameter for adjusting the balance between α and α, and is a constant that weights each of them. α takes a value greater than 0 and less than 1. The larger α is, the greater the impact on learning due to the estimation results when labeled data 202 is used.
[0048] Thereafter, the update unit 15 obtains the feature extraction layer parameters of the deep learning model 110, the discrimination layer parameters for the unlabeled data 201, and the discrimination layer parameters for the labeled data 202 so as to minimize the calculated final loss. Then, the update unit 15 updates the deep learning model 110 held by the model output unit 13 with the obtained feature extraction layer parameters of the deep learning model 110 and the discrimination layer parameters for the unlabeled data 201. Also, the update unit 15 updates the deep learning model 110 held by the model output unit 13 with the obtained feature extraction layer parameters of the deep learning model 110 and the discrimination layer parameters for the labeled data 202. For example, the update unit 15 updates L total Minimize f, L sup and L unsup This updates the model output unit 13 and the deep learning model 110 held by each of the model output units 13.
[0049] In this way, in the learning device 1 according to the present embodiment, the deep learning model 110 for the unlabeled data 201 and the deep learning model 110 for the labeled data 202 are learned separately and simultaneously in parallel. However, the deep learning model 110 for the unlabeled data 201 and the deep learning model 110 for the labeled data 202 have the same feature extraction layer and different discrimination layers. Then, in the recognition phase after learning, the deep learning model 110 for the labeled data 202 that has been learned and is held by the model output unit 13 is used to recognize unknown image data.
[0050] 2 is a diagram for explaining the learning method according to the embodiment 1. Next, the overall flow of learning in this embodiment will be explained with reference to FIG.
[0051] First, a plurality of unlabeled data 201 and a plurality of labeled data 202 are prepared and stored in an unlabeled DB 111 and a labeled DB 112, respectively. As shown in Fig. 2, no correct answer is assigned to the unlabeled data, but labels such as flowers, cars, and fish are assigned to the labeled data 202.
[0052] Next, for each of the labeled data 201 and the labeled data 202, the first model output unit 131 and the second model output unit 132 extract features using the feature extraction layer of the deep learning model 110 (step S1).
[0053] Next, learning using the unlabeled data 201 proceeds in the direction of the upper arrow on the paper surface from the feature extraction layer to the discrimination layer in FIG. 2. Then, the pseudo label generation unit 12 performs classification by clustering and adds pseudo labels. Thereafter, the first loss calculation unit 141, the second loss calculation unit 142, and the update unit 15 simultaneously perform learning using the unlabeled data 201 using the pseudo labels and learning using the labeled data 202 using the labels (steps S2 and S3). Through this learning, the feature extraction layer, the discrimination layer for the unlabeled data 201, and the discrimination layer for the labeled data 202 of the deep learning model 110 are simultaneously trained.
[0054] 3 is a flowchart of the entire learning process according to the embodiment 1. Next, the entire flow of the learning process according to the embodiment 1 will be described with reference to FIG.
[0055] The learning device 1 acquires unlabeled data 201 and stores it in the unlabeled DB 111. The learning device 1 also acquires labeled data 202 and stores it in the labeled DB 112 (step S11).
[0056] The update unit 15 acquires a count threshold value input from an external terminal device or the like (step S12).
[0057] Next, the update unit 15 initializes the number of learning times to set it to 0 (step S13).
[0058] The pseudo label generating unit 12 reads out a plurality of unlabeled data 201 from the unlabeled DB 111, classifies the data, generates a pseudo label for each class, and assigns the pseudo label to each class (step S14).
[0059] The first model output unit 131 and the second model output unit 132, the first loss calculation unit 141 and the second loss calculation unit 142, and the update unit 15 perform simultaneous learning using the labeled data 202 and the unlabeled data 201 (step S15).
[0060] After that, the update unit 15 determines whether the learning count exceeds the count threshold (step S16). If the learning count is equal to or less than the count threshold (step S16: No), the update unit 15 adds 1 to the learning count to increment the learning count (step S17). Then, the learning process returns to step S14.
[0061] On the other hand, if the number of learning times exceeds the number threshold value (step S16: Yes), the update unit 15 causes the learning device 1 to end the learning process.
[0062] Fig. 4 is a flowchart of simultaneous learning using labeled data and unlabeled data. Next, the flow of simultaneous learning using labeled data and unlabeled data will be described with reference to Fig. 4. Each process shown in Fig. 4 corresponds to an example of the process executed in step S15 in Fig. 3.
[0063] The second model output unit 132 reads out a plurality of labeled data 202 from the labeled DB 112. Then, the second model output unit 132 inputs each of the read out labeled data 202 to the feature extraction layer of the deep learning model 110. After that, the second model output unit 132 acquires an output from the deep learning model 110 (step S101).
[0064] The second loss calculation unit 142 acquires the labels assigned to the labeled data 202 read by the second model output unit 132 from the labeled DB 112. Then, the second loss calculation unit 142 compares the output value corresponding to each labeled data 202 acquired from the second model output unit 132 with the label assigned to the labeled data 202, and calculates a loss when the labeled data 202 is used (step S102).
[0065] The first model output unit 131 reads out a plurality of unlabeled data 201 from the labeled DB 111. Then, the first model output unit 131 inputs each of the read out unlabeled data 201 to the feature extraction layer of the deep learning model 110. After that, the first model output unit 131 acquires an output from the deep learning model 110 (step S103).
[0066] The first loss calculation unit 141 compares the output values corresponding to each unlabeled data 201 obtained from the first model output unit 131 with the pseudo labels obtained from the pseudo label generation unit 12, and calculates the loss when the unlabeled data 201 is used (step S104).
[0067] The updating unit 15 acquires the loss when the labeled data 202 is used from the second loss calculation unit 142. Also, the updating unit 15 acquires the loss when the unlabeled data 201 is used from the first loss calculation unit 141. Then, the updating unit 15 calculates the overall loss by using the weights for the loss when the labeled data 202 is used and the loss when the unlabeled data 201 is used (step S105).
[0068] Thereafter, the update unit 15 updates the deep learning models 110 included in each of the first model output unit 131 and the second model output unit 132 so as to minimize the overall loss (step S106).
[0069] As described above, the learning device according to the present embodiment divides unlabeled data into a plurality of clusters, assigns a pseudo label to each cluster, and executes learning of a deep learning model using labeled data, unlabeled data, and pseudo labels. This allows the learning device to simultaneously learn the feature extraction layer and the discrimination layer of the deep learning model using both labeled data and unlabeled data. Therefore, even when learning is performed using a large amount of unlabeled data and a small amount of labeled data, optimal recognition performance can be obtained, and the recognition performance of the deep learning model can be improved. EXAMPLES
[0070] 5 is a block diagram of a learning device according to Example 2. The learning device 1 according to this example is different from Example 1 in that it performs single-task learning using one discrimination layer. In the following description, the functions of each unit similar to those of Example 1 will be omitted.
[0071] In the learning device 1 according to the present embodiment, the number of labels represented by the labeled data 202 is equal to the number of clusters obtained by clustering the unlabeled data. That is, in the learning device 1 according to the present embodiment, the number of labels represented by the labeled data 202 is equal to t u There are 201 unlabeled data. i If you classify into t clusters, u =t i It is.
[0072] The pseudo label generation unit 12 performs clustering using the unlabeled data 201 in the same manner as in the first embodiment, and divides the unlabeled data 201 into a plurality of clusters. At this time, the pseudo label generation unit 12 classifies the unlabeled data 201 into the same number of clusters as the number of labels represented by the labeled data 202. Then, the pseudo label generation unit 12 assigns a pseudo label to each cluster. After that, the pseudo label generation unit 12 outputs the generated pseudo labels to the loss calculation unit 14.
[0073] The model output unit 13 reads a plurality of pieces of unlabeled data 201 from the unlabeled DB 111. The model output unit 13 also reads a plurality of pieces of labeled data 202 from the labeled DB 112. Then, the model output unit 13 integrates the read unlabeled data 201 and labeled data 202 to generate integrated data. Then, the model output unit 13 inputs the integrated data to the deep learning model 110 to obtain an output.
[0074] For example, when the integrated data is x, the model output unit 13 acquires y, which is the output from the deep learning model 110 expressed by the following formula (6).
[0075]
number
[0076] Here, f represents the feature extraction layer of the deep learning model 110. That is, f(x) is the output from the feature extraction layer. h represents the classification layer of the deep learning model 110. That is, h(f(x)) is the output obtained by inputting the output value from the feature extraction layer to the classification layer.
[0077] Thereafter, the model output unit 13 outputs the output value for each integrated data to the loss calculation unit 14.
[0078] The loss calculation unit 14 receives an input of an output value from the deep learning model 101 for each integrated data from the model output unit 13. In addition, the loss calculation unit 14 acquires a label representing each labeled data 201 stored in the labeled DB 112 from the labeled DB 112. In addition, the loss calculation unit 14 receives an input of a pseudo label for each class from the pseudo label generation unit 12.
[0079] Next, the loss calculation unit 14 generates an integrated label by integrating the labels and the pseudo labels acquired from the labeled DB 112. For example, since the number of labels and the pseudo labels acquired from the labeled DB 112 is the same, the loss calculation unit 14 generates an integrated label by replacing each of the pseudo labels with a label that is determined to indicate the same thing.
[0080] Thereafter, the loss calculation unit 14 compares the output value from the feature extraction layer of the deep learning model 101 for each integrated data with the integrated label corresponding to each integrated data, and calculates the loss when the integrated data is used.
[0081] For example, if a set including all of the integrated data x is denoted by D and the integrated label is denoted by t, the loss calculation unit 14 calculates L, which is the loss when the integrated data is used, by using the following formula (7). Here, CE is a general cross entropy loss.
[0082]
number
[0083] Thereafter, the loss calculation unit 14 outputs the calculated loss to the update unit 15. For example, the loss calculation unit 14 outputs to the update unit 15 L, which is the loss when the integrated data is used and is calculated using Equation (7).
[0084] The update unit 15 receives an input of the loss from the loss calculation unit 14. Then, the update unit 15 determines parameters of the deep learning model 110 that minimize the loss. After that, the update unit 15 updates the deep learning model 110 held by the model output unit 13 using the determined parameters.
[0085] For example, when the update unit 15 acquires L, which is the loss when the integrated data is used, from the loss calculation unit 14, the update unit 15 updates f, which is the feature extraction layer, and h, which is the classification layer, so as to minimize L. That is, in this embodiment, for both the unlabeled data 201 and the labeled data 202, learning is performed using one deep learning model 110 having similar feature extraction layers and classification layers.
[0086] Fig. 6 is a diagram for explaining a learning method according to Example 2. Details of the learning device 1 according to this embodiment will be described with reference to Fig. 6.
[0087] The model output unit 13 reads the unlabeled data 201 and the labeled data 202 to generate integrated data. Next, the model output unit 13 inputs the integrated data to the deep learning model 101 and obtains an output from the deep learning model 101 corresponding to each integrated data (step S201).
[0088] The pseudo label generation unit 12 performs clustering using the unlabeled data 201 in the same manner as in the first embodiment, and divides the unlabeled data 201 into clusters in the same number as the labels representing the labeled data 202 stored in the labeled DB 112. Then, the pseudo label generation unit 12 assigns a pseudo label to each cluster (step S202).
[0089] The loss calculation unit 14 generates an integrated label by integrating the pseudo label with a label representing the labeled data 202 stored in the labeled DB 112. Then, the loss calculation unit 14 compares an output value corresponding to each integrated data with the integrated label to calculate a loss. The update unit 15 updates and learns the feature extraction layer and the discrimination layer of the deep learning model 110 included in the model output unit 13 so as to minimize the loss calculated by the loss calculation unit 14 (step S203).
[0090] As described above, the learning device according to the present embodiment classifies unlabeled data into clusters of the same number as the number of labels representing labeled data. The learning device then generates integrated data by integrating labeled data and unlabeled data, integrates the labels of the labeled data and pseudo labels to generate integrated labels, and performs learning using the integrated data and the integrated labels. This makes it possible to train a deep learning model by single-task learning using a single discrimination layer. Even with this method, even when training is performed using a large amount of unlabeled data and a small amount of labeled data, it is possible to obtain optimal recognition performance and improve the recognition performance of the deep learning model. EXAMPLES
[0091] Next, a description will be given of Example 3. In Examples 1 and 2, the case where image data is used as learning data has been described as an example, but learning can be performed using unlabeled data and labeled data other than the image data in the same manner.
[0092] For example, the learning device 1 can also learn the deep learning model 110 using moving images as learning data. A moving image is a collection of RGB values corresponding to the passage of time for each pixel on a screen. In this case, by using the trained deep learning model 110, it becomes possible to identify the type of unknown moving images.
[0093] In addition, the learning device 1 can also learn the deep learning model 110 using joint data as learning data. FIG. 7 is a diagram showing an example of learning data used in the third embodiment. The joint data is data that represents the spatial positions of joints of the human body, such as the wrist and elbow, as represented by each point of the image 300 in FIG. 7. For example, the data is represented by xyz coordinates in a three-dimensional space, and by xy coordinates in a two-dimensional plane. Furthermore, in the case of joint data that includes human movement, sensor data such as acceleration information at each point when the person moves and information from a gyro sensor are added. In that case, by using the learned deep learning model 110, it is possible to identify what kind of movement the person is making.
[0094] As described above, the learning device according to each embodiment can learn a deep learning model using other data, such as video image data, joint data, etc. Even when data other than image data is used, optimal recognition performance can be obtained by learning using a large amount of unlabeled data and a small amount of labeled data, thereby making it possible to improve the recognition performance of the deep learning model.
[0095] (Hardware configuration) Fig. 8 is a diagram showing an example of a hardware configuration of a learning device. The learning device 1 shown in Figs. 1 and 5 is realized by a computer 90 in Fig. 8. For example, the computer 90 is a server.
[0096] The computer 90 includes a processor 901, a main memory device 902, an auxiliary memory device 203, an input device 904, an output device 905, a media drive device 906, an input / output interface 907, and a communication control device 908. The components of the computer 90 are connected to each other via a bus 909.
[0097] The processor 901 is, for example, a CPU (Central Processing Unit). The computer 90 may have a plurality of processors 901. Furthermore, the computer 90 may have a GPU (Graphics Processing Unit) or the like as the processor 901. The processor 901 loads a program into a main storage device 902 and executes the program.
[0098] The main storage device 902 is, for example, a random access memory (RAM). The auxiliary storage device 903 is, for example, a non-volatile storage device such as a hard disk drive (HDD) or a solid-state drive (SSD). For example, the auxiliary storage device 903 realizes the function of the storage unit 11 in FIGS.
[0099] The input device 904 is, for example, a keyboard, a pointing device, or a combination thereof. The pointing device may be, for example, a mouse, a touch pad, or a touch screen. The output device 905 is, for example, a display, a speaker, or a combination thereof. The display may be a touch screen.
[0100] The input / output interface 907 is connected to a PCIe (Peripheral Component Interconnect express) device or the like, and transmits and receives data to and from the connected device.
[0101] The communication control device 908 is, for example, a wired LAN (Local Area Network) interface, a wireless LAN interface, or a combination thereof. The computer 90 is connected to a network such as a wireless LAN or a wired LAN via the communication control device 908. Specifically, the communication control device 908 may be an external NIC (Network Interface Card) or an on-board network interface controller.
[0102] The storage medium 91 is an optical disk such as a CD (Compact Disc) or a DVD (Digital Versatile Disk), a magneto-optical disk, a magnetic disk, a semiconductor memory card such as a flash memory, etc. The medium drive device 906 is a device that writes and reads data to and from the inserted storage medium 91.
[0103] The program executed by the processor 901 may be pre-installed in the auxiliary storage device 903. Alternatively, the program may be provided in a state stored in the storage medium 91, read from the storage medium 91 by the medium drive device 906, copied to the auxiliary storage device 903, and then loaded into the main storage device 902. Alternatively, the program may be downloaded to the computer 90 from a program provider on a network via the network and the communication control device 908, and installed therein.
[0104] For example, the processor 901 executes a program to realize the functions of the pseudo label generation unit 12, the model output unit 13, the loss calculation unit 14, and the update unit 15 illustrated in FIGS. [Explanation of symbols]
[0105] 1 Learning device 11 Storage section 12 Pseudo label generation unit 13 Model output section 14 Loss calculation section 15 Update section 131 First model output section 132 Second model output section 141 1st Loss Calculation Department 142 Second loss calculation section
Claims
1. A storage unit that stores a plurality of labeled data in which a label representing a correct answer is associated with target data, a plurality of unlabeled data that is target data that is not associated with a correct answer and is greater in number than the labeled data, and a deep learning model; a pseudo label generation unit that generates pseudo labels based on the unlabeled data and the deep learning model and assigns the pseudo labels to the unlabeled data; a first model output unit that acquires the deep learning model and stores it as a first deep learning model, and inputs the unlabeled data into the first deep learning model to obtain a first output value; a second model output unit that acquires the deep learning model and stores it as a second deep learning model, and inputs the labeled data into the second deep learning model to obtain a second output value; a loss calculation unit that calculates a first loss using the first output value obtained by the first model output unit and the pseudo label after the pseudo label generation unit assigns the pseudo label, and calculates a second loss using the second output value obtained by the second model output unit and the label; an update unit that updates the deep learning model based on both the first loss and the second loss calculated by the loss calculation unit; An information processing device comprising:
2. The update unit performs a first update for the first deep learning model and a second update for the second deep learning model based on both the first loss and the second loss.
2. The information processing apparatus according to claim 1,
3. The deep learning model has a feature extraction layer and a classification layer, The information processing device according to claim 2, characterized in that, as the first update and the second update, the update unit performs similar updates on the feature extraction layers included in the first deep learning model and the second deep learning model, and performs different updates on the first identification layer included in the first deep learning model and the second identification layer included in the second deep learning model.
4. The information processing device according to claim 1 , characterized in that the pseudo label generation unit classifies a plurality of the unlabeled data into a predetermined number of clusters based on output values obtained by inputting the plurality of unlabeled data into the deep learning model, and assigns the pseudo label to each of the clusters.
5. A model output unit that integrates the unlabeled data and the labeled data to create integrated data, and inputs the integrated data to the deep learning model to obtain an output value, The loss calculation unit generates an integrated label by integrating the label and the pseudo label included in the labeled data, and calculates a loss based on the output value obtained by the model output unit and the integrated label. The update unit updates the deep learning model based on the loss calculated by the loss calculation unit.
2. The information processing apparatus according to claim 1,
6. An information processing method for performing learning using a deep learning model, comprising: a plurality of labeled data in which a label representing a correct answer is associated with target data; a plurality of unlabeled data which is target data that is not associated with a correct answer and is more in number than the labeled data; generating pseudo labels based on the unlabeled data and the deep learning model, and assigning the pseudo labels to the unlabeled data; obtaining the deep learning model and retaining it as a first deep learning model; inputting the unlabeled data into the first deep learning model to obtain a first output value; obtaining the deep learning model and storing it as a second deep learning model, and inputting the labeled data into the second deep learning model to obtain a second output value; After the pseudo label is assigned, a first loss is calculated using the first output value and the pseudo label; Calculating a second loss using the second output value and the label; The deep learning model is updated based on both the calculated first loss and the calculated second loss.
23. An information processing method comprising:
7. An information processing program for causing a computer to execute learning using a deep learning model, the information processing program including: a plurality of labeled data in which a label representing a correct answer is associated with target data; a plurality of unlabeled data which is target data that is not associated with a correct answer and is greater in number than the labeled data; generating pseudo labels based on the unlabeled data and the deep learning model, and assigning the pseudo labels to the unlabeled data; obtaining the deep learning model and retaining it as a first deep learning model; inputting the unlabeled data into the first deep learning model to obtain a first output value; obtaining the deep learning model and storing it as a second deep learning model, and inputting the labeled data into the second deep learning model to obtain a second output value; After the pseudo label is assigned, a first loss is calculated using the first output value and the pseudo label; Calculating a second loss using the second output value and the label; The deep learning model is updated based on both the calculated first loss and the calculated second loss. An information processing program that causes the computer to execute a process.