Information providing device, information providing method, and information providing program
The information providing device enhances transfer learning by automatically identifying similar datasets and hyperparameters, addressing inefficiencies in conventional methods through feature extraction and similarity calculation, thereby improving transfer learning efficiency and accuracy.
Patent Information
- Application Number
- JP2023523881
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-05-27
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-05-27
AI Technical Summary
Conventional transfer learning relies heavily on developer intuition and manual work for selecting transfer source datasets and tuning hyperparameters, with varying results due to unknown dataset similarity and lack of a standard index, leading to inefficiencies.
An information providing device that extracts features of lower dimension from multiple datasets and calculates similarity using a trained model, automatically identifying similar datasets and optimal hyperparameters through self-supervised learning.
Enables efficient transfer learning by automating the selection of similar datasets and hyperparameters, improving the accuracy and consistency of transfer learning processes.
Smart Images

Figure 0007586314000001 
Figure 0007586314000002 
Figure 0007586314000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an information providing device, an information providing method, and an information providing program. [Background technology]
[0002] Deep neural networks (DNNs) are capable of making highly accurate predictions in image processing and natural language processing. However, training DNNs is costly.
[0003] For example, the cost of training a DNN includes the cost of collecting a dataset including the assignment of correct labels (annotations), the computational cost of improving accuracy, and the tuning cost of exploring multiple hyperparameters for each example.
[0004] Transfer learning has been proposed as a method to reduce such costs when introducing DNNs into business operations.
[0005] Transfer learning is a technique that uses a dataset (source dataset) different from the target dataset or a trained model to perform learning with less data or computational time.
[0006] Transfer learning also includes techniques such as fine-tuning and domain adaptation.
[0007] Fine-tuning is a method in which a model is pre-trained on a source dataset and the trained parameters are used as initial values for training the target dataset.
[0008] Domain adaptation is a method of simultaneously training the source and target datasets using the same model, and solving tasks on the target dataset using knowledge of the source dataset. [Prior art documents] [Non-patent literature]
[0009] [Non-Patent Document 1] Kornblith, Simon, Jonathon Shlens, and Quoc V. Le. "Do better imagenet models transfer better?" Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2019. Summary of the Invention [Problem to be solved by the invention]
[0010] However, conventional techniques have the problem that they may not be able to perform transfer learning efficiently. Conventional transfer learning relies heavily on the developer's intuition and experience, and manual work is required for selecting the source dataset and tuning parameters.
[0011] The dataset that is effective for transfer learning is not self-evident, and the results of transfer learning vary greatly depending on the relationship (similarity) between the target dataset and the source dataset. For example, a model pre-trained on ImageNet (large scale, large volume) may be inferior to a model that has not been pre-trained depending on the target dataset (for example, see Non-Patent Document 1).
[0012] On the other hand, the degree of similarity between data sets is generally unknown, and no de facto standard metric has yet emerged.
[0013] In addition, in transfer learning, it is necessary to select hyperparameters that match the target dataset and the source dataset. On the other hand, deep learning models have many hyperparameters, and tuning is also necessary during transfer learning. [Means for solving the problem]
[0014] In order to solve the above-mentioned problems and achieve the objective, the information provision device is characterized by having a feature extraction unit that extracts multiple feature quantities from a dataset by inputting multiple datasets into a model that outputs lower-dimensional feature quantities than the dataset, and a similarity calculation unit that calculates a similarity between the multiple feature quantities extracted by the feature extraction unit. Effect of the Invention
[0015] According to the present invention, transfer learning can be performed efficiently. [Brief description of the drawings]
[0016] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information providing device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining a method for measuring the similarity. [Diagram 3] FIG. 3 is a diagram for explaining a model learning method. [Figure 4] FIG. 4 is a diagram illustrating the information providing process. [Diagram 5] FIG. 5 is a flowchart showing the flow of the learning process. [Figure 6] FIG. 6 is a flowchart showing the flow of the similarity measurement process. [Figure 7] FIG. 7 is a flowchart showing the flow of the information provision process. [Figure 8] FIG. 8 shows the results of the experiment. [Figure 9] FIG. 9 shows the results of the experiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a computer that executes an information provision program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0017] Hereinafter, an embodiment of an information providing device, an information providing method, and an information providing program according to the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiment described below.
[0018] [Configuration of the first embodiment] 1 is a diagram illustrating a configuration example of an information providing device according to a first embodiment. The information providing device 10 calculates similarity between data sets and provides information based on the calculated similarity. For example, the information providing device 10 provides information for identifying a source data set similar to a target data set in transfer learning.
[0019] In addition, the information providing device 10 performs a learning process of a model for calculating the similarity. The information providing device 10 may calculate the similarity using the trained model, or may provide the trained model to another device or the like.
[0020] As shown in FIG. 1, the information providing device 10 includes an input / output unit 11, a storage unit 12, and a control unit 13.
[0021] The input / output unit 11 is an interface for inputting and outputting data. For example, the input / output unit 11 may be a communication interface such as a network interface card (NIC) for performing data communication with other devices via a network. The input / output unit 11 may also be an interface for connecting an input device such as a mouse or keyboard, and an output device such as a display.
[0022] The storage unit 12 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), an optical disk, etc. The storage unit 12 may be a semiconductor memory in which data can be rewritten, such as a random access memory (RAM), a flash memory, or a non-volatile static random access memory (NVSRAM). The storage unit 12 stores an operating system (OS) and various programs executed by the information providing device 10. The storage unit 12 also stores model information 121.
[0023] The model information 121 is information such as parameters for constructing a model, and is appropriately updated in the learning process. In addition, the updated model information 121 may be output to another device or the like via the input / output unit 11.
[0024] The control unit 13 controls the entire information providing device 10. The control unit 13 is, for example, an electronic circuit such as a central processing unit (CPU), a micro processing unit (MPU), or a graphics processing unit (GPU), or an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 13 also has an internal memory for storing programs and control data that define various processing procedures, and executes each process using the internal memory. The control unit 13 also functions as various processing units by the operation of various programs. For example, the control unit 13 has a feature extraction unit 131, a similarity calculation unit 132, a loss function calculation unit 133, an update unit 134, a candidate extraction unit 135, and a provision unit 136.
[0025] The feature extraction unit 131 extracts a plurality of feature amounts from a dataset by inputting a plurality of datasets to a model that outputs a feature amount of a lower dimension than the dataset.
[0026] The similarity calculation unit 132 calculates the similarity between the multiple feature amounts extracted by the feature extraction unit 131 .
[0027] A method of measuring the similarity by the feature extraction unit 131 and the similarity calculation unit 132 will be described with reference to Fig. 2. Fig. 2 is a diagram for explaining the method of measuring the similarity.
[0028] 2, the feature extraction unit 131 uses a model F to extract features from the data set A and the data set B. The model F is a deep neural network for feature extraction.
[0029] The data set includes a plurality of data samples. Furthermore, the feature extraction unit 131 extracts a feature amount for each data sample.
[0030] In the example in Figure 2, data set A contains I data samples x A i (where i is an integer from 0 to I). The feature extraction unit 131 extracts feature quantities f A i Extract.
[0031] Here, since the data sets used in DNN are high-dimensional, it is difficult to directly measure the similarity between data sets. Therefore, the feature extraction unit 131 extracts feature quantities by reducing the dimensionality of the data sets as shown in FIG. 2. For example, A i x A i It is lower dimensional than
[0032] Furthermore, the feature extraction unit 131 aggregates the extracted features. In the example of FIG. 2, the feature extraction unit 131 aggregates the extracted features f A i We use one feature f´ A The information is summarized as follows.
[0033] In this way, the feature extraction unit 131 can aggregate the feature amounts output by the model for each data sample included in the data set into the feature amount of one data sample. For example, the feature extraction unit 131 can use statistics such as the average and variance of each element of a plurality of data samples as the aggregated feature amount.
[0034] Then, the similarity calculation unit 132 calculates the similarity between the feature amounts aggregated by the feature extraction unit 131. For example, the aggregated feature amount f′ A and f´ B If f is a vector, the similarity calculation unit 132 calculates the vector f′ A and vector f´ B Distance d AB The similarity calculation unit 132 may calculate the 2-Wasserstein distance as the distance between the vectors.
[0035] In addition, the feature extraction unit 131 extracts features by a model trained by self-supervised learning using a source dataset in transfer learning. Then, the similarity calculation unit 132 calculates the similarity between the features of the source dataset and the features of a target dataset in transfer learning.
[0036] The loss function calculation unit 133 calculates a loss function for learning the model, and the update unit 134 updates the parameters of the model so that the loss function is optimized.
[0037] The parameters of model F are stored in the storage unit 12 as model information 121. The update unit 134 updates the model information 121.
[0038] A learning method for model F will be described with reference to Fig. 3. Fig. 3 is a diagram for explaining the model learning method.
[0039] Model F is used to measure the similarity between target data and multiple source data when identifying source data similar to the target data in transfer learning.
[0040] At this time, the information providing device 10 is assumed to have previously trained the model F on an arbitrary task such as classification using the source data set group. In the embodiment, the information providing device 10 uses self-supervised learning to learn the model F. Self-supervised learning does not require annotations, making it easy to handle multiple data sets together.
[0041] In addition, the information providing device 10 uses MoCo (Reference: He, Kaiming, et al. "Momentum contrast for unsupervised visual representation learning." Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2020.), which is one of the contrastive learning methods, as a self-supervised learning method.
[0042] As shown in FIG. 3, the loss function calculation unit 133 calculates a plurality of data sets (D0 to D N ) into model F, and based on the obtained features, the loss function L of contrastive loss is q Calculate.
[0043] Here, contrastive loss is the loss in the task of generating a query and a ground truth key image from an input image using two different image transformations, and finding the correct combination of the query and ground truth key from a set of keys obtained from other images via a DNN.
[0044] The q on the right-hand side of the loss function in Figure 3 is the output of model F obtained from the query image. + is the output of model F obtained from the correct key obtained by augmenting the same image as the query. K is the total number of key images including the correct key. τ is the temperature coefficient.
[0045] The candidate extraction unit 135 and the provision unit 136 support actual transfer learning by providing information that identifies source data similar to the target data. The information provision process will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining the information provision process.
[0046] Dataset D in Figure 4 T is the target data set. Also, data sets D0 to D N are multiple source datasets.
[0047] The similarity calculator corresponds to the feature extraction unit 131 and the similarity calculation unit 132 that use the trained model F.
[0048] At this time, the similarity calculation unit 132 calculates one target data set (D T ) and multiple source datasets (D0 to D T ) and calculate the similarity between the features.
[0049] Then, based on the calculated similarity, the candidate extraction unit 135 extracts, as candidates, source datasets whose feature similarity with the target dataset is at or above a predetermined rank.
[0050] Furthermore, the providing unit 136 provides the user with information for identifying the source datasets extracted as candidates from among the source datasets.
[0051] In the example of FIG. 4, the candidate extraction unit 135 calculates the similarity d 0T , d 1T , …, d NT Create a ranking in order of similarity. For example, the top three similarities d NT , d 1T , d 0T The source dataset D corresponds to N , D1, D0 are extracted.
[0052] The providing unit 136 provides the extracted transfer source data set D N, D1, D0 are the corresponding hyperparameters H N , H1, and H0 are provided to the user. Note that the optimal hyperparameters for each source dataset are assumed to have already been determined by a method such as grid search during the past model construction process.
[0053] In this way, a plurality of combinations of source data and hyperparameters provided together may be used together in transfer learning.
[0054] [Processing of the first embodiment] The flow of processing by the information providing device 10 will be described with reference to the flowcharts shown in FIGS.
[0055] 5 is a flowchart showing the flow of the learning process. As shown in FIG 5, first, the information providing device 10 reads learning data from the source data set group (step S101).
[0056] Next, the information providing device 10 extracts features from the learning data by the DNN model F (step S102).
[0057] Here, the information providing device 10 calculates a loss function of the pre-learning task in the feature space (step S103). Then, the information providing device 10 updates the parameters of the model F by backpropagation of the loss function (step S104).
[0058] At this time, if the maximum number of learning steps>the number of learning steps (Step S105, True), the information providing device 10 returns to Step S101 and repeats the process. On the other hand, if the maximum number of learning steps>the number of learning steps is not satisfied (Step S105, False), the information providing device 10 ends the process.
[0059] Fig. 6 is a flowchart showing the flow of the similarity measurement process. As shown in Fig. 6, first, the information providing device 10 reads a data sample from the transfer source data set (step S201).
[0060] Next, the information providing device 10 extracts features from the source data sample by the DNN model F (step S202). Furthermore, the information providing device 10 aggregates the feature vectors for each source data sample into a single feature vector (for example, the mean or variance) (step S203).
[0061] The information providing device 10 reads a data sample from the target data set (step S204).
[0062] Then, the information providing device 10 extracts features of the target data sample by the DNN model F (step S205). Furthermore, the information providing device 10 aggregates the feature vectors for each target data sample into a single feature vector, similar to the source data set (step S206).
[0063] The information providing device 10 calculates the similarity between the feature vectors of the aggregated target dataset and the transition source dataset, for example, by the 2-Wasserstein distance (step S207).
[0064] 7 is a flowchart showing the flow of the information providing process. First, the information providing device 10 calculates the similarity between the target dataset and the N source datasets (step S301).
[0065] Next, the information providing device 10 calculates the data set similarity {d iT} j N The information providing device 10 sorts the source dataset (distance: ascending order / score: descending order) by (step S302). Then, the information providing device 10 extracts the Top-K source dataset ids from the ranking obtained by the sorting (K≦N: any integer) (step S303).
[0066] Here, the information providing device 10 reads the datasets and hyperparameters associated with the K source dataset ids (step S304). Then, the information providing device 10 issues a URI (Uniform Resource Identifier) that can be downloaded by the user, and outputs the datasets and hyperparameters (step S305).
[0067] [Advantages of the First Embodiment] As described above, the feature extraction unit 131 extracts multiple feature quantities from a data set by inputting multiple data sets to a model that outputs feature quantities of a lower dimension than the data sets. The similarity calculation unit 132 calculates the similarity between the multiple feature quantities extracted by the feature extraction unit 131.
[0068] In this way, the information providing device 10 can automatically calculate the similarity between data sets. As a result, according to the present embodiment, similar data sets can be identified, and transfer learning can be efficiently performed.
[0069] The feature extraction unit 131 aggregates the features output by the model for each data sample included in the data set into the features of one data sample. The similarity calculation unit 132 calculates the similarity between the features aggregated by the feature extraction unit 131.
[0070] As a result, according to this embodiment, the distance between feature amounts can be easily calculated.
[0071] The feature extraction unit 131 extracts features from a model trained by self-supervised learning using a source dataset in transfer learning. The similarity calculation unit 132 calculates the similarity between the features of the source dataset and the features of a target dataset in transfer learning.
[0072] In this manner, in this embodiment, self-supervised learning that does not require annotations can be used to efficiently learn a model that measures similarity.
[0073] The similarity calculation unit 132 calculates the similarity between the features of one target dataset and each of the multiple source datasets. The provision unit 136 provides the user with information for identifying a source dataset whose feature similarity with the target dataset is equal to or higher than a predetermined rank among the source datasets.
[0074] This allows the information providing device 10 to recommend a source dataset similar to the target dataset to the user. Therefore, according to the present embodiment, transfer learning can be performed efficiently.
[0075] [experiment] An experiment was conducted by actually implementing the above embodiment, and a source dataset and hyperparameters (architecture) were selected based on the similarity of the datasets using the above embodiment.
[0076] The experimental setup is as follows. Dataset Target dataset: Oxford Pets Source dataset: A subset of ImageNet divided into 11 classes Neural network architecture: Experiment 1: ResNet-50 Experiment 2: ResNet-50, ResNet-101, ResNext-50-32x4d, ResNext-101-32-4d, Wide-ResNet-50, Wide-ResNet-101
[0077] (Experiment 1) Fig. 8 shows the results of experiment 1 in which the source dataset was selected based on the similarity of the datasets. Fig. 8 shows the results of the experiment.
[0078] In the example of Figure 8, a feature extractor (Model F, self-supervised learning model Moco) was trained using all data in the source dataset. Then, the feature extractor was used to measure the dataset similarity between the target dataset and the source dataset.
[0079] Furthermore, we measured the test accuracy by fine-tuning the trained model with each subset as the transfer data set on Oxford Pets. Figure 8 visualizes the correlation between dataset similarity and test accuracy (ACC@1).
[0080] From the correlation shown in FIG. 8, it can be said that the embodiment is effective in selecting an effective source dataset.
[0081] (Experiment 2) Figure 9 shows the results of experiment 2, in which hyperparameters (architectures) were selected based on the similarity of the data sets. Figure 9 shows the results of the experiment.
[0082] In the example of Figure 9, we trained class classification with each architecture using the target dataset and the source dataset, and measured the test accuracy. Then, we measured the dataset similarity between the target dataset and the source dataset using a feature extractor.
[0083] Furthermore, we created a ranking of architectures by test accuracy for each dataset (in descending order of test accuracy) and measured the Mean Average Precision (MAP) of the ranking between the target dataset and the source dataset. Figure 9 visualizes the correlation between dataset similarity and MAP.
[0084] From the correlation shown in FIG. 9, it can be said that the embodiment is effective in selecting effective hyperparameters.
[0085] [System configuration, etc.] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or a part of it can be functionally or physically distributed or integrated in any unit according to various loads, usage conditions, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic. Note that the program may be executed not only by the CPU but also by other processors such as a GPU.
[0086] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.
[0087] [program] As an embodiment, the information providing device 10 can be implemented by installing an information providing program that executes the above-mentioned processes as package software or online software on a desired computer. For example, the information providing program can be executed by an information processing device, so that the information processing device can function as the information providing device 10. The information processing device here includes desktop or notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone System), as well as slate terminals such as PDA (Personal Digital Assistant), etc.
[0088] The information providing device 10 may also be implemented as a server device that provides services related to the above-mentioned processing to a client terminal device used by a user. For example, the server device is implemented as a server device that provides a similarity measurement service that receives a target dataset and a plurality of source datasets as inputs and outputs the similarity between the target dataset and each source dataset. In this case, the server device may be implemented as a Web server or as a cloud that provides services related to the above-mentioned processing by outsourcing.
[0089] 12 is a diagram showing an example of a computer that executes an information provision program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.
[0090] The memory 1010 includes a read only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.
[0091] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the information providing device 10 is implemented as a program module 1093 in which computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing the same process as the functional configuration of the information providing device 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced with an SSD (Solid State Drive).
[0092] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.
[0093] Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)). The program module 1093 and the program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070. [Explanation of symbols]
[0094] 10 Information provision device 11 Input / output section 12 Storage section 121 Model Information 13 Control section 131 Feature Extraction Unit 132 Similarity calculation part 133 Loss function calculation part 134 Update Department 135 Candidate Extraction Unit 136 Provision Department
Claims
1. A model that outputs, from a dataset, feature quantities of lower dimensions than the dataset, and a feature extraction unit that extracts a plurality of feature quantities by inputting a plurality of datasets to a model that has been trained by self-supervised learning using a transfer source dataset in transfer learning; a similarity calculation unit that calculates a similarity between the feature amount of the transfer source dataset extracted by the feature extraction unit and the feature amount of a target dataset in the transfer learning; An information providing device comprising:
2. The feature extraction unit aggregates the features output by the model for each data sample included in the data set into a feature of one data sample; 2. The information providing device according to claim 1, wherein the similarity calculation unit calculates a similarity between the feature amounts collected by the feature extraction unit.
3. The information providing unit may further include a providing unit for providing information to a user. The similarity calculation unit calculates a similarity between features between one target dataset and each of a plurality of source datasets, The information providing device according to claim 1 or 2, characterized in that the providing unit provides a user with information for identifying a source dataset among the source datasets having a feature similarity with the target dataset that is equal to or higher than a predetermined rank.
4. An information providing method executed by an information providing device, comprising: A feature extraction step of extracting a plurality of features by inputting a plurality of datasets into a model that outputs, from a dataset, features of a lower dimension than the dataset, and that has been trained by self-supervised learning using a transfer source dataset in transfer learning; a similarity calculation step of calculating a similarity between the feature amount of the transfer source dataset extracted by the feature extraction step and the feature amount of a target dataset in the transfer learning; 13. An information providing method comprising:
5. An information providing program for causing a computer to function as the information providing device according to any one of claims 1 to 3.
Citation Information
Patent Citations
Learning device and program
JP2014022837A
Learning device, control method of learning device, and program
JP2016224821A
Data extraction apparatus, data extraction method and data extraction program
JP2020149080A
Method to measure similarity of datasets for given ai task
US20200364611A1
Augmentation device, augmentation method, and augmentation program
WO2020170803A1
