Deep transfer learning method, device, electronic device and storage medium
By annotating and dividing the source domain image, the candidate pretrained model is trained and the similarity calculation is performed, and the target pretrained model is selected for fine adjustment, which solves the problem of negative transfer in transfer learning and improves the effect of transfer learning.
Patent Information
- Application Number
- CN202010103333.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-02-18
- Publication Date
- 2025-08-05
- Estimated Expiration
- 2040-02-18
AI Technical Summary
The existing transfer learning methods are prone to negative transfers during the target domain transfer process, resulting in less significant results.
By obtaining source domain images of multiple categories from the target address, labeling and dividing them into multiple sub-source domain data sets, using each sub-source domain data set to train candidate pre-training models, calculate the similarity between the target domain data set and the sub-source domain data set, and selecting the target pre-training model and fine-tuning to obtain the best classification model.
It effectively avoids negative transfers in the transfer learning process and improves the effect of transfer learning.
Smart Images
Figure CN113344016B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision technology, and in particular, to a deep transfer learning method, apparatus, electronic device, and storage medium. Background Art
[0002] Computer vision is a science that studies how to enable machines to "see" and "process". Its goal is to enable machines to observe and understand the world through vision like humans and have the ability to adapt to the environment independently. In the field of computer vision, transfer learning is a widely used technology that can transfer the knowledge learned in one field to another, playing the role of accelerating the model training speed and improving the model prediction accuracy. Currently, transfer learning trains models based on large-scale general datasets. This general dataset can be face image data collected at the gate channel, or face image data obtained from an open-source database, or vehicle image data captured on the highway, etc. Then, the trained model is fine-tuned. However, large-scale general datasets are prone to negative transfer during the process of migrating to the target domain, and the effect of transfer learning is not significant. Summary of the Invention
[0003] In view of the above problems, this application provides a deep transfer learning method, apparatus, electronic device, and storage medium, which is beneficial to avoiding the occurrence of negative transfer during the transfer learning process, thereby improving the effect of transfer learning.
[0004] To achieve the above object, in the first aspect of the embodiments of this application, a deep transfer learning method is provided. The method includes:
[0005] Obtain source domain images belonging to multiple categories from the target address, label the source domain images, and obtain a source domain dataset with category labels;
[0006] Divide the source domain dataset into multiple sub-source domain datasets, and use each sub-source domain dataset to train a corresponding candidate pre-trained model;
[0007] When target domain images belonging to the target category are input, label the target domain images to obtain a target domain dataset with category labels, and calculate the similarity between the target domain dataset and each sub-source domain dataset;
[0008] Select a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain dataset and each sub-source domain dataset;
[0009] Fine-tune the target pre-trained model using the target domain dataset to obtain the best classification model for the target domain dataset.
[0010] In combination with the first aspect, in a possible implementation manner, calculating the similarity between the target domain dataset and each of the sub-source domain datasets includes:
[0011] Calculating the first earth mover's distance between the target domain dataset and each of the sub-source domain datasets;
[0012] Obtaining the similarity between the target domain dataset and each of the sub-source domain datasets according to the first earth mover's distance.
[0013] In combination with the first aspect, in a possible implementation manner, selecting a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain dataset and each of the sub-source domain datasets includes:
[0014] Obtaining a target sub-source domain dataset whose similarity with the target domain dataset is greater than or equal to a preset value;
[0015] Determining the candidate pre-trained model trained by using the target sub-source domain dataset as the target pre-trained model.
[0016] In combination with the first aspect, in a possible implementation manner, dividing the source domain dataset into multiple sub-source domain datasets includes:
[0017] Calculating the second earth mover's distance between any two data in the source domain dataset;
[0018] Obtaining the similarity between the two arbitrary data according to the second earth mover's distance;
[0019] Dividing the source domain dataset into multiple sub-source domain datasets based on the similarity between the two arbitrary data.
[0020] In combination with the first aspect, in a possible implementation manner, training a corresponding candidate pre-trained model by using each of the sub-source domain datasets includes:
[0021] Inputting each of the sub-source domain datasets into a preset neural network for training to obtain the candidate pre-trained model corresponding to each of the sub-source domain datasets;
[0022] The method further includes:
[0023] If the similarity between the target domain dataset and each of the sub-source domain datasets is less than the preset value, inputting the target domain dataset into the neural network for training to obtain the best classification model of the target domain dataset.
[0024] A second aspect of the embodiments of the present application provides a deep transfer learning device, and the device includes:
[0025] A source domain data acquisition module, configured to acquire source domain images belonging to multiple categories from a target address, label the source domain images, and obtain a source domain data set with category labels;
[0026] A first model acquisition module, configured to divide the source domain data set into multiple sub-source domain data sets, and use each sub-source domain data set to train a corresponding candidate pre-trained model;
[0027] A similarity calculation module, configured to, when a target domain image belonging to a target category is input, label the target domain image to obtain a target domain data set with category labels, and calculate the similarity between the target domain data set and each sub-source domain data set;
[0028] A second model acquisition module, configured to select a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain data set and each sub-source domain data set;
[0029] A model fine-tuning module, configured to fine-tune the target pre-trained model using the target domain data set to obtain an optimal classification model for the target domain data set.
[0030] In a third aspect of the embodiments of the present application, an electronic device is provided. The electronic device includes an input device and an output device, and further includes a processor adapted to implement one or more instructions; and a computer storage medium storing one or more instructions, the one or more instructions being adapted to be loaded and executed by the processor to perform the following steps:
[0031] Acquire source domain images belonging to multiple categories from a target address, label the source domain images, and obtain a source domain data set with category labels;
[0032] Divide the source domain data set into multiple sub-source domain data sets, and use each sub-source domain data set to train a corresponding candidate pre-trained model;
[0033] When a target domain image belonging to a target category is input, label the target domain image to obtain a target domain data set with category labels, and calculate the similarity between the target domain data set and each sub-source domain data set;
[0034] Select a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain data set and each sub-source domain data set;
[0035] Fine-tune the target pre-trained model using the target domain data set to obtain an optimal classification model for the target domain data set.
[0036] In a fourth aspect of the embodiments of the present application, a computer storage medium is provided. The computer storage medium stores one or more instructions, and the one or more instructions are adapted to be loaded and executed by a processor to perform the following steps:
[0037] Obtain source domain images belonging to multiple categories from a target address, label the source domain images, and obtain a source domain data set with category labels;
[0038] Divide the source domain data set into multiple sub-source domain data sets, and use each sub-source domain data set to train a corresponding candidate pre-trained model;
[0039] When a target domain image belonging to a target category is input, label the target domain image to obtain a target domain data set with category labels, and calculate the similarity between the target domain data set and each sub-source domain data set;
[0040] Select a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain data set and each sub-source domain data set;
[0041] Fine-tune the target pre-trained model using the target domain data set to obtain the best classification model for the target domain data set.
[0042] The above solution of the present application has at least the following beneficial effects: It can be seen that in the embodiments of the present application, source domain images belonging to multiple categories are obtained from a target address, the source domain images are labeled to obtain a source domain data set with category labels; the source domain data set is divided into multiple sub-source domain data sets, and each sub-source domain data set is used to train a corresponding candidate pre-trained model; when a target domain image belonging to a target category is input, the target domain image is labeled to obtain a target domain data set with category labels, and the similarity between the target domain data set and each sub-source domain data set is calculated; according to the similarity between the target domain data set and each sub-source domain data set, a target pre-trained model is selected from the candidate pre-trained models; the target domain data set is used to fine-tune the target pre-trained model to obtain the best classification model for the target domain data set. In this way, the source domain data set is divided into multiple sub-source domain data sets of different categories, a candidate pre-trained model is trained on each sub-source domain data set, and then a target pre-trained model is selected through the similarity between the target domain data set and the sub-source domain data sets. The sub-source domain data set corresponding to the target pre-trained model is used as the source domain of the target domain data set, and fine-tuning the target pre-trained model completes the transfer learning from a specific sub-source domain data set (with high similarity to the target domain data set) to the target domain data set. Using only one specific sub-source domain data set for transfer learning is beneficial to avoiding the negative transfer situation in the transfer learning process caused by the differences between the source domain data sets, thereby improving the effect of transfer learning. Brief Description of Drawings
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0044] Figure 1 It is a schematic diagram of a network architecture provided by an embodiment of the present application;
[0045] Figure 2 It is a schematic flowchart of a deep transfer learning method provided by an embodiment of the present application;
[0046] Figure 3 It is an example diagram of splitting a source domain dataset into sub-source domain datasets provided by an embodiment of the present application;
[0047] Figure 4 It is a schematic flowchart of another deep transfer learning method provided by an embodiment of the present application;
[0048] Figure 5 It is a schematic flowchart of another deep transfer learning method provided by an embodiment of the present application;
[0049] Figure 6 It is a schematic diagram of the structure of a deep transfer learning device provided by an embodiment of the present application;
[0050] Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed Embodiments
[0051] In order to enable those skilled in the art to better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of them. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.
[0052] As used in the description, claims and drawings of this application, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or devices. In addition, terms such as "first", "second" and "third" are used to distinguish different objects, rather than to describe a specific order.
[0053] First, in combination with the relevant drawings, an example is given to introduce the network system architecture to which the solution of the embodiment of this application may be applied. Please refer to Figure 1 , Figure 1 which is a schematic diagram of a network architecture provided for an embodiment of this application. As Figure 1 shown, it includes a server and a terminal. Among them, the server and the terminal communicate through a network, and the networks that can be used for their communication include but are not limited to virtual private networks, local area networks, and metropolitan area networks. The terminal can be used for users to label the collected source domain dataset and target domain dataset, and send requests for classifying the source domain dataset, training the classification model of the sub-source domain dataset, calculating the similarity between the sub-source domain data and the target domain dataset, and fine-tuning the classification model, etc. in the form of code or other forms, and can receive the relevant processing results returned by the server, such as: similarity calculation results, classification model fine-tuning results, etc. The terminal includes but is not limited to desktop computers, laptop computers, tablet computers, and supercomputers. The server can be a single server, a server cluster, or a cloud server, and relevant algorithms are deployed on its processor, such as: similarity algorithms, neural network algorithms, etc., which can be used to perform corresponding processing according to various requests submitted by the terminal side, such as: running a neural network algorithm to extract data feature values and training a classification model on the sub-source domain dataset, running a similarity algorithm to calculate the similarity between each sub-source domain dataset and the target domain dataset, etc. The server is the execution entity of the entire network system. It is precisely because of Figure 1 the network system architecture shown that the deep transfer learning method in this application can be implemented.
[0054] Based on Figure 1 the network architecture shown, the deep transfer learning method provided for the embodiment of this application will be elaborated in detail below in combination with other drawings.
[0055] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a deep transfer learning method provided for an embodiment of this application. As Figure 2 shown, it includes steps S21 - S25:
[0056] S21. Obtain source domain images belonging to multiple categories from the target address, annotate the source domain images, and obtain a source domain data set with category labels.
[0057] In a specific embodiment of the present application, the target address includes but is not limited to a third-party platform (such as: a search engine address), an open-source database on the network, a local database (such as: a database for storing images captured by cameras, video cameras, etc.). The multiple categories cover most scenarios and include images in different fields, such as: animals, buildings, faces, vehicles, etc., so that the trained classification model can acquire knowledge in different fields. Here, the annotation of the source domain data can be manual annotation or machine annotation, and no specific limitation is made.
[0058] S22. Divide the source domain data set into multiple sub-source domain data sets, and use each sub-source domain data set to train a corresponding candidate pre-trained model.
[0059] In a specific embodiment of the present application, after obtaining the source domain data set, the source domain data set is divided into Figure 3 n sub-source domain data sets as shown. The distributions between different sub-source domain data sets are different, while the data within the same sub-source domain data set follows the same distribution. Specifically, the source domain data set is input into a preset convolutional neural network for eigenvalue extraction to obtain the eigenvalues of each data in the source domain data set. Based on the eigenvalues, the similarity between any two data in the source domain data set is calculated, and the data with a similarity greater than or equal to a specific threshold is divided into a sub-source domain data set. Thus, the sub-source domain data set 1, sub-source domain data set 2... sub-source domain data set i... sub-source domain data set n as shown in Figure 3 are obtained. The similarity between any two of the above data can be measured by the Earth Mover's Distance (EMD), or by the Euclidean distance, or by the Manhattan distance, etc.
[0060] After dividing into n sub-source domain data sets, a classification model is trained on each sub-source domain data set. Specifically, a preset neural network is used for training. For example, the sub-source domain data set 1 is input into a convolutional neural network for classification prediction. After multiple iterations until the loss function value of the neural network converges, the classification model corresponding to the sub-source domain data set 1 is obtained. The classification model corresponding to each sub-source domain data set is used as a candidate pre-trained model for transfer learning of the target domain data set. Optionally, the above neural network can be VGGNet, ResNet, MobileNet, etc., and no specific limitation is made here.
[0061] S23. When a target domain image belonging to the target category is input, annotate the target domain image to obtain a target domain dataset with category labels, and calculate the similarity between the target domain dataset and each of the sub-source domain datasets.
[0062] In a specific embodiment of this application, after preparing a candidate pre-trained model, input the target domain dataset, which is an image from a specific scenario or belongs to a specific category. The category it belongs to can be one of the multiple categories that the source domain images belong to, for example: face images, or it can be a category outside the multiple categories that the source domain images belong to, for example: furniture. Among them, the annotation of the target domain image is the same as that of the source domain image. After the server obtains the target domain dataset, calculate the similarity between the target domain dataset and each sub-source domain dataset. First, input the target domain dataset into a preset convolutional neural network for eigenvalue extraction. Based on the eigenvalues of the target domain dataset and the eigenvalues of each sub-source domain dataset extracted in advance, calculate the EMD distance between the target domain dataset and each sub-source domain dataset using the following formula:
[0063]
[0064] where, d ij = ||g(s i ) - g(t i )||, S represents the sub-source domain dataset, T represents the target domain dataset, d ij represents the Euler distance between the data in the sub-source domain dataset and the data in the target domain dataset, g(s i ) represents the eigenvalue of the data in the sub-source domain dataset, g(t i ) represents the eigenvalue of the data in the target domain dataset, f ij represents the optimal solution of the EMD optimization problem, m represents the number of data in the sub-source domain dataset, and n represents the number of data in the target domain dataset. After obtaining the EMD distance between the target domain dataset and each sub-source domain dataset, use the following formula to calculate the similarity between the target domain dataset and each sub-source domain dataset: similarity(S,T) = e -λd(S,T) , where λ is a preset constant, and similarity(S,T) is the similarity between the sub-source domain dataset and the target domain dataset.
[0065] S24. Select a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain dataset and each of the sub-source domain datasets.
[0066] In a specific embodiment of the present application, after obtaining the similarity between the target domain dataset and each sub-source domain dataset, the similarity is screened, and the target sub-source domain datasets with similarities greater than or equal to a preset value are selected from the n sub-source domain datasets. The candidate pre-trained model obtained by training on the target sub-source domain datasets is determined as the target pre-trained model for transfer learning of the target domain dataset. For example: If Figure 3 only the similarity between the sub-source domain dataset 2 and the target domain dataset is greater than or equal to the preset value, then the candidate pre-trained model corresponding to the resource dataset 2 is determined as the target pre-trained model. Of course, in order to ensure the effect of transfer learning, the preset value can be set to a relatively high value here, so as to facilitate the selection of target sub-source domain datasets that are closer to the target domain dataset.
[0067] S25, fine-tuning the target pre-trained model using the target domain dataset to obtain the best classification model for the target domain dataset.
[0068] In a specific embodiment of the present application, after selecting the target pre-trained model, the target domain dataset is input into the target pre-trained model for fine-tuning, in order to update parameters such as the weights and biases of some layers of the target pre-trained model, making the target pre-trained model more suitable for the target domain dataset. The target pre-trained model with fixed parameters after training is the best classification model obtained by the target domain dataset through transfer learning.
[0069] It can be seen that in the embodiments of this application, source domain images belonging to multiple categories are obtained from a target address, the source domain images are labeled to obtain a source domain dataset with category labels; the source domain dataset is divided into multiple sub-source domain datasets, and corresponding candidate pre-trained models are trained using each sub-source domain dataset; when a target domain image belonging to a target category is input, the target domain image is labeled to obtain a target domain dataset with category labels, and the similarity between the target domain dataset and each sub-source domain dataset is calculated; according to the similarity between the target domain dataset and each sub-source domain dataset, a target pre-trained model is selected from the candidate pre-trained models; the target pre-trained model is fine-tuned using the target domain dataset to obtain the best classification model for the target domain dataset. In this way, the source domain dataset is divided into multiple sub-source domain datasets of different categories, a candidate pre-trained model is trained on each sub-source domain dataset, and then a target pre-trained model is selected through the similarity between the target domain dataset and the sub-source domain datasets. The sub-source domain dataset corresponding to the target pre-trained model is used as the source domain of the target domain dataset, and fine-tuning the target pre-trained model completes the transfer learning from a specific sub-source domain dataset (with high similarity to the target domain dataset) to the target domain dataset. Using only one specific sub-source domain dataset for transfer learning helps to avoid the situation of negative transfer during the transfer learning process caused by differences between source domain datasets, thereby improving the effect of transfer learning.
[0070] Please refer to Figure 4 , Figure 4 which is a schematic flowchart of another deep transfer learning method provided by the embodiments of this application. As Figure 4 shown, it includes steps S41 - S47:
[0071] S41, Obtain source domain images belonging to multiple categories from a target address, label the source domain images to obtain a source domain dataset with category labels;
[0072] S42, Divide the source domain dataset into multiple sub-source domain datasets;
[0073] S43, Train corresponding candidate pre-trained models using each sub-source domain dataset;
[0074] S44, When a target domain image belonging to a target category is input, label the target domain image to obtain a target domain dataset with category labels, and calculate the first earth mover's distance between the target domain dataset and each sub-source domain dataset;
[0075] S45, According to the first earth mover's distance, obtain the similarity between the target domain dataset and each sub-source domain dataset;
[0076] S46. Select a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain dataset and each of the sub-source domain datasets;
[0077] S47. Fine-tune the target pre-trained model using the target domain dataset to obtain the best classification model for the target domain dataset.
[0078] In a possible implementation manner, the dividing the source domain dataset into multiple sub-source domain datasets includes:
[0079] Calculate the second Earth Mover's Distance (EMD) between any two data in the source domain dataset;
[0080] Obtain the similarity between the two data according to the second EMD;
[0081] Based on the similarity between the two data, divide the source domain dataset into multiple sub-source domain datasets.
[0082] In this implementation manner, the EMD is also used as the measure of the similarity between the data in the source domain dataset. The preset convolutional neural network is used to extract the feature values of the data in the source domain dataset, and then the above-mentioned d(S,T) calculation formula is used to calculate the second EMD between any two data (here, S and T respectively represent any two data in the source domain data). Based on the second EMD, the similarity between the two data is calculated. If the similarity is greater than or equal to a specific threshold, it is considered that the two data can be divided into the same sub-source domain dataset, and thus the source domain dataset is divided into multiple sub-source domain datasets.
[0083] In a possible implementation manner, the method further includes:
[0084] If the similarity between the target domain dataset and each of the sub-source domain datasets is less than the preset value, input the target domain dataset into the neural network for training to obtain the best classification model for the target domain dataset.
[0085] In this implementation manner, when the similarity between the target domain dataset and each sub-source domain dataset is less than the preset value, it means that the distributions of all sub-source domain datasets and the target domain dataset are inconsistent. If the candidate pre-trained model is continued to be selected for transfer learning, negative transfer may occur. At this time, directly using the target domain dataset to train the neural network to obtain the best classification model for the target domain dataset is beneficial to avoid the situation of negative transfer.
[0086] It should be noted that Figure 4 The steps S41 - S47 shown in Figure 2The relevant descriptions have been given in the illustrated embodiments, and the same or similar beneficial effects can be achieved. To avoid repetition, they will not be elaborated here.
[0087] It can be seen that in the embodiment of the present application, source domain images belonging to multiple categories are obtained from a target address, the source domain images are labeled to obtain a source domain data set with category labels; the source domain data set is divided into multiple sub-source domain data sets; a corresponding candidate pre-trained model is trained using each sub-source domain data set; when a target domain image belonging to a target category is input, the target domain image is labeled to obtain a target domain data set with category labels, and the first earth mover's distance between the target domain data set and each sub-source domain data set is calculated; according to the first earth mover's distance, the similarity between the target domain data set and each sub-source domain data set is obtained; according to the similarity between the target domain data set and each sub-source domain data set, a target pre-trained model is selected from the candidate pre-trained models; the target pre-trained model is fine-tuned using the target domain data set to obtain the best classification model for the target domain data set. In this way, the source domain data set is divided into multiple sub-source domain data sets of different categories, a candidate pre-trained model is trained on each sub-source domain data set, and then a target pre-trained model is selected through the similarity between the target domain data set and the sub-source domain data sets. The sub-source domain data set corresponding to the target pre-trained model is used as the source domain of the target domain data set, and fine-tuning the target pre-trained model completes the transfer learning from a specific sub-source domain data set (with high similarity to the target domain data set) to the target domain data set. Using only one specific sub-source domain data set for transfer learning helps to avoid the negative transfer situation during the transfer learning process caused by the differences between source domain data sets, thereby improving the effect of transfer learning.
[0088] To better understand the transfer learning solution proposed in the present application, now Figure 5 a simple description of the embodiment of the present application is given. As Figure 5 shown, the embodiment of the present application is divided into two stages. The first stage is mainly the acquisition and processing stage of the source domain data set, including obtaining the source domain data set from the terminal, calculating the similarity between any two data in the source domain data set to divide the source domain data set into multiple sub-source domain data sets, and training a candidate pre-trained model on each sub-source domain data set. The second stage is mainly to process the target domain data set input by the terminal. First, the similarity between the target domain data set and each sub-source domain data set is calculated, and then it is detected whether there is a sub-source domain data set with a similarity greater than or equal to a preset value. If so, the candidate pre-trained model trained on this sub-source domain data set is used as the target pre-trained model and fine-tuned. If not, the sub-source domain data set is no longer used for transfer learning, and the target domain data set is directly used to train the classification model to avoid the situation of negative transfer, resulting in a poor effect of transfer learning.
[0089] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of a deep transfer learning device provided by an embodiment of this application. As Figure 6 shown, the device includes:
[0090] A source domain data acquisition module 61, configured to acquire source domain images belonging to multiple categories from a target address, label the source domain images, and obtain a source domain data set with category labels;
[0091] A first model acquisition module 62, configured to divide the source domain data set into multiple sub-source domain data sets, and train a corresponding candidate pre-trained model using each sub-source domain data set; A similarity calculation module 63, configured to, when a target domain image belonging to a target category is input, label the target domain image to obtain a target domain data set with category labels, and calculate the similarity between the target domain data set and each sub-source domain data set;
[0092] A second model acquisition module 64, configured to select a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain data set and each sub-source domain data set;
[0093] A model fine-tuning module 65, configured to fine-tune the target pre-trained model using the target domain data set to obtain an optimal classification model for the target domain data set.
[0094] In a possible implementation manner, in terms of calculating the similarity between the target domain data set and each sub-source domain data set, the similarity calculation module 63 is specifically configured to:
[0095] Calculate the first earth mover's distance between the target domain data set and each sub-source domain data set;
[0096] Obtain the similarity between the target domain data set and each sub-source domain data set according to the first earth mover's distance.
[0097] In a possible implementation manner, in terms of selecting a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain data set and each sub-source domain data set, the second model acquisition module 64 is specifically configured to:
[0098] Obtain a target sub-source domain data set whose similarity with the target domain data set is greater than or equal to a preset value;
[0099] Determine the candidate pre-trained model trained using the target sub-source domain data set as the target pre-trained model.
[0100] In a possible implementation manner, in terms of dividing the source domain data set into multiple sub-source domain data sets, the first model acquisition module 62 is specifically configured to:
[0101] Calculate the second earth mover's distance between any two data in the source domain data set;
[0102] Obtain the similarity between any two data according to the second earth mover's distance;
[0103] Based on the similarity between any two data, divide the source domain data set into multiple sub-source domain data sets.
[0104] In a possible implementation manner, in terms of training each sub-source domain data set to obtain a corresponding candidate pre-trained model, the first model acquisition module 62 is specifically configured to:
[0105] Input each sub-source domain data set into a preset neural network for training to obtain the candidate pre-trained model corresponding to each sub-source domain data set;
[0106] The second model acquisition module 64 is further specifically configured to:
[0107] If the similarity between the target domain data set and each sub-source domain data set is less than the preset value, input the target domain data set into the neural network for training to obtain the best classification model of the target domain data set.
[0108] According to an embodiment of the present application, Figure 6 Each unit in the shown deep transfer learning device can be separately or entirely combined into one or several other units to form, or a certain one (some) of the units can be further split into multiple smaller units with functional division to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In actual applications, the function of one unit can also be realized by multiple units, or the functions of multiple units are realized by one unit. In other embodiments of the present application, based on the deep transfer learning device, other units can also be included. In actual applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units. It should be noted that the deep transfer learning device provided by the embodiments of the present application can be applied to the image data processing scenario.
[0109] According to another embodiment of the present application, it can be achieved by running on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM) that can execute as Figure 2 or Figure 4a computer program (including program code) for each step involved in the corresponding method shown in the figure, to construct a deep transfer learning device as shown in Figure 6 and to implement the deep transfer learning method of the embodiments of the present application. The computer program may be recorded on, for example, a computer-readable recording medium, loaded into the above computing device through the computer-readable recording medium, and run therein.
[0110] Based on the descriptions of the above method embodiments and device embodiments, the embodiments of the present application further provide an electronic device. Please refer to Figure 7 , the electronic device at least includes a processor 71, an input device 72, an output device 73, and a computer storage medium 74. Among them, the processor 71, input device 72, output device 73, and computer storage medium 74 in the electronic device can be connected through a bus or other means.
[0111] The computer storage medium 74 can be stored in the memory of the electronic device. The computer storage medium 74 is used to store a computer program. The computer program includes program instructions. The processor 71 is used to execute the program instructions stored in the computer storage medium 74. The processor 71 (or CPU (Central Processing Unit, central processor)) is the computing core and control core of the electronic device, which is adapted to implement one or more instructions, and is specifically adapted to load and execute one or more instructions to implement the corresponding method process or corresponding function.
[0112] In one embodiment, the processor 71 of the electronic device provided by the embodiments of the present application can be used to perform a series of transfer learning processes:
[0113] Obtain source domain images belonging to multiple categories from a target address, label the source domain images, and obtain a source domain data set with category labels;
[0114] Divide the source domain data set into multiple sub-source domain data sets, and use each sub-source domain data set to train a corresponding candidate pre-trained model;
[0115] When a target domain image belonging to a target category is input, label the target domain image to obtain a target domain data set with category labels, and calculate the similarity between the target domain data set and each sub-source domain data set;
[0116] According to the similarity between the target domain data set and each sub-source domain data set, select a target pre-trained model from the candidate pre-trained models;
[0117] Fine-tune the target pre-trained model using the target domain data set to obtain the best classification model for the target domain data set.
[0118] In one embodiment, the processor 71 executes calculating the similarity between the target domain dataset and each of the sub-source domain datasets, including:
[0119] Calculating the first earth mover's distance between the target domain dataset and each of the sub-source domain datasets;
[0120] Obtaining the similarity between the target domain dataset and each of the sub-source domain datasets according to the first earth mover's distance.
[0121] In another embodiment, the processor 71 executes selecting a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain dataset and each of the sub-source domain datasets, including:
[0122] Obtaining a target sub-source domain dataset whose similarity with the target domain dataset is greater than or equal to a preset value;
[0123] Determining the candidate pre-trained model trained by using the target sub-source domain dataset as the target pre-trained model.
[0124] In another embodiment, the processor 71 executes dividing the source domain dataset into multiple sub-source domain datasets, including:
[0125] Calculating the second earth mover's distance between any two data in the source domain dataset;
[0126] Obtaining the similarity between any two data according to the second earth mover's distance;
[0127] Dividing the source domain dataset into multiple sub-source domain datasets based on the similarity between any two data.
[0128] In another embodiment, the processor 71 executes training a corresponding candidate pre-trained model by using each of the sub-source domain datasets, including:
[0129] Inputting each of the sub-source domain datasets into a preset neural network for training to obtain the candidate pre-trained model corresponding to each of the sub-source domain datasets;
[0130] The processor 71 is further specifically configured to: if the similarity between the target domain dataset and each of the sub-source domain datasets is less than the preset value, input the target domain dataset into the neural network for training to obtain the best classification model of the target domain dataset.
[0131] Exemplarily, the above-mentioned electronic device may be a device such as a computer, a server, a computer mainframe, etc. The electronic device may include but is not limited to a processor 71, an input device 72, an output device 73, and a computer storage medium 74. Those skilled in the art can understand that the schematic diagram is only an example of the electronic device, and does not constitute a limitation on the electronic device. It may include more or fewer components than those shown in the figure, or combine some components, or different components.
[0132] It should be noted that since the processor 71 of the electronic device implements the steps in the above-mentioned deep transfer learning method when executing a computer program, the embodiments of the above-mentioned deep transfer learning method are all applicable to this electronic device, and can achieve the same or similar beneficial effects.
[0133] The embodiment of the present application also provides a computer storage medium (Memory). The computer storage medium is a memory device in the electronic device, and is used to store programs and data. It can be understood that the computer storage medium here can include both the built-in storage medium in the terminal, and of course can also include the extended storage medium supported by the terminal. The computer storage medium provides a storage space, and this storage space stores the operating system of the terminal. And, one or more instructions suitable for being loaded and executed by the processor 71 are stored in this storage space. These instructions may be one or more computer programs (including program codes). It should be noted that the computer storage medium here may be a high-speed RAM memory, or a non-volatile memory, such as at least one disk memory; optionally, it may also be at least one computer storage medium located far from the aforementioned processor 71. In one embodiment, one or more instructions stored in the computer storage medium can be loaded and executed by the processor 71 to implement the corresponding steps of the above-mentioned deep transfer learning method; specifically, one or more instructions in the computer storage medium are loaded and executed by the processor 71 as follows: [[ID=!]]
[0134] Obtain source domain images belonging to multiple categories from the target address, annotate the source domain images, and obtain a source domain data set with category labels;
[0135] Divide the source domain data set into multiple sub-source domain data sets, and use each sub-source domain data set to train a corresponding candidate pre-trained model;
[0136] When a target domain image belonging to the target category is input, annotate the target domain image to obtain a target domain data set with category labels, and calculate the similarity between the target domain data set and each sub-source domain data set;
[0137] Select a target pre-trained model from the candidate pre-trained models according to the similarity between the target domain dataset and each sub-source domain dataset;
[0138] Fine-tune the target pre-trained model using the target domain dataset to obtain the best classification model for the target domain dataset.
[0139] In one example, when one or more instructions in the computer storage medium are loaded by the processor 71, the following steps are further executed:
[0140] Calculate the first Wasserstein distance between the target domain dataset and each sub-source domain dataset;
[0141] Obtain the similarity between the target domain dataset and each sub-source domain dataset according to the first Wasserstein distance.
[0142] In one example, when one or more instructions in the computer storage medium are loaded by the processor 71, the following steps are further executed:
[0143] Obtain a target sub-source domain dataset whose similarity with the target domain dataset is greater than or equal to a preset value;
[0144] Determine the candidate pre-trained model trained using the target sub-source domain dataset as the target pre-trained model.
[0145] In one example, when one or more instructions in the computer storage medium are loaded by the processor 71, the following steps are further executed:
[0146] Calculate the second Wasserstein distance between any two data in the source domain dataset;
[0147] Obtain the similarity between the any two data according to the second Wasserstein distance;
[0148] Divide the source domain dataset into multiple sub-source domain datasets based on the similarity between the any two data.
[0149] In one example, when one or more instructions in the computer storage medium are loaded by the processor 71, the following steps are further executed: Input each sub-source domain dataset into a preset neural network for training to obtain the candidate pre-trained model corresponding to each sub-source domain dataset;
[0150] In one example, when one or more instructions in the computer storage medium are loaded by the processor 71, the following steps are further executed: If the similarity between the target domain dataset and each sub-source domain dataset is less than the preset value, input the target domain dataset into the neural network for training to obtain the best classification model for the target domain dataset.
[0151] Exemplarily, the computer program of the computer storage medium includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.
[0152] It should be noted that since the computer program of the computer storage medium implements the steps in the above-mentioned deep transfer learning method when executed by a processor, all embodiments of the above-mentioned deep transfer learning method are applicable to this computer storage medium and can achieve the same or similar beneficial effects.
[0153] The above has introduced the embodiments of the present application in detail. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.
Claims
1. A deep transfer learning method, characterized in that: The method comprises: Obtain source domain images belonging to multiple categories from the target address, annotate the source domain images, and obtain a source domain dataset with category labels; Dividing the source domain dataset into multiple sub-source domain datasets, and using each sub-source domain dataset to train a corresponding candidate pre-training model; When a target domain image belonging to a target category is input, the target domain image is annotated to obtain a target domain dataset with category labels, and the similarity between the target domain dataset and each of the sub-source domain datasets is calculated; Selecting a target pre-training model from the candidate pre-training models according to the similarity between the target domain dataset and each of the sub-source domain datasets; Fine-tuning the target pre-trained model using the target domain dataset to obtain an optimal classification model for the target domain dataset; The calculating the similarity between the target domain dataset and each of the sub-source domain datasets includes: Calculating a first moving distance between the target domain dataset and each of the sub-source domain datasets; According to the first earth-moving distance, the similarity between the target domain dataset and each sub-source domain dataset is obtained using the following formula: similarity(S,T)=e -λd(S,T) Wherein, λ is a preset constant, similarity(S,T) is the similarity between the sub-source domain dataset and the target domain dataset, d(S,T) is the first moving distance, and e is a natural constant.
2. The method according to claim 1, characterized in that The selecting a target pre-training model from the candidate pre-training models according to the similarity between the target domain dataset and each of the sub-source domain datasets includes: Acquire a target sub-source domain dataset whose similarity to the target domain dataset is greater than or equal to a preset value; The candidate pre-training model trained using the target sub-source domain dataset is determined as the target pre-training model.
3. The method according to claim 1 or 2, characterized in that The dividing the source domain dataset into a plurality of sub-source domain datasets includes: Calculating the second moving distance between any two data in the source domain dataset; Obtaining the similarity between the two arbitrary data according to the second earth-moving distance; Based on the similarity between the two arbitrary data, the source domain dataset is divided into a plurality of sub-source domain datasets.
4. The method according to claim 2, characterized in that The method of using each sub-source domain dataset to train and obtain a corresponding candidate pre-training model includes: Inputting each of the sub-source domain datasets into a preset neural network for training to obtain the candidate pre-trained model corresponding to each of the sub-source domain datasets; The method further comprises: If the similarity between the target domain dataset and each of the sub-source domain datasets is less than the preset value, the target domain dataset is input into the neural network for training to obtain the optimal classification model for the target domain dataset.
5. A deep transfer learning device, characterized in that: The device comprises: A source domain data acquisition module is used to acquire source domain images belonging to multiple categories from a target address, annotate the source domain images, and obtain a source domain dataset with category labels; A first model acquisition module is configured to divide the source domain dataset into a plurality of sub-source domain datasets, and train each of the sub-source domain datasets to obtain a corresponding candidate pre-training model; a similarity calculation module, configured to, when a target domain image belonging to a target category is input, label the target domain image to obtain a target domain dataset with a category label, and calculate the similarity between the target domain dataset and each of the sub-source domain datasets; A second model acquisition module is configured to select a target pre-training model from the candidate pre-training models based on the similarity between the target domain dataset and each of the sub-source domain datasets; A model fine-tuning module, configured to fine-tune the target pre-trained model using the target domain dataset to obtain an optimal classification model for the target domain dataset; In terms of calculating the similarity between the target domain dataset and each of the sub-source domain datasets, the similarity calculation module is specifically configured to: Calculating a first moving distance between the target domain dataset and each of the sub-source domain datasets; According to the first earth-moving distance, the similarity between the target domain dataset and each sub-source domain dataset is obtained using the following formula: similarity(S,T)=e -λd(S,T) Wherein, λ is a preset constant, similarity(S,T) is the similarity between the sub-source domain dataset and the target domain dataset, d(S,T) is the first moving distance, and e is a natural constant.
6. The device according to claim 5, characterized in that In terms of selecting a target pre-trained model from the candidate pre-trained models based on the similarity between the target domain dataset and each of the sub-source domain datasets, the second model acquisition module is specifically configured to: Acquire a target sub-source domain dataset whose similarity to the target domain dataset is greater than or equal to a preset value; The candidate pre-training model trained using the target sub-source domain dataset is determined as the target pre-training model.
7. An electronic device comprising an input device and an output device, characterized in that: Also includes: a processor adapted to implement one or more instructions; as well as, A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the method according to any one of claims 1 to 4.
8. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 4.