Migration learning method, device, equipment and storage medium

Abstract

Embodiments of the present application disclose a transfer learning method and device, equipment and a storage medium, wherein the transfer learning method comprises: obtaining an original data set and a task data set; determining a target subset of the original data set divided on a domain related to the task data set; obtaining a first network model trained by using the target subset; wherein the first network model is built based on a pre-training model and a first head network; the pre-training model is trained by using the original data set; training a built second network model by using the task data set to obtain a trained second network model; wherein the second network model is built based on the first network model and a second head network.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of computer vision technology, and in particular to a transfer learning method, apparatus, device, and storage medium. Background Technology

[0002] Numerous experiments have demonstrated that models pre-trained on large-scale datasets exhibit excellent generalization capabilities. Researchers have attempted to transfer these powerful pre-trained models to downstream tasks, thereby improving downstream task metrics. However, in certain business scenarios, such as when downstream data volume is limited, directly fine-tuning the pre-trained model can lead to poor performance on certain types of downstream tasks. Alternatively, in cloud services, due to resource constraints, it is often necessary to freeze the backbone layers of the pre-trained model and only fine-tune the head layer based on a specific downstream task to reduce memory and computational costs. This, too, can result in a decline in the final model's performance. Summary of the Invention

[0003] In view of this, embodiments of this application provide at least one transfer learning method, apparatus, device, and storage medium.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] On one hand, embodiments of this application provide a transfer learning method, the method comprising:

[0006] Obtain the original dataset and the task dataset; determine the target subset of the original dataset in the domain covered by the task dataset; obtain a first network model trained using the target subset; wherein the first network model is built based on a pre-trained model and a first head network; the pre-trained model is trained using the original dataset; use the task dataset to train a second network model to obtain a trained second network model; wherein the second network model is built based on the first network model and the second head network.

[0007] In the above embodiments, the original dataset of the source domain and the task dataset of the target domain are first obtained. Then, it is determined which domain the task dataset belongs to within the original dataset, and a target subset of the original dataset is determined within that corresponding domain. Next, a first network model pre-trained using the target subset is obtained. Finally, a second network model is built based on the first network model and the second head network, and transfer learning is performed on the task dataset. This allows for the reuse of pre-trained model parameters, achieving parameter-based transfer learning. Furthermore, after traditional pre-training, domain-based parameters for each subset of the source domain are learned based on the first head network of the first network model. These domain-based parameters can achieve a feature selection effect during downstream transfer, thereby mitigating the difficulty of direct model transfer caused by significant differences between upstream and downstream dataset domains.

[0008] In some possible embodiments, determining the target subset of the original dataset in the domain involved in the task dataset includes: clustering the original dataset to obtain N subsets corresponding to the domains; N is an integer greater than or equal to 2; and determining the target subset from the N subsets that belongs to the same domain as the task dataset.

[0009] In the above embodiments, the original dataset is first refined by clustering and divided into subsets belonging to each domain. Then, the task dataset is determined to be a subset of data from a certain domain in the original dataset. Thus, during the transfer process, the trained first network model corresponding to that domain can be used for feature selection, reducing the problem of decreased transfer performance caused by large domain differences between the original dataset and the task dataset.

[0010] In some possible embodiments, determining the target subset from the N subsets that belongs to the same domain as the task dataset includes: determining the domain to which each frame of the task dataset belongs from the N domains corresponding to the N subsets; determining the target domain corresponding to the task dataset according to the proportion of the domain to which each frame of the task dataset belongs in the N domains; and taking the subset of the N subsets corresponding to the target domain as the target subset.

[0011] In the above embodiments, firstly, it is determined that each frame of image in the task dataset belongs to a specific domain among N domains. Then, the domain with the highest percentage of all images in the task dataset is identified as the target domain. This allows for the determination of the subset of images belonging to the same domain as the task dataset. In this way, it is possible to accurately determine whether the domain of the task dataset corresponds to the target domain and the corresponding target subset in the original dataset.

[0012] In some possible embodiments, determining the domain to which each frame of an image in the task dataset belongs from the N domains corresponding to the N subsets includes: determining a first distance between each frame of an image in the task dataset and the class centers of the N subsets; wherein the value of the class center is the average value of all data in the subsets in each dimension; and determining that each frame of an image belongs to the domain corresponding to the target class center if there is a target class center whose first distance satisfies a preset threshold.

[0013] In the above embodiments, by calculating the first distance between each frame image in the task dataset and the class centers of N sub-datasets, it is determined which domain each frame image belongs to, thereby facilitating the subsequent determination that the entire task dataset corresponds to a certain domain in the original dataset, so as to accurately select the first network model trained in that domain during transfer.

[0014] In some possible embodiments, the first network model is trained by the following steps: pre-training the backbone network using the original dataset to obtain the pre-trained model; adding the first head network to the network structure of the pre-trained model for each domain-specific sub-dataset to obtain the first network model for the corresponding domain; and training the first head network using the sub-dataset for each domain to obtain the trained first network model.

[0015] In the above embodiments, after training the pre-trained model using the original dataset, a first head network is added, and the first head network is trained using the corresponding subset of each domain, thereby enabling domain-based parameter learning of the subset of each domain in the original dataset. In this way, the domain-based parameters in the trained first network model can play a role in feature selection during downstream transfer.

[0016] In some possible embodiments, training the first head network using the subset of data points corresponding to each domain to obtain the first network model corresponding to the respective domain includes: processing the subset of data points using the first network model and outputting the prediction results of the subset of data points; determining a first loss of the subset of data points through the first network model based on the prediction results of the subset of data points and the labels carried by the subset of data points; and updating the parameters of the first head network in the first network model based on the first loss, while keeping the parameters of the backbone network in the first network model fixed, until the first loss reaches the convergence condition, thereby obtaining the trained first network model.

[0017] In the above embodiments, with the parameters of the backbone network in the first network model fixed, the first head network in the first network model is trained by backpropagation using a subset of datasets corresponding to each domain. When the first loss reaches the convergence condition, the domain-based parameters of the first head network reach their optimal values, resulting in a trained first network model. Thus, domain-based parameters are learned through subsets of datasets corresponding to each domain, aiming to achieve feature selection during subsequent transfer learning using these learned domain-based parameters.

[0018] In some possible embodiments, training a second network model using the task dataset to obtain a trained second network model includes: processing the task dataset using the second network model and outputting a prediction result for the task dataset; determining a second loss for the task dataset through the second network model based on the prediction result and the labels carried by the task dataset; and updating the parameters of the second head network in the second network model based on the second loss, while keeping the parameters of the backbone network and the first head network in the second network model fixed, until the second loss reaches the convergence condition.

[0019] In the above embodiments, with the parameters of the backbone network and the first head network in the second network model fixed, the second head network in the second network model is trained by backpropagation using a task dataset. When the second loss reaches the convergence condition, the task-based parameters of the second head network reach their optimal values, resulting in a trained second network model. This enables rapid learning of the parameters for a specific pre-task during downstream transfer, resulting in a fully trained second network model.

[0020] In some possible embodiments, the second head network includes a fully connected layer, and the method further includes: adding the fully connected layer after the output layer of the first network model based on the number of categories of the data in the task dataset to obtain the constructed second network model; the number of neurons in the fully connected layer corresponds to the number of categories of the data in the task dataset.

[0021] In the above embodiments, by adding a fully connected layer that matches the number of data categories in the task dataset after the output layer of the first network model, the task-based parameters can be effectively learned and the overall transfer efficiency can be improved.

[0022] In some possible embodiments, the task dataset is a set of face samples, and the method further includes: acquiring face images of the target object; acquiring a second network model trained using the face dataset; and performing face recognition on the face images using the second network model to obtain the face recognition result of the target object.

[0023] In the above embodiments, since the second network model is trained using a face sample set, i.e., a large amount of face sample data, it reduces the problems of reduced object retrieval accuracy caused by the limitations of object features themselves and the high similarity rate of actual appearance features of different objects, and can be applied to their respective face recognition scenarios.

[0024] Furthermore, embodiments of this application provide a transfer learning apparatus, the apparatus comprising:

[0025] The first acquisition module is used to acquire the original dataset and the task dataset;

[0026] The determination module is used to determine the target subset of the original dataset on the domain involved in the task dataset;

[0027] The second acquisition module is used to acquire a first network model trained using the target subset; wherein the first network model is built based on a pre-trained model and a first head network; the pre-trained model is trained using the original dataset;

[0028] The transfer learning module is used to train the constructed second network model using the task dataset to obtain the trained second network model; wherein the second network model is constructed based on the first network model and the second head network.

[0029] In another aspect, embodiments of this application provide a computer device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0030] In another aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0031] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0033] Figure 1 A schematic diagram of an optional flow of the transfer learning method provided in an embodiment of this application;

[0034] Figure 2 A schematic diagram of an optional flow of the transfer learning method provided in an embodiment of this application;

[0035] Figure 3 A schematic diagram of an optional flow of the transfer learning method provided in an embodiment of this application;

[0036] Figure 4 A schematic diagram of an optional flow of the transfer learning method provided in an embodiment of this application;

[0037] Figure 5 A schematic diagram of a multi-adaptive head-based transfer learning algorithm framework provided in an embodiment of this application;

[0038] Figure 6 A logical flowchart of a transfer learning method provided in an embodiment of this application;

[0039] Figure 7 This is a schematic diagram of the composition structure of a transfer learning device provided in an embodiment of this application;

[0040] Figure 8 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] The terms “first / second / third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0045] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0046] Transfer learning is a popular method in computer vision because it can build accurate models in less time. It involves starting with patterns learned from solving various problems previously, omitting the process of learning from scratch. The parameters of a pre-trained model are transferred to a new model to aid in its training. Since most data or tasks are correlated, transfer learning allows learned model parameters to be shared with the new model in some way, thereby accelerating and optimizing the model's learning efficiency.

[0047] In the field of computer vision, transfer learning is typically represented using pre-trained models. These pre-trained models are usually trained on large benchmark datasets to solve similar problems. Transfer learning can be applied to computer vision, image classification, text classification, action recognition, natural language processing, video surveillance, public opinion analysis, human-computer interaction, and more.

[0048] Transfer learning can be categorized into four main types based on its learning methods: sample-based transfer learning, feature-based transfer learning, model-based transfer learning, and relation-based transfer learning. Model-based transfer learning involves building models with shared parameters. Given that the structure of neural networks can be directly transferred, this technique is widely used in neural networks. For example, the classic fine-tuning of neural networks is a good example of model parameter transfer.

[0049] Model-based transfer learning, also known as parametric knowledge-based transfer learning, refers to methods that find shared parameter information between the source and target domains to achieve transfer learning. It allows the reuse of pre-trained model parameters, thereby accelerating model training and saving training costs. This transfer method requires the assumption that the data in the source and target domains can share some model parameters.

[0050] The domain is the core of learning, primarily composed of two parts: data and the probability distribution that generated that data. It can be represented as D = {χ, P(X)}, where χ represents the feature space and P(X) represents the marginal probability distribution. A domain can be understood as a specific area at a given moment; for example, book reviews and TV series reviews can be considered two different domains. Transfer learning includes two types: the source domain (the domain of existing knowledge) and the target domain (the domain to be learned).

[0051] Task: The result of learning, including the objective function and the learning result, can be represented as T={y,f(·)}, where y represents the label space and f(·) represents the objective prediction function.

[0052] Traditional computer vision-based transfer learning methods mainly include adapter-based methods, fine-tuning, and linear probe methods. However, these methods all aim to learn parameters based on a specific task, and when there is a large domain gap between the source and target domains, model transfer often becomes extremely difficult.

[0053] This application provides a transfer learning method, which can be executed by a processor of a computer device. The computer device refers to a device with transfer learning capabilities, such as a server, laptop, tablet, desktop computer, smart TV, set-top box, or mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device).

[0054] Figure 1 This is a schematic diagram illustrating the implementation process of a transfer learning method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps S101 to S104:

[0055] Step S101: Obtain the original dataset and the task dataset.

[0056] The original dataset is a large-scale benchmark dataset, also known as an upstream dataset. Pre-trained models are typically trained on large benchmark datasets to solve similar problems. The original dataset can encompass multiple domains, such as natural scenes, video analysis scenarios, and data acquisition scenarios.

[0057] The task dataset is the dataset that the migration is directed to. It is the business scenario data in the process of executing a specific task downstream, that is, images in a specific domain of a specific task, such as street pedestrian images in a detection task or real-time face images in a data acquisition task.

[0058] Step S102: Determine the target subset of the original dataset on the domain involved in the task dataset.

[0059] First, the original dataset is divided into multiple highly correlated, non-overlapping subsets by dividing it into domains. Each subset contains similar data and is called a domain. Then, the target subset corresponding to the domain involved in the task dataset is determined from the multiple subsets.

[0060] In implementation, clustering is first used to determine which domain the task dataset belongs to. Then, from the multiple subsets of the original dataset, subsets belonging to the same domain as the task dataset are selected as the target subset. For example, the original dataset is divided into subset N1 in domain A, subset N2 in domain B, and subset N3 in domain C. Assuming the task dataset belongs to domain C, the target subset is determined to be subset N3.

[0061] Step S103: Obtain the first network model trained using the target subset.

[0062] The first network model is built based on a pre-trained model and a first head network; the pre-trained model is trained using the original dataset.

[0063] The first head network is a domain-based network structure, which typically includes a multilayer perceptron (MLP) to learn the parameters of each domain.

[0064] In implementation, a pre-trained model is first trained on any network structure using the original dataset, or a pre-trained model already trained on the original dataset is directly obtained. Then, a domain-based first head network is added to the network structure of the pre-trained model to obtain the constructed first network model. This first head network is then trained using a subset of the original dataset divided into the corresponding domain. Finally, during downstream transfer learning, after determining the target subset corresponding to the domain of the task dataset, the first network model trained on this target subset is directly obtained for subsequent transfer learning.

[0065] It's important to note that a multilayer perceptron (MLP) is a feedforward artificial neural network model that maps multiple input datasets to a single output dataset. An MLP is an Artificial Neural Network (ANN), and the layers of an MLP are fully connected. Typically, an MLP has an input layer at the bottom, hidden layers in the middle, and an output layer at the end. Besides the input and output layers, an MLP can have multiple hidden layers; the simplest MLP has only one hidden layer, i.e., a three-layer structure.

[0066] In this embodiment, the input layer X of the first head network is actually the feature extracted from a subset of data in a certain domain of the original dataset. "From input layer to hidden layer" is a fully connected layer, and "from hidden layer to output layer" is a classifier (softmax) regression. Thus, by training the first head network with the subset of data in the corresponding domain, it can learn the parameters of different domains.

[0067] Step S104: Using the task dataset, train the constructed second network model to obtain the trained second network model.

[0068] The second network model is built upon the first network model and the second head network. For example, after the output layer of the first network model, a fully connected layer matching the number of categories contained in the task dataset is added as the second head network. During training, the weights of the pre-trained layers in the other layers of the first network model remain unchanged; only the weights of the newly added layers are randomly initialized, thus using the new task dataset to train the second head network in the second network model.

[0069] In implementation, the model parameters of the first network model are first obtained, including the pre-trained model parameters and the domain-based parameters in the first head network. Then, the model parameters of the first network model are applied to the initialization of the second network model based on transfer learning. Finally, the task dataset is input and the parameters of the second head network are adjusted in reverse to perform task-specific transfer learning.

[0070] In this embodiment, the original dataset of the source domain and the task dataset of the target domain are first obtained. Then, it is determined that the domain of the task dataset corresponds to a certain domain and a corresponding target subset of the original dataset. Next, a first network model pre-trained using the target subset is obtained. Finally, a second network model is built based on the first network model and the second head network to perform transfer learning on the task dataset. This allows for the reuse of pre-trained model parameters, achieving parameter-based transfer learning. Furthermore, after traditional pre-training, domain-based parameters of each subset of the source domain are learned based on the first head network of the first network model. These domain-based parameters can achieve a feature selection effect during downstream transfer, thereby mitigating the difficulty of direct model transfer caused by significant differences between upstream and downstream dataset domains.

[0071] In some embodiments, step S104 can be implemented by steps 1041 to 1043:

[0072] Step 1041: Process the task dataset using the second network model and output the prediction results for the task dataset.

[0073] Since the second network model is formed by adding a new task-based second head network after the output layer of the first network model, the weights of each parameter in the already trained first network model can be directly loaded to initialize the second network model, and only the second head network in the second network model can be trained using the task dataset.

[0074] In implementation, the task dataset is input into the initialized second network model, where feature extraction, pooling, and normalization are performed sequentially. The final second head network outputs the prediction result for each data point in the task dataset. For example, if the task dataset consists of real-time captured images of objects to be identified, the second network model detects the category and location of each object. The objects can be pedestrians, faces, moving objects, etc., and this embodiment does not limit the specific objects identified.

[0075] Step 1042: Based on the prediction results of the task dataset and the labels carried by the task dataset, determine the second loss of the task dataset through the second network model.

[0076] The task dataset is labeled with instance-level tags before training and used as the training sample set, where each sample image contains several instances y. i = {(b, c)}, where each instance includes the candidate box position b = (b cx ,b cy ,b w ,b h The one-hot category label c∈{0,1} is used for both the single-hot and other categories. |C| , |C| It is the total number of all categories.

[0077] For example, if an image contains different animals such as cats and dogs, the instance-level annotation results in candidate bounding box 1 for the location of the cat and the category "cat", and candidate bounding box 2 for the location of the dog and the category "dog". If an image contains multiple cats, each cat is an instance, and the instance-level annotation results in candidate bounding box for the location of each cat and the category "cat".

[0078] For different specific tasks, the second loss can be a classification loss such as cross-entropy loss, focus loss, or mean squared error loss, or a regression loss such as point regression loss or bounding box regression loss.

[0079] Step 1043: Based on the second loss, while keeping the parameters of the backbone network and the first head network in the second network model fixed, update the parameters of the second head network in the second network model until the second loss reaches the convergence condition.

[0080] It should be noted that, on the one hand, considering the small number of images in the task dataset, the primary consideration is to prevent overfitting, so the weights of the network layers in the first network model are kept unchanged. On the other hand, since the task dataset and the target subset are similar, it can be assumed that the high-level features (weights) learned by the first network model on the target subset are also applicable to the task dataset.

[0081] Since the features extracted at low levels of the network are such as edges, lines, and corners in an image, which are not closely related to the target task dataset, while the features extracted at high levels are closely related to the target task, the first and second network models can share low-level parameters, requiring only retraining of the high-level weights. Therefore, during model transfer, only the second head network in the second network model needs to learn task-based parameters. The second loss is then used to backpropagate and train the second head network. When the prediction results are close to the labels of the task dataset, the second loss converges, resulting in the trained second network model.

[0082] In the above embodiments, with the parameters of the backbone network and the first head network in the second network model fixed, the second head network in the second network model is trained by backpropagation using a task dataset. When the second loss reaches the convergence condition, the task-based parameters in the second head network reach their optimal values, resulting in a trained second network model. This enables rapid learning of task-specific parameters during downstream transfer, leading to a fully trained second network model.

[0083] In some embodiments, step S104 may be followed by steps S105 to S106:

[0084] Step S105: Obtain the face image of the target object.

[0085] The target audience can be understood as pedestrians, tourists, visitors, or users in shopping malls or scenic spots who need to identify their identities, or as electronic users who need to verify their identities when logging into websites or forums.

[0086] In some implementations, the face image can be an image captured in real time by an image acquisition device installed on the electronic device, such as a camera module; in some implementations, the face image can be an image transmitted to the electronic device by other devices via instant messaging for face recognition; in some implementations, the face image can also be an image to be processed obtained by the electronic device from its local photo album in response to a task processing instruction; in some implementations, the face image can be an image frame from a video stream. This application does not impose any limitations on these embodiments.

[0087] Step S106: Perform face recognition on the face image using the second network model to obtain the face recognition result of the target object.

[0088] The second network model is obtained by transfer training based on the transfer learning method provided in the embodiments of this application. It should be noted that the second network model, by training on a large amount of face sample data, can achieve the characteristic that the feature vectors of similar images are close in space, while the feature vectors of dissimilar images are far apart in space. Thus, feature comparison can be used to quickly identify which object a face belongs to, or to find the target object in the personnel database.

[0089] The facial recognition result may include the target object's identity identifier, facial features, etc., and may also include labeling information for processing by the backend service platform, such as pass, fail, and pending review, where pending review means that manual review or secondary verification is required.

[0090] In the above embodiments, since the second network model is trained using a face sample set, i.e., a large amount of face sample data, it reduces the problems of reduced object retrieval accuracy caused by the limitations of object features themselves and the high similarity rate of actual appearance features of different objects, and can be applied to their respective face recognition scenarios.

[0091] In some embodiments, the second header network includes a fully connected layer. For example... Figure 2 As shown, the method includes the following steps S201 to S204:

[0092] Step S201: Obtain the original dataset and the task dataset.

[0093] Step S202: Cluster the original dataset to obtain N sub-datasets corresponding to the domains.

[0094] Here, N is an integer greater than or equal to 2; the number of domains N can be set empirically and adjusted according to the effect of model transfer.

[0095] In practice, the original dataset is refined through clustering. The clustering results can be used to indicate the category to which each image data belongs in the original dataset, or the probability or likelihood of each image data belonging to a certain category. Thus, the original dataset is divided into subsets belonging to various domains based on the clustering results.

[0096] There are many ways to divide the original dataset for clustering, such as K-means clustering or mean-shift clustering. This application does not limit the specific methods used. Some clustering algorithms, such as K-means, require pre-setting the number of clusters before clustering, i.e., pre-setting how many clusters the original dataset needs to be divided into, with each subset of the dataset corresponding to one cluster.

[0097] Step S203: Determine the target subset from the N subset datasets that belongs to the same domain as the task dataset.

[0098] Here, by calculating the similarity between the task dataset and N subsets, the subset most similar to the task dataset can be identified. This most similar subset is then used as the target subset belonging to the same domain as the task dataset. In other words, the task dataset is determined as a subset of data from a specific domain in the original dataset. Therefore, during transfer learning, a first network model trained using the target subset belonging to the same domain can be obtained, allowing direct reuse of pre-trained model parameters and domain-based parameters.

[0099] Step S204: Obtain the first network model trained using the target subset.

[0100] Here, the first network model is built based on a pre-trained model and a first head network; the pre-trained model is trained using the original dataset.

[0101] Since the domain-based parameters in the first network model trained using the target subset have already been learned, the difficulty of direct model transfer caused by large differences in domains between upstream and downstream datasets is mitigated.

[0102] Step S205: After the output layer of the first network model, add the fully connected layer based on the task dataset to obtain the constructed second network model.

[0103] The number of neurons in the fully connected layer corresponds to the number of categories in the task dataset. In implementation, the number of neurons in the fully connected layer is first determined based on the number of categories in the task dataset, and the weights of each neuron are randomly initialized and added after the output layer of the first network model to obtain the constructed second network model.

[0104] A fully connected layer combines all local features into global features, which are then used to calculate the final score for each class. The convolutions and pooling before the fully connected layer are equivalent to feature extraction, while the subsequent fully connected layer performs feature weighting. In this embodiment, the fully connected layer acts as a "classifier" in the entire first network model, used to learn task-based parameters.

[0105] In this way, by adding a fully connected layer after the output layer of the first network model to match the number of data categories in the task dataset, it is possible to effectively learn task-based parameters and improve overall transfer efficiency.

[0106] Step S206: Using the task dataset, train the constructed second network model to obtain the trained second network model.

[0107] Here, the second network model is built based on the first network model and the second head network.

[0108] Step S206 above corresponds to step S104 in the previous embodiment. When implementing it, you can refer to the specific implementation of step S104 above.

[0109] In this embodiment, the original dataset is first refined by clustering and divided into subsets belonging to various domains. Then, the task dataset is determined to be a subset of data from a certain domain in the original dataset. Thus, during the transfer process, the trained first network model corresponding to that domain can be used for feature selection, alleviating the problem of decreased transfer performance caused by the large domain differences between the original dataset and the task dataset.

[0110] based on Figure 2 , Figure 3 The following is a schematic diagram of an optional flow of the transfer learning method provided in the embodiments of this application, such as... Figure 3 As shown, step S203, "determining the target subset from the N subset datasets that belongs to the same domain as the task dataset," can be achieved through the following steps S301 to S303:

[0111] Step S301: Determine the domain to which each frame of the image in the task dataset belongs from the N domains corresponding to the N subset datasets.

[0112] The domain to which each frame belongs is determined by calculating the first distance between each frame of the image in the task dataset and the class centers of each of the N domains.

[0113] In some implementations, a first distance is determined between each frame image in the task dataset and the class centers of the N subsets; wherein the value of the class center is the average value of all data in the subsets in each dimension; if there is a target class center whose first distance satisfies a preset threshold, it is determined that each frame image belongs to the domain corresponding to the target class center.

[0114] Alternatively, in some implementations, a first distance is determined between each frame image in the task dataset and the class centers of the N sub-datasets; the domain corresponding to the class center with the smallest first distance is taken as the domain to which the corresponding frame image belongs.

[0115] In this way, by calculating the first distance between each frame of the task dataset and the class centers of the N sub-datasets, we can determine which domain each frame of the image belongs to. This makes it easier to further determine which domain the entire task dataset corresponds to in the original dataset, so as to accurately select the first network model trained in that domain during transfer.

[0116] Step S302: Determine the target domain corresponding to the task dataset according to the proportion of the domain to which each frame image in the task dataset belongs in the N domains;

[0117] For example, the original dataset is divided into N domains {D1, D2, D3, ..., D...} N The number of images falling into domain D1 in the task dataset is S1, the number of images falling into domain D2 is S2, and the number of images falling into domain D3 is S3. If the value S2 is greater than S1 and greater than S3, the target domain corresponding to the task dataset is determined to be S2.

[0118] Step S303: Select the subset of the N subsets corresponding to the target domain as the target subset.

[0119] In this embodiment, it is first determined that each frame of the task dataset belongs to a specific domain among N domains. Then, the domain with the highest percentage of all images in the task dataset is identified as the target domain. This allows for the determination of the subset of data belonging to the same domain as the task dataset. In this way, it is possible to accurately determine whether the domain of the task dataset corresponds to the target domain and its corresponding subset in the original dataset.

[0120] Figure 4 The following is a schematic diagram of an optional flow of the transfer learning method provided in the embodiments of this application, such as... Figure 4 As shown, the method includes at least the following steps S401 to S403:

[0121] Step S401: Use the original dataset to pre-train the backbone network to obtain the pre-trained model.

[0122] The final pre-trained model can extract general features with strong generalization. That is, for any image data, the backbone network can output general image features. Thus, the pre-trained model can be used to solve similar problems. Therefore, the pre-trained model is used as the base model for model transfer.

[0123] Step S402: For each subset of data points corresponding to a domain, add the first head network to the network structure of the pre-trained model to obtain the first network model for the corresponding domain.

[0124] The first head network is a domain-based network structure, typically composed of a multilayer perceptron, used to learn the parameters of each domain. The structure of a multilayer perceptron includes batch normalization, fully connected layers, ReLU activation functions, etc.

[0125] Assuming the input layer of the first head network, i.e. the multilayer perceptron, is represented by vector X, the output of the hidden layer connected to the input layer is f(W1*X+b1), where W1 is the weight, also known as the connection coefficient, b1 is the bias, and the function f can be a commonly used activation function such as the sigmoid function or the tanh function.

[0126] Therefore, each neuron in each layer of the first-level head network has a weight, a bias, and an activation function for each input. The task of transfer learning is to find the values ​​of these parameters, such as weights and biases, so that the output meets the requirements of the specific downstream task.

[0127] Step S403: Train the first head network using the subset of data corresponding to each domain to obtain the trained first network model.

[0128] Here, step S403 is performed on each of the N domains corresponding to the subset of the original dataset to obtain N domain-based first network models.

[0129] In this embodiment of the application, after training a pre-trained model using the original dataset, a first head network is added, and the first head network is trained using the subset of datasets corresponding to each domain, thereby enabling domain-based parameter learning of the subsets of each domain in the original dataset. In this way, the domain-based parameters in the trained first network model can play a role in feature selection during downstream transfer.

[0130] In some embodiments, step S403 may include steps 4031 to 4033:

[0131] Step 4031: Process the subset of data using the first network model and output the prediction results for the subset of data.

[0132] First, the backbone network in the first network model extracts features from the subset of data. Then, the data features corresponding to the subset of data are input into the first head network. Within the first head network, a forward process involves: the data features are processed by the first layer of neurons (multiplied by weights, added with biases, and activated once to obtain the output); the output of the first layer is then used as the input to the second layer, processed by the second layer of neurons to obtain the second layer's output, and so on until the output layer is processed, finally yielding the prediction result.

[0133] Step 4032: Based on the prediction results of the subset and the labels carried by the subset, determine the first loss of the subset through the first network model.

[0134] Before training, the subset of data is labeled with instance-level tags and used as the training sample set. Each sample image contains several instances, and each instance includes the candidate bounding box location and a one-hot class label. For different specific tasks, the first loss can be classification loss, regression loss, etc.

[0135] Step 4033: Based on the first loss, while keeping the parameters of the backbone network in the first network model fixed, update the parameters of the first head network in the first network model until the first loss reaches the convergence condition, and obtain the trained first network model.

[0136] For each subset of data points corresponding to a domain, during transfer learning, it is only necessary to freeze the relevant layers of the pre-trained part of the first network model and modify the parameters of the first head network to make the prediction results closer to the true labels.

[0137] In the above embodiments, with the parameters of the backbone network in the first network model fixed, the first head network in the first network model is trained by backpropagation using a subset of datasets corresponding to each domain. When the first loss reaches the convergence condition, the domain-based parameters of the first head network reach their optimal values, resulting in a trained first network model. Thus, domain-based parameters are learned through subsets of datasets corresponding to each domain, aiming to achieve feature selection during subsequent transfer learning using these learned domain-based parameters.

[0138] The above transfer learning method will be described below with reference to a specific embodiment. However, it is worth noting that this specific embodiment is only for better illustration of this application and does not constitute an improper limitation of this application.

[0139] Typically, during downstream transfer, related technologies only learn task-based parameters. However, due to significant differences between upstream and downstream data, it is difficult to learn task-specific parameters. Furthermore, while large-scale pre-trained models can extract generalized features with strong generalization capabilities, sometimes the downstream task data volume is too small or its distribution differs from the upstream data distribution. This makes it difficult to obtain suitable feature mappings for downstream tasks by simply fine-tuning the model weights, thus hindering the transfer of the model to downstream tasks.

[0140] This application provides a novel transfer learning method, such as... Figure 5 As shown in the embodiments of this application, a transfer learning algorithm framework based on multiple adaptive heads is proposed. The overall algorithm framework includes three stages: pre-training, domain parameter learning, and task dataset fine-tuning. Domain-based parameter learning is introduced between the traditional upstream pre-training and downstream transfer stages, thereby making the process of the pre-trained model transferring from the original domain to the target domain smoother and preventing the model transfer difficulty caused by large domain differences.

[0141] The first stage is the pre-training part, which uses the original dataset 51 to train the backbone network 52 to obtain the pre-trained model.

[0142] The second stage is the domain parameter learning part. The original dataset 51 is divided into domains to obtain a subset for each domain. For each subset, a first head network is added after the backbone network of the pre-trained model obtained in the first stage to obtain the first network model 501 for each domain. The first head network for the corresponding domain is then trained using the subset of the dataset. This stage aims to reduce the problem of decreased transfer performance caused by large domain differences between the upstream original dataset and the downstream task dataset.

[0143] This embodiment of the application takes three domains as an example, refining the original dataset into sub-datasets 511, 512, and 513, which correspond to the first head network 531, 532, and 533 respectively. It should be noted that training is performed based on fixed parameters of the backbone network 52; that is, the backbone network 52 is not trainable, and only the first head network is trainable. Furthermore, for each specific sub-dataset, only one of the first head network 531, 532, and 533 is selected for training at any given time.

[0144] The third stage is the transfer learning part based on the task dataset. Taking task dataset 54 as an example, task dataset 54 is determined to be a dataset of a certain domain defined in the first stage. A second head network 55 is added to the first network model 501 based on this domain to obtain the second network model 502. The parameters of the second head network 55 in the second network model 502 are fine-tuned using task dataset 54 to complete the transfer learning.

[0145] This embodiment assumes that the target subset of the original dataset divided in the domain involved in the task dataset 54 is subset 512. Therefore, a second head network 55 is added to the first network model trained using subset 512 obtained in the previous stage, that is, a second head network 55 is added after the first head network 532. It should be noted that training is performed with the parameters of the backbone network 52 and the first head network 532 fixed, that is, the backbone network 52 and the first head network 532 are not trainable, only the second head network 55 is trainable.

[0146] Figure 6 The logical flowchart of the transfer learning method provided in the embodiments of this application is as follows: Figure 6 As shown, the method includes the following steps:

[0147] Step S601: Obtain the original dataset and the task dataset.

[0148] The original dataset is a large-scale upstream dataset that can contain natural scenes, monitoring scenes, and data collection scenes from multiple domains. The task dataset is the dataset to which the data is ultimately transferred; it generally refers to images from a specific domain within a specific task, such as street pedestrian images in a task of detecting pedestrians.

[0149] Step S602: Train the pre-trained model based on the original dataset.

[0150] A pre-trained model is obtained by training with large-scale upstream data in any network structure; for any frame of image data, the backbone network part of the pre-trained model can output a general representation.

[0151] Step S603: Divide the original dataset into domains to obtain subsets belonging to each domain.

[0152] The large-scale original dataset from upstream is refined into multiple smaller datasets that are highly correlated and have no overlap. In implementation, existing clustering methods are used to divide the large-scale original dataset into N subsets. Each subset is referred to as a dataset within the same domain because the data within it is relatively similar.

[0153] The execution order of the above steps S602 and S603 is not fixed and they can be performed simultaneously. This application embodiment does not limit this.

[0154] Step S604: Use the subset dataset to train the domain parameters of the pre-trained model to obtain the first network model based on the domain.

[0155] Here, for each domain's subset of data, a new domain-based network structure, namely the first head network, is added after the output layer of the pre-trained model obtained in step S602. This first head network is then trained using this subset of data. This operation is performed on each domain's subset of data in step S603, thus obtaining N domain-specific first network models, where N is the number of domains.

[0156] This application's embodiments, based on traditional transfer learning, add a domain-based parameter learning component. Specifically, after the traditional pre-training stage, a new domain-based network structure is added after the backbone network of the pre-trained model, and these parameters are learned using the corresponding domain's dataset. The aim is to achieve feature selection by utilizing the already learned domain parameters when learning task-based parameters in downstream transfer learning, thus mitigating the difficulty in learning task-based parameters due to significant differences between upstream and downstream data.

[0157] Step S605: Fine-tune the first network model corresponding to the domain of the task dataset to obtain the second network model that has been trained.

[0158] Here, firstly, for the task dataset, the clustering process in step S603 determines the task dataset to belong to a certain domain. Since each domain can be regarded as a class center, the distance between each frame of the task dataset and each domain (class center) is calculated to determine which domain each frame belongs to. The domain with the highest proportion in each frame is taken as the domain of the entire task dataset.

[0159] Then, obtain the first network model based on the domain of the task dataset obtained in the previous step S604, add a new network structure based on the specific task, namely the second head network, after the output layer of the first network model, and train the second head network using the task dataset to complete the transfer.

[0160] This application's embodiments divide the upstream original dataset into subsets belonging to multiple domains and add a domain-based network structure after the pre-trained model to learn domain-based parameters, thereby achieving feature selection during downstream transfer processing. This can improve downstream task metrics when the task data volume is small and the task dataset is only similar to a portion of the original dataset.

[0161] In this embodiment, after training a pre-trained model based on the original dataset, the model learns domain-based parameters using subsets of each domain. These domain-based parameters can then play a feature selection role during downstream transfer, thereby mitigating the difficulty of direct model transfer caused by significant differences between upstream and downstream datasets. This also addresses the problem of poor model transfer performance caused by fixed backbone network parameters in cloud service scenarios.

[0162] Based on the foregoing embodiments, this application provides a transfer learning device, which includes various modules and sub-modules and units included in each module, and can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0163] Figure 7 This is a schematic diagram of the composition structure of a transfer learning device provided in an embodiment of this application, as shown below. Figure 7 As shown, the transfer learning device 700 includes: a first acquisition module 701, a determination module 702, a second acquisition module 703, and a transfer training module 704, wherein:

[0164] The first acquisition module 701 is used to acquire the original dataset and the task dataset;

[0165] The determining module 702 is used to determine the target subset of the original dataset on the domain involved in the task dataset;

[0166] The second acquisition module 703 is used to acquire a first network model trained using the target subset; wherein the first network model is built based on a pre-trained model and a first head network; the pre-trained model is trained using the original dataset;

[0167] The transfer training module 704 is used to train the constructed second network model using the task dataset to obtain the trained second network model; wherein the second network model is constructed based on the first network model and the second head network.

[0168] In some possible embodiments, the determining module 702 includes a clustering submodule and a first determining submodule, wherein: the clustering submodule is used to cluster the original dataset to obtain N sub-datasets corresponding to domains; N is an integer greater than or equal to 2; the first determining submodule is used to determine the target subset from the N sub-datasets that belongs to the same domain as the task dataset.

[0169] In some possible embodiments, the first determining submodule includes a first determining unit, a second determining unit, and a third determining unit, wherein: the first determining unit is configured to determine the domain to which each frame image in the task dataset belongs from the N domains corresponding to the N subset datasets; the second determining unit is configured to determine the target domain corresponding to the task dataset according to the proportion of the domain to which each frame image in the task dataset belongs in the N domains; and the third determining unit is configured to take the subset dataset corresponding to the target domain from the N subset datasets as the target subset.

[0170] In some possible embodiments, the first determining unit includes a first determining subunit and a second determining subunit, wherein: the first determining subunit is used to determine a first distance between each frame image in the task dataset and the class centers of the N sub-datasets; wherein the value of the class center is the average value of all data in the sub-datasets in each dimension; the second determining subunit is used to determine that each frame image belongs to the domain corresponding to the target class center if there is a target class center whose first distance satisfies a preset threshold.

[0171] In some possible embodiments, the device 600 further includes a pre-training module, a first construction module, and a domain parameter training module, wherein: the pre-training module is used to pre-train the backbone network using the original dataset to obtain the pre-trained model; the first construction module is used to add the first head network to the network structure of the pre-trained model for each domain-corresponding sub-dataset to obtain a constructed first network model for the corresponding domain; the domain parameter training module is used to train the first head network using each domain-corresponding sub-dataset to obtain the trained first network model.

[0172] In some possible embodiments, the domain parameter training module includes a first prediction submodule, a second determination submodule, and a first update submodule, wherein: the first prediction submodule is used to process the subset of data through the first network model and output the prediction result of the subset of data; the second determination submodule is used to determine the first loss of the subset of data through the first network model based on the prediction result of the subset of data and the label carried by the subset of data; the first update submodule is used to update the parameters of the first head network in the first network model based on the first loss, while keeping the parameters of the backbone network in the first network model fixed, until the first loss reaches the convergence condition, thereby obtaining the trained first network model.

[0173] In some possible embodiments, the transfer training module 704 includes a second prediction submodule, a third determination submodule, and a second update submodule, wherein: the second prediction submodule is used to process the task dataset through the second network model and output the prediction result of the task dataset; the third determination submodule is used to determine the second loss of the task dataset through the second network model based on the prediction result of the task dataset and the label carried by the task dataset; the second update submodule is used to update the parameters of the second head network in the second network model based on the second loss, while keeping the parameters of the backbone network and the first head network in the second network model fixed, until the second loss reaches the convergence condition.

[0174] In some possible embodiments, the second head network includes a fully connected layer, and the device 700 further includes a second building module for adding the fully connected layer after the output layer of the first network model based on the task dataset to obtain the built second network model; the number of neurons in the fully connected layer corresponds to the number of categories of data in the task dataset.

[0175] In some possible embodiments, the task dataset is a set of face samples, and the device further includes a third acquisition module and a recognition module, wherein: the third acquisition module is used to acquire face images of the target object; acquire a second network model trained using the face dataset; and the recognition module is used to perform face recognition on the face images through the second network model to obtain the face recognition result of the target object.

[0176] The descriptions of the above apparatus embodiments are similar to those of the above transfer learning method embodiments, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this disclosure can be used to execute the methods described in the above method embodiments. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0177] It should be noted that, in the embodiments of this application, if the above-described transfer learning method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0178] This application provides a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0179] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0180] This application provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0181] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0182] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0183] It should be noted that, Figure 8 This is a schematic diagram of a hardware entity of a computer device in an embodiment of this application, such as... Figure 8 As shown, the hardware entity of the computer device 800 includes: a processor 801, a communication interface 802, and a memory 803, wherein:

[0184] Processor 801 typically controls the overall operation of computer device 800.

[0185] The communication interface 802 enables computer devices to communicate with other terminals or servers over a network.

[0186] The memory 803 is configured to store instructions and applications executable by the processor 801, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 801 and various modules in the computer device 800. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 801, the communication interface 802, and the memory 803 can be performed via bus 804.

[0187] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0188] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0189] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0190] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0191] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0192] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0193] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0194] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A transfer learning method, characterized in that, The method comprises: obtaining an original data set and a task data set; the task data set is a face image sample set; determining a target subset of the original data set divided on a domain involved in the task data set; obtaining a first network model trained by using the target subset; wherein the first network model is built based on a pre-training model and a first head network; the pre-training model is trained by using the original data set; training a second network model built by using the task data set to obtain a trained second network model; wherein the second network model is built based on the first network model and a second head network; the first network model is trained by the following steps: pre-training a backbone network by using the original data set to obtain the pre-training model; adding the first head network based on the network structure of the pre-training model for each domain corresponding sub-data set to obtain the first network model of the corresponding domain built; the first head network is a network structure containing a multi-layer perceptron, which is used for domain-based parameter learning to realize feature selection; training the first head network by using the sub-data set corresponding to each domain while fixing the parameters of the backbone network in the first network model to obtain the trained first network model.

2. The method of claim 1, wherein, The determination of the target subset of the original data set divided on the domain involved in the task data set comprises: clustering the original data set to obtain N domain corresponding sub-data sets; N is an integer greater than or equal to 2; determining the target subset of the same domain as the task data set from the N sub-data sets.

3. The method of claim 2, wherein, The determination of the target subset of the same domain as the task data set from the N sub-data sets comprises: determining the domain to which each frame of image in the task data set belongs from the N domains corresponding to the N sub-data sets; determining the target domain corresponding to the task data set according to the proportion of the domain to which each frame of image in the task data set belongs in the N domains; taking the sub-data set corresponding to the target domain in the N sub-data sets as the target subset.

4. The method of claim 3, wherein, The determination of the domain to which each frame of image in the task data set belongs from the N domains corresponding to the N sub-data sets comprises: determining the first distance between each frame of image in the task data set and the class center of the N sub-data sets; wherein the value of the class center is the average value of all data in the sub-data set in each dimension; in the case that there is a target class center whose first distance meets a preset threshold, determining that the each frame of image belongs to the domain corresponding to the target class center.

5. The method of claim 1, wherein, The training of the first head network by using the sub-data set corresponding to each domain to obtain the trained first network model comprises: processing the sub-data set through the first network model to output the prediction result of the sub-data set; determining the first loss of the sub-data set through the first network model based on the prediction result of the sub-data set and the label carried by the sub-data set; Based on the first loss, parameters of the first head network in the first network model are updated while parameters of a backbone network in the first network model are fixed until the first loss reaches a convergence condition, to obtain a trained first network model.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: processing the task data set through the second network model to output a prediction result of the task data set; based on the prediction result of the task data set and the label carried by the task data set, determining a second loss of the task data set through the second network model; based on the second loss, parameters of the second head network in the second network model are updated while parameters of the backbone network in the second network model and the first head network are fixed until the second loss reaches a convergence condition.

7. The method according to any one of claims 1 to 5, wherein The second head network comprises a fully connected layer, and the method further comprises: after the output layer of the first network model, the fully connected layer is added based on the task data set to obtain the second network model; the number of neurons of the fully connected layer conforms to the number of categories of data in the task data set.

8. The method according to any one of claims 1 to 5, wherein, The method further comprises: obtaining a face image of a target object collected; performing face recognition on the face image through the second network model to obtain a face recognition result of the target object.

9. A transfer learning device, characterized in that, The device comprises a first acquisition module, a determination module, a second acquisition module, a transfer training module, a pre-training module, a first building module, and a domain parameter training module, wherein: the first acquisition module is configured to acquire an original data set and a task data set; the task data set is a face image sample set; the determination module is configured to determine a target subset of the original data set divided on a domain involved in the task data set; the second acquisition module is configured to acquire a first network model trained using the target subset; wherein the first network model is built based on a pre-trained model and a first head network; the pre-trained model is trained using the original data set; the transfer training module is configured to train a second network model built using the task data set to obtain a trained second network model; wherein the second network model is built based on the first network model and a second head network; the pre-training module is configured to pre-train a backbone network using the original data set to obtain the pre-trained model; the first building module is configured to add the first head network to the network structure of the pre-trained model for each domain corresponding to a subset of data to obtain a first network model of the corresponding domain built; the first head network is a network structure comprising a multilayer perceptron, and is configured to perform domain-based parameter learning to realize feature selection; the domain parameter training module is configured to train the first head network using the subset of data corresponding to each domain while parameters of the backbone network in the first network model are fixed to obtain the trained first network model.

10. A computer device comprising a memory and a processor, the memory storing a computer program capable of running on the processor, characterized in that, The processor, when executing the program, implements the steps in the method of any one of claims 1 to 8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program, when executed by the processor, implements the steps in the method of any one of claims 1 to 8.

Images