Method and device for obtaining cross-domain learning model

Abstract

The embodiment of the present application discloses a method and device for obtaining a cross-domain learning model. The method includes: obtaining training data from multiple source domains; using the training data from one part of the source domain as meta-training data, and using the training data from another part of the source domain as meta-test data; using the meta-training data and meta-test data to train the cross-domain learning model, and the training objectives include: the benefits of the cross-domain learning model trained with the meta-training data on the meta-test data meet the supermodularity of the convex game; and the output results of the cross-domain learning model for the meta-training data and meta-test data meet expectations; the trained cross-domain learning model is used to analyze and predict the input data from the target domain, and output the prediction results, and the target domain is one of the multiple source domains or other domains in addition to the multiple source domains. The cross-domain generalization performance of the model can be improved through this application.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and device for obtaining a cross-domain learning model. Background Art

[0002] As a core component of artificial intelligence, machine learning has gradually gained widespread application in various scenarios. Machine learning uses machine learning models to analyze and predict data. These models include classification models, regression models, ranking models, and time series prediction models.

[0003] With the increasing popularity of machine learning applications, more and more models have cross-domain requirements. For example, in the image recognition scenario, the image recognition model established for images in some domains can also be applied to image recognition in other domains. For another example, in the information recommendation scenario, the information recommendation model established for some domains can also be applied to information recommendation in other domains. In other words, the cross-domain learning model that needs to be obtained has relatively good generalization performance. However, the generalization performance of the cross-domain learning model obtained in the existing methods is poor, and it can only have a good effect in the domain to which the training data belongs, and the model effect in other domains is poor. Summary of the Invention

[0004] In view of this, the present application provides a method and apparatus for obtaining a cross-domain learning model to improve the cross-domain generalization performance of the model.

[0005] This application provides the following solutions:

[0006] According to a first aspect, a method for obtaining a cross-domain learning model is provided, the method comprising:

[0007] Obtain training data from multiple source domains;

[0008] The training data from one part of the source domain is used as meta-training data, and the training data from the other part of the source domain is used as meta-test data;

[0009] The cross-domain learning model is trained using the meta-training data and the meta-test data, wherein the training objectives include a first training objective and a second training objective; the first training objective is that the return of the cross-domain learning model trained using the meta-training data on the meta-test data complies with the supermodularity of the convex game; and the second training objective is that the output results of the cross-domain learning model for the meta-training data and the meta-test data meet expectations;

[0010] The trained cross-domain learning model is used to analyze and predict input data from a target domain and output a prediction result, where the target domain is one of the multiple source domains or any other domain other than the multiple source domains.

[0011] According to an achievable method in an embodiment of the present application, before using the training data from one part of the source domain as meta-training data and using the training data from another part of the source domain as meta-test data, the method further includes:

[0012] Fourier augmentation is performed on the training data from the multiple source domains to obtain augmented training data of each source domain.

[0013] According to an achievable manner in an embodiment of the present application, the method further includes: constructing a first data set and a second data set using the meta-training data, obtaining an intersection of the first data set and the second data set as a third data set, and obtaining a union of the first data set and the second data set as a fourth data set;

[0014] Training the cross-domain learning model using the meta-training data and the meta-test data includes: performing meta-training on the cross-domain learning model using the first data set, the second data set, the third data set, and the fourth data set, respectively, to obtain first model parameters, second model parameters, third model parameters, and fourth model parameters; respectively obtaining first loss function values, second loss function values, third loss function values, and fourth loss function values corresponding to the meta-test data on the first model parameters, the second model parameters, the third model parameters, and the fourth model parameters;

[0015] The first training objective includes: minimizing the difference obtained by subtracting the sum of the first loss function and the second loss function values from the sum of the third loss function value and the fourth loss function value.

[0016] According to an achievable method in an embodiment of the present application, the training includes:

[0017] In each round of iteration, the value of the total loss function is determined, and the model parameters are updated using the value of the total loss function until the preset training end conditions are met;

[0018] The total loss function is obtained by weighted summation of the fifth loss function and the sixth loss function, the fifth loss function is pre-constructed based on the first training objective, and the sixth loss function is pre-constructed based on the second training objective.

[0019] According to an achievable manner in an embodiment of the present application, the method further includes:

[0020] Determining a contribution score of the training data to the first training objective;

[0021] Deleting training data whose contribution score meets a preset low-quality score standard from the training data;

[0022] The training data obtained after the deletion process is used to update the model parameters to achieve the second training goal.

[0023] According to an achievable method in an embodiment of the present application, determining the contribution score of the training sample to the first training objective includes:

[0024] Determine a contribution score of the training sample to the first training objective by multiplying the gradient of the training sample with the feature representation of the training sample by a fifth loss function, wherein the fifth loss function is pre-constructed based on the first training objective.

[0025] According to an achievable method in an embodiment of the present application, the training data of the multiple source domains include: images of multiple fields and their classification labels, and the cross-domain learning model is an image recognition model; or,

[0026] The training data of the multiple source domains include: users from multiple countries and promotion data that the users are interested in, and the cross-domain learning model is an information recommendation model.

[0027] According to a second aspect, a device for obtaining a cross-domain learning model is provided, the device comprising:

[0028] A sample acquisition unit is configured to acquire training data from multiple source domains;

[0029] a sample partitioning unit configured to use the training data from one part of the source domain as meta-training data and the training data from another part of the source domain as meta-test data;

[0030] a model training unit configured to train the cross-domain learning model using the meta-training data and the meta-test data, wherein the training objectives include a first training objective and a second training objective; the first training objective is that the payoff of the cross-domain learning model trained using the meta-training data on the meta-test data complies with the supermodularity of the convex game; and the second training objective is that the output results of the cross-domain learning model for the meta-training data and the meta-test data meet expectations;

[0031] The trained cross-domain learning model is used to analyze and predict input data from a target domain and output a prediction result. The target domain is one of the multiple source domains or any other domain other than the multiple source domains.

[0032] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods according to the first aspect are implemented.

[0033] According to a fourth aspect, an electronic device is provided, comprising:

[0034] one or more processors; and

[0035] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the first aspects above.

[0036] According to the specific embodiments provided in this application, this application can have the following technical effects:

[0037] 1) This application introduces the supermodularity of convex games into the training of cross-domain learning models, encouraging training data from various fields to contribute to improving the generalization performance of the model, so that the cross-domain learning model can better utilize information in different fields, learn models with stronger generalization performance, and improve the model effect in fields other than the source domain.

[0038] 2) Using Fourier augmentation technology to augment the training data in the source domain can increase the diversity of training data and better improve the generalization performance of the cross-domain learning model.

[0039] 3) Based on the contribution of the training data to the first training objective, low-quality training samples that meet the preset low-quality scoring criteria are filtered out. This can avoid the negative impact of noisy and redundant data on cross-domain learning, effectively ensure that the training data improves the generalization performance of the model, and thus learn a more stable model.

[0040] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 An exemplary system architecture diagram to which embodiments of the present application can be applied is shown;

[0043] Figure 2A flow chart of the method for obtaining a cross-domain learning model provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the basic principles of model training provided in the embodiments of the present application;

[0045] Figure 4 A schematic block diagram illustrating an apparatus for acquiring a cross-domain learning model according to one embodiment is shown;

[0046] Figure 5 This is an architectural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0048] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "an", "the" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0049] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0050] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0051] To facilitate understanding of this application, we first briefly describe the system architecture on which this application is based. Figure 1 An exemplary system architecture to which the embodiments of the present application can be applied is shown. Figure 1 As shown in , the system mainly includes a device for obtaining a cross-domain learning model and a data warehouse of N domains, where N is a positive integer greater than 1.

[0052] The device for obtaining the cross-domain learning model obtains relevant data of N fields from the data warehouse as training data to train the cross-domain learning model. The purpose is to enable the obtained cross-domain learning model to not only analyze and predict the input data of the above N fields and obtain accurate prediction results, but also to be able to analyze and predict other fields ( Figure 1 The input data of N+1 domains (represented in ) can be analyzed and predicted to obtain accurate prediction results, which makes the cross-domain learning model have cross-domain generalization performance.

[0053] The cross-domain learning model can be a classification model, a regression model, a ranking model, a time series prediction model, etc. The prediction results can be a classification result, a regression prediction result, a ranking result, a numerical prediction result at a specific moment, etc.

[0054] The aforementioned areas can be different service areas, different regions, different types of data sets, and so on.

[0055] The device for obtaining the cross-domain learning model can be set up on the server side or on a computer terminal with powerful computing capabilities. The server side can be a single server, a server group consisting of multiple servers, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system. It solves the problems of difficult management and poor service scalability in traditional physical hosts and virtual private servers (VPs).

[0056] It should be understood that Figure 1 The number of data warehouses and domains in the example is only illustrative. Any number of data warehouses and domains can be used according to implementation needs.

[0057] Figure 2 A flow chart of a method for obtaining a cross-domain learning model provided in an embodiment of the present application, wherein the method comprises Figure 1 The device for obtaining the cross-domain learning model in the system shown in the figure can be an application located on the server side, or a functional unit such as a plug-in or software development kit (SDK) in the application located on the server side, or a computer terminal with strong computing power. Figure 2 As shown in , the method includes the following steps:

[0058] Step 202: Obtain training data from multiple source domains.

[0059] Step 204: The training data from one part of the source domain is used as meta-training data, and the training data from the other part of the source domain is used as meta-testing data.

[0060] Step 206: Use the meta-training data and the meta-test data to train the cross-domain learning model, where the training objectives include a first training objective and a second training objective; the first training objective is: the benefits of the cross-domain learning model trained using the meta-training data on the meta-test data conform to the supermodularity of the convex game; the second training objective is: the output results of the cross-domain learning model for the meta-training data and the meta-test data meet expectations; the trained cross-domain learning model is used to analyze and predict input data from the source domain or input data from domains other than the source domain, and output prediction results.

[0061] In the embodiment of the present application, the existing domain used to train the cross-domain learning model is referred to as the source domain, and the supermodularity of convex games is introduced into the training of the cross-domain learning model, so that the cross-domain learning model can better utilize information in different domains, learn a model with stronger generalization performance, and improve the model effect in other domains besides the source domain.

[0062] The above steps are described in detail below. First, the above step 202, namely "obtaining training data from multiple source domains", is described in detail with reference to the embodiment.

[0063] In this embodiment, an existing domain with a certain amount of data can be used as the source domain. The content of the training data is related to the specific application scenario and model type. For supervised learning models, the training data usually contains samples and labels for the samples.

[0064] For example, in image recognition applications, image samples and their classification labels from multiple fields can be used as training samples to train image recognition models. Fields can include different service areas, such as healthcare, technology, and entertainment. They can also include datasets of different types, such as paintings, sketches, cartoons, photos, and silhouettes.

[0065] For example, in an information recommendation application scenario, users from multiple countries and the promotional data that users follow can be used as training data to train the information recommendation model. Countries are different fields divided by different regions.

[0066] Furthermore, in order to increase the diversity of the training data in the source domain and better improve the generalization performance of the model, the training data from multiple source domains can be Fourier augmented to obtain the augmented training data of each source domain. Figure 3 As shown in , it is assumed that the training data of multiple source domains are represented as D s , perform Fourier augmentation to obtain augmented data Will as augmented training data.

[0067] Since the phase component of the Fourier spectrum retains the basic semantics of the original signal, while the amplitude component contains low-level statistical features, the training data can be Fourier transformed to obtain amplitude and phase components. The basic semantic information contained in the phase component is then retained, while the amplitude component is perturbed to generate a new amplitude component. The phase component and the new amplitude component are then inverse Fourier transformed to obtain the augmented data.

[0068] When perturbing the amplitude component, random perturbation can be adopted, or linear interpolation of the amplitude components of two training samples in the training data of any source domain can be performed, or the amplitude components of any two training samples can be exchanged, and so on.

[0069] The above step 204, i.e., "using the training data from one part of the source domain as meta-training data and using the training data from another part of the source domain as meta-testing data", is described in detail below with reference to an embodiment.

[0070] This step actually involves partitioning the training data into meta-training data and meta-testing data based on different domains. Training data from one source domain is used as meta-training data, while training data from another source domain is used as meta-testing data. This effectively uses training data from one source domain to verify the effectiveness of model training on another source domain (i.e., the simulated target domain). For example, assuming there is training data from N source domains, training data from n1 of these source domains is used as meta-training data, while training data from the remaining N-n1 source domains is used as meta-testing data. The selection of the n1 source domains can be random or based on a specific strategy.

[0071] If there is augmented training data, such as Figure 3 As shown in s Randomly divided into two parts according to the field: and And, from the augmented data Confirmed Corresponding augmented data You can As meta-training data, as meta-test data.

[0072] Furthermore, if Figure 3As shown in , in order to introduce the supermodularity of convex games in the subsequent model training process, the meta-training data can be used to construct coalitions and generate multiple data sets: S, T, S∪T, S∩T. Among them, S is the first data set consisting of training data randomly selected from the meta-training data, T is the second data set consisting of training data randomly selected from the meta-training data, S∪T is the third data set consisting of the union of the first data set and the second data, and S∩T is the fourth data set consisting of the intersection of the first data set and the second data.

[0073] It should be noted here that the limitations such as "first" and "second" involved in the embodiments of the present application do not have restrictions on size, order, quantity, etc., and are only used to distinguish them in name, for example, "first data set" and "second data set" are used for two data sets.

[0074] The above step 206, namely “training a cross-domain learning model using meta-training data and meta-testing data”, is described in detail below with reference to an embodiment.

[0075] The cross-domain learning model involved in the embodiments of the present application can be various types of deep learning models, such as classification models, regression models, ranking models, time series prediction models, etc. When starting training, the model parameters are first initialized, such as Figure 3 As shown in , the model can be represented as f(·,θ), where the model parameters θ are initialized.

[0076] Model training has two main training objectives:

[0077] The first training goal is to ensure that the benefits of the cross-domain learning model trained with meta-training data on meta-test data conform to the supermodularity of convex games.

[0078] The second training goal is to ensure that the output results of the cross-domain learning model for meta-training data and meta-test data meet expectations.

[0079] First, how to perform training to achieve the above-mentioned first training goal is described.

[0080] The cross-domain learning model can be meta-trained using the first data set, the second data set, the third data set, and the fourth data set to obtain first model parameters, second model parameters, third model parameters, and fourth model parameters. The first loss function value, the second loss function value, the third loss function value, and the fourth loss function value corresponding to the meta-test data on the first model parameters, the second model parameters, the third model parameters, and the fourth model parameters are obtained respectively. The model parameters are updated using the above-mentioned first loss function value, the second loss function value, the third loss function value, and the fourth loss function value to minimize the difference obtained by subtracting the sum of the first loss function and the second loss function values from the sum of the third loss function value and the fourth loss function value.

[0081] Combine Figure 3 , we can use four data sets: S, T, S∪T, S∩T to meta-train the model f(·,θ). Taking the classification model as an example, we can first construct a supervised classification loss function to update the model parameters of f(·,θ). This update is actually a fake update, that is, it is not a real update of the model parameters, but an algorithm is used to calculate what the updated model parameters are after meta-training the model f(·,θ) using S, T, S∪T, S∩T. It is assumed to be expressed as: θ' S ,θ' T ,θ' S∪T and θ' S∩T . Based on the updated model parameters, the meta-test data Constructing supervised classification loss function And calculate respectively in θ' S ,θ' T ,θ' S∪T and θ' S∩T On the meta-test data The loss function value obtained and

[0082] As one of the preferred implementation methods, this application formalizes the domain generalization problem into a convex game problem between domains, ensures the effectiveness of data information migration through the supermodularity of convex games, and makes more effective use of cross-domain data information.

[0083] According to the supermodularity definition of convex games, the benefits obtained from the intersection of two data sets plus the benefits obtained from the union of the two data sets must be no less than the sum of the benefits obtained by each of the two data sets. This is the only way to achieve better cooperation between various fields. Based on this theory, a regularization loss function L can be constructed in the embodiment of this application. sm :

[0084]

[0085] The first training goal can be seen as training the cross-domain learning model to minimize the regularization loss function L sm , minimize the above regularization loss function L sm In fact, it is to encourage training data in various fields to contribute to improving the generalization performance of the model.

[0086] The following describes how to perform training to achieve the above-mentioned second training goal.

[0087] The second training goal is to make the output of the training data meet the expectations. The training data includes meta-training data and meta-test data. For supervised learning, the output is usually to minimize the difference between the output and the label in the corresponding training data. This training goal can be used to construct the loss function L sup This loss function is primarily used for supervised learning and can be configured based on different model types. For classification models, for example, Hingle loss and cross-entropy loss functions can be constructed. For regression models, for example, squared loss, absolute loss, Huber loss, and so on can be constructed.

[0088] But considering the sample regularization loss function L in the training data sm The greater the contribution, the less conducive it is to improving the generalization performance of the model, which means the lower the sample quality. Therefore, as a preferred embodiment, the contribution score of the training samples to the first training objective can be determined, and the training samples whose contribution scores meet the preset low-quality score standard can be deleted from the training samples to obtain high-quality training samples; the obtained high-quality training samples are used to update the model parameters to achieve the second training objective.

[0089] Combine Figure 3 To illustrate, in the embodiment of the present application, the product of the gradient of the training data and the feature representation of the training sample can be used to evaluate the contribution of the training data to the first training objective. The regularization term loss function L can be calculated first. sm The gradient of sample x in the training data is expressed as Then As the contribution score of the training data corresponding to sample x. The higher the score, the greater the negative impact of the training data on the generalization performance of the model and the worse the quality. The contribution score can be used to filter samples, for example, training data The training data with contribution score greater than or equal to the preset score threshold are deleted. The training data with the top k contribution scores are deleted, where k is a preset positive integer. The deleted training data is expressed as: The deleted training data is expressed as: Calculate the loss function Lsup , the second training goal can be seen as training the cross-domain learning model to minimize L sup .

[0090] By filtering low-quality training samples, we can avoid the negative impact of noisy and redundant data on cross-domain learning, effectively ensure that cross-domain training data improves the generalization performance of the model, and thus learn a more stable model.

[0091] In the above model training process, the first training objective and the second training objective can be regarded as two training tasks, and the training is performed alternately to update the model parameters. The first training objective and the second training objective can also be regarded as a whole training task, for example, to construct the total loss function L = L sup +ωL sm , update the model parameters to minimize L. Where ω is the preset weight value, which can be an empirical value.

[0092] The model parameters can be updated in each iteration using methods such as gradient descent using the value of the loss function L until a preset training termination condition is met. The training termination condition may include, for example, the value of the loss function L being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0093] After completing model training through the above process, a cross-domain learning model is obtained. This cross-domain learning model can be used to analyze and predict input data in the target domain and output prediction results. The target domain can be one of the above source domains or another domain in addition to the source domain. In other words, the trained model is not a model that is targeted at a specific domain and can only analyze and predict input data in a specific domain. Instead, it is a cross-domain model that can achieve good prediction results for the same task in various domains and has high generalization performance.

[0094] Here are two application scenarios:

[0095] Application scenario 1:

[0096] In the image recognition scenario, image data in the fields of sketches, photos, cartoons, etc. is relatively easy to obtain and the data volume is large. Therefore, we can obtain image data in the fields of sketches, photos, cartoons, etc., label the objects in the image data, and use the image data and its labels as training data D s . s Perform Fourier augmentation to obtain augmented data

[0097] D s Randomly divided into two parts according to the field: and Assume that the image data of sketches and photos are randomly taken as The cartoon image data is used as After model training using the method described in the above method embodiment, the resulting image recognition model not only has good recognition capabilities on image data in the source domain (i.e., the second training objective), but also has good cross-domain generalization performance, i.e., the sum of the gains on the union and intersection of the training data in each domain is greater than the sum of the gains on the training data in each domain separately (the first training objective). In other words, the trained image recognition model is not only capable of accurately recognizing image data in the fields of sketches, photos, and cartoons, but also image data in other fields such as silhouettes.

[0098] Application Scenario 2:

[0099] In the information recommendation scenario, a large amount of user promotion data for countries such as country A, country B, and country C has been obtained, such as user information and promotion data that users follow. Therefore, we can obtain user promotion data for countries such as country A, country B, and country C, and form sample pairs such as "user-promotion data that users follow", and use these sample pairs as training data D s . s Perform Fourier augmentation to obtain augmented data

[0100] D s Randomly divided into two parts according to the field: and Assume that the user promotion data of country A and country B are randomly selected as The user promotion data of country C is used as After model training using the method described in the above method embodiment, the resulting information recommendation model not only has good information recommendation capabilities for users in the source domain (i.e., the second training objective), but also has good cross-domain generalization performance, that is, the sum of the benefits of the union and intersection of the training data in each domain is greater than the sum of the benefits of the training data in each domain separately (the second training objective). In other words, the trained information recommendation model can not only accurately recommend information to users in countries such as Country A, Country B, and Country C, but also accurately recommend information to users in other countries such as Country D.

[0101] By using the method provided in the embodiment of the present application in this application scenario, cross-domain learning of information can be performed between different countries, so that the basic semantic information of each country can be shared, and a model that can support multi-country information recommendation can be constructed. In other words, the unified model learned across domains has good generalization performance, and in terms of prediction accuracy, it even has better results than the models learned separately using user promotion data of a single country.

[0102] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] According to an embodiment of another aspect, a device for acquiring a cross-domain learning model is provided. Figure 4 FIG. 1 is a schematic block diagram of an apparatus for obtaining a cross-domain learning model according to an embodiment. Figure 4 As shown, the apparatus 400 includes: a sample acquisition unit 401, a sample division unit 402, and a model training unit 403. It may also include a sample augmentation unit 404, an alliance construction unit 405, and a sample filtering unit 406. The main functions of each component unit are as follows:

[0104] The sample acquisition unit 401 is configured to acquire training data from multiple source domains.

[0105] The sample partitioning unit 402 is configured to use the training data from one part of the source domain as meta-training data, and use the training data from another part of the source domain as meta-test data.

[0106] The training data from one part of the source domain is used as meta-training data, and the training data from the other part of the source domain is used as meta-test data. In fact, the training data from one part of the source domain is used to verify the model training effect on the other part of the source domain (i.e., the simulated target domain).

[0107] The model training unit 403 is configured to train the cross-domain learning model using meta-training data and meta-test data, and the training objectives include a first training objective and a second training objective; the first training objective is: the benefits of the cross-domain learning model trained using the meta-training data on the meta-test data conform to the supermodularity of the convex game; the second training objective is: the output results of the cross-domain learning model for the meta-training data and the meta-test data meet expectations.

[0108] The cross-domain learning models involved in the embodiments of the present application can be a variety of deep learning models, such as classification models, regression models, sorting models, time series prediction models, etc.

[0109] The cross-domain learning model obtained by the above training is used to analyze and predict the input data from the target domain and output the prediction results. The target domain is one of the multiple source domains or other domains in addition to the multiple source domains.

[0110] As one possible implementation, the sample augmentation unit 404 is configured to perform Fourier augmentation on the training data from multiple source domains respectively to obtain augmented training data of each source domain.

[0111] Specifically, the sample augmentation unit 404 may perform Fourier transform on the training data to obtain an amplitude component and a phase component, then perturb the amplitude component to generate a new amplitude component, and then perform inverse Fourier transform on the phase component and the new amplitude component to obtain augmented data.

[0112] Furthermore, the alliance construction unit 405 is configured to construct a first data set and a second data set using the meta-training data, obtain the intersection of the first data set and the second data set as a third data set, and obtain the union of the first data set and the second data set as a fourth data set.

[0113] Accordingly, the model training unit 403 is specifically configured to: perform meta-training on the cross-domain learning model using the first data set, the second data set, the third data set, and the fourth data set, respectively, to obtain first model parameters, second model parameters, third model parameters, and fourth model parameters; and obtain first loss function values, second loss function values, third loss function values, and fourth loss function values corresponding to the meta-test data on the first model parameters, the second model parameters, the third model parameters, and the fourth model parameters, respectively. In this case, the first training objective includes minimizing the difference obtained by subtracting the sum of the first loss function and the second loss function values from the sum of the third loss function value and the fourth loss function value.

[0114] As one possible approach, the secondary training objectives include:

[0115] Minimize the difference between the output of the cross-domain learning model for samples in the meta-training data and the labels corresponding to the samples.

[0116] Furthermore, the sample filtering unit 406 may be configured to: determine a contribution score of the training data to the first training objective; and delete, from the training data, training data whose contribution score meets a preset low-quality score standard.

[0117] Accordingly, the model training unit 403 updates the model parameters using the training data obtained after the deletion process to achieve the second training goal.

[0118] As one of the feasible ways, the sample filtering unit 406 can be specifically configured to determine the contribution score of the training sample to the first training objective by using the product of the gradient of the training sample of the fifth loss function and the feature representation of the training sample; wherein the fifth loss function is pre-constructed based on the first training objective.

[0119] It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0120] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0121] And an electronic device comprising:

[0122] one or more processors; and

[0123] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

[0124] in, Figure 5 The electronic device architecture is shown as an example, and may include a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, and a memory 520. The processor 510, the video display adapter 511, the disk drive 512, the input / output interface 513, the network interface 514, and the memory 520 may be communicatively connected via a communication bus 530.

[0125] The processor 510 may be implemented as a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and may be used to execute relevant programs to implement the technical solutions provided in this application.

[0126] The memory 520 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 520 can store an operating system 521 for controlling the operation of the electronic device 500, and a basic input and output system (BIOS) 522 for controlling the low-level operation of the electronic device 500. In addition, a web browser 523, a data storage management system 524, and a device 525 for obtaining a cross-domain learning model, etc. can also be stored. The above-mentioned device 525 for obtaining a cross-domain learning model can be an application program that specifically implements the operations of the aforementioned steps in an embodiment of the present application. In short, when the technical solution provided in this application is implemented by software or firmware, the relevant program code is stored in the memory 520 and is called and executed by the processor 510.

[0127] The input / output interface 513 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0128] The network interface 514 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0129] The bus 530 comprises a pathway for transmitting information between the various components of the device (eg, the processor 510 , the video display adapter 511 , the disk drive 512 , the input / output interface 513 , the network interface 514 , and the memory 520 ).

[0130] It should be noted that although the above device only shows a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, a memory 520, a bus 530, etc., in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may also include only the components necessary to implement the solution of the present application, and does not necessarily include all the components shown in the figure.

[0131] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0132] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0133] The above is a detailed introduction to the technical solutions provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting this application.

Claims

1. A method for obtaining a cross-domain learning model, characterized in that: The method includes: Obtain training data from multiple source domains; The training data from one part of the source domain is used as meta-training data, and the training data from the other part of the source domain is used as meta-test data; The cross-domain learning model is trained using the meta-training data and the meta-test data, wherein the training objectives include a first training objective and a second training objective; the first training objective is that the return of the cross-domain learning model trained using the meta-training data on the meta-test data complies with the supermodularity of the convex game; and the second training objective is that the output results of the cross-domain learning model for the meta-training data and the meta-test data meet expectations; The trained cross-domain learning model is used to analyze and predict input data from a target domain and output a prediction result, wherein the target domain is one of the multiple source domains or any other domain other than the multiple source domains; The training data of the multiple source domains include: images of multiple fields and their classification labels, and the cross-domain learning model is an image recognition model; or The training data of the multiple source domains include: users from multiple countries and promotion data that the users are interested in, and the cross-domain learning model is an information recommendation model.

2. The method according to claim 1, characterized in that Before using the training data from one part of the source domain as meta-training data and the training data from another part of the source domain as meta-testing data, the method further includes: Fourier augmentation is performed on the training data from the multiple source domains to obtain augmented training data of each source domain.

3. The method according to claim 1, characterized in that The method further includes: constructing a first data set and a second data set using the meta-training data, obtaining an intersection of the first data set and the second data set as a third data set, and obtaining a union of the first data set and the second data set as a fourth data set; Training the cross-domain learning model using the meta-training data and the meta-test data includes: performing meta-training on the cross-domain learning model using the first data set, the second data set, the third data set, and the fourth data set, respectively, to obtain first model parameters, second model parameters, third model parameters, and fourth model parameters; respectively obtaining first loss function values, second loss function values, third loss function values, and fourth loss function values corresponding to the meta-test data on the first model parameters, the second model parameters, the third model parameters, and the fourth model parameters; The first training objective includes: minimizing the difference obtained by subtracting the sum of the first loss function and the second loss function values from the sum of the third loss function value and the fourth loss function value.

4. The method according to any one of claims 1 to 3, characterized in that The training includes: In each round of iteration, the value of the total loss function is determined, and the model parameters are updated using the value of the total loss function until the preset training end conditions are met; The total loss function is obtained by weighted summation of the fifth loss function and the sixth loss function, the fifth loss function is pre-constructed based on the first training objective, and the sixth loss function is pre-constructed based on the second training objective.

5. The method according to claim 1, wherein The method further includes: Determining a contribution score of the training data to the first training objective; Deleting training data whose contribution score meets a preset low-quality score standard from the training data; The training data obtained after the deletion process is used to update the model parameters to achieve the second training goal.

6. The method according to claim 5, characterized in that Determining a contribution score of the training data to the first training objective includes: Determine a contribution score of the training sample to the first training objective by multiplying the gradient of the training sample with the feature representation of the training sample by a fifth loss function, wherein the fifth loss function is pre-constructed based on the first training objective.

7. A device for obtaining a cross-domain learning model, characterized in that: The device includes: A sample acquisition unit is configured to acquire training data from multiple source domains; a sample partitioning unit configured to use the training data from one part of the source domain as meta-training data and the training data from another part of the source domain as meta-test data; a model training unit configured to train the cross-domain learning model using the meta-training data and the meta-test data, wherein the training objectives include a first training objective and a second training objective; the first training objective is that the payoff of the cross-domain learning model trained using the meta-training data on the meta-test data complies with the supermodularity of the convex game; and the second training objective is that the output results of the cross-domain learning model for the meta-training data and the meta-test data meet expectations; The trained cross-domain learning model is used to analyze and predict input data from a target domain and output a prediction result, wherein the target domain is one of the multiple source domains or any other domain other than the multiple source domains; The training data of the multiple source domains include: images of multiple fields and their classification labels, and the cross-domain learning model is an image recognition model; or, The training data of the multiple source domains include: users from multiple countries and promotion data that the users are interested in, and the cross-domain learning model is an information recommendation model.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. An electronic device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method according to any one of claims 1 to 6.

Images