A domain generalization method based on model sample matching

Abstract

The application provides a domain generalization method based on model sample matching, comprising the following steps: step S1: obtaining a data set, and adapting the prediction of a pre-trained model to a target domain through linear label space transformation; step S2: dynamically recommending a suitable pre-trained model to predict each test sample through a model characteristic matching network; and step S3: improving the stability of model convergence through a loss function.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of pattern recognition and computer vision, and particularly relates to a domain generalization method based on model sample matching. BACKGROUND

[0002] The development of artificial intelligence has greatly changed the pace of life and lifestyle of modern people, among which, neural networks are also increasingly valued. At present, neural networks mostly rely on training on large data sets to obtain a model for various tasks, but in some tasks, the objects to be recognized are unchanged, but the forms change, such as real cars and cars in paintings, although they are both cars but the forms are different, at this time, if there is a data set of one kind and wants to identify another kind, it is necessary to prepare another data set, the process is tedious. Therefore, the domain generalization method emerges as the times require, the domain generalization can predict the target domain data set in the case of only source domain data set and a small amount of target domain data, thereby reducing the data set preparation process in between and increasing the flexibility of application. SUMMARY

[0003] Therefore, in view of the actual needs and the blank in the prior art, the application provides a domain generalization method based on model sample matching, which can effectively solve the problem of domain generalization.

[0004] The technical scheme adopted by the application to solve the technical problem is specifically:

[0005] A domain generalization method based on model sample matching, comprising the following steps:

[0006] Step S1: obtaining a data set, and adapting the prediction of a pre-trained model to a target domain through a linear label space transformation;

[0007] Step S2: dynamically recommending a pre-trained model to predict each test sample through a model characteristic matching network;

[0008] Step S3: improving the stability of model convergence through a loss function.

[0009] Further, step S1 specifically comprises the following steps:

[0010] Step S11: obtaining a public domain self-adaptive picture data set and obtaining the related annotations of the training data;

[0011] Step S12: making the label spaces shared by the source domain and the target domain consistent through a label space adapter; the adapter is a linear mapping between two label spaces: pre-training→source / target.

[0012] Further, in step S12, a pre-trained model f k, and an adaptive model h φ ; the output of the pre-trained model f k is converted and adapted to the target domain in the following way:

[0013] y ik = h φ (f k (x i ))

[0014] where x i represents the i-th sample, and y ik represents the prediction of the i-th sample output by the k-th model.

[0015] Further, step S2 specifically comprises the following steps:

[0016] Step S21: define a model scheduler with parameter p, taking sample x i as input, and determine the weight w k of the model f k assigned to the sample x i , where where w k represents the estimation of the relative matching degree between the model f k in the model pool and the sample; based on the constructed model pool and the model scheduler, the prediction of each test sample is a set of predictions of the scheduled models; the final prediction y' i is calculated as:

[0017]

[0018] Finally, the loss function L D (φ, p) is calculated as follows:

[0019]

[0020] where φ represents the parameters of the adaptive model, and D represents the total number of samples;

[0021] Step S22: match the score m i = [m i1 ,…,m ik ] ∈ R K , where m ik represents the prediction of the i-th sample output by the k-th model: sample x i and model is calculated according to the latent feature c = {c1,…,c K}, where K represents the number of models; first, the sample and the model are embedded, and then converted into a joint latent space; a pre-trained model from the model pool characteristic extractor of the sample x i generating a sample embedding e i ; for each model f k embedding, a learnable embedding e k is introduced, which is adapted by mapping to the same space for fitness matching score;

[0022] Step S23: After constructing a series of ranking models according to the matching metric, the matching scores are standardized by the Softmax function; before reasoning for a given sample, further select the models with the highest matching scores.

[0023] Further, step S3 specifically includes the following steps:

[0024] update the matching network and the adapter by the loss L ens of ensemble learning:

[0025] L ens (φ, p) = l(y′ i , y i )

[0026] calculate the loss L ada of label space adapter learning:

[0027]

[0028] where w k represents the weight of the kth model;

[0029] The total loss L that needs to be minimized is defined as follows:

[0030] L = L ens (φ, p) + L ada (φ).

[0031] And an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of a model sample matching based domain generalization method as described above when executing the program.

[0032] A non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a model sample matching based domain generalization method as described above.

[0033] Compared with the prior art, the present application and its preferred schemes have at least the following outstanding effects:

[0034] 1. The adaptability of the pre-trained model can be improved, and the prediction of the pre-trained model is adapted to the target domain through linear label space transformation, thereby improving the generalization ability of the model.

[0035] 2、Can improve the ability of model self-adaptive prediction sample, predict each test sample by dynamically recommending appropriate pre-training model, improve the prediction performance of model on unknown sample;

[0036] 3、Can improve the convergence stability of model, and the model sample matching learning loss function improves the stability of model convergence. BRIEF DESCRIPTION OF DRAWINGS

[0037] The application will be further described in detail below in combination with the drawings and specific embodiments:

[0038] Figure 1 The method flowchart of the embodiment of the application is shown in the following. DETAILED DESCRIPTION

[0039] In order to make the features and advantages of the patent more obvious and easy to understand, the following examples are specifically described as follows:

[0040] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise indicated, all technical and scientific terms used in the specification have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0041] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and furthermore, it should be understood that when the terms "comprise" and / or "include" are used in the specification, they indicate the presence of a feature, step, operation, device, component and / or combination thereof.

[0042] As shown in the following table, the embodiment provides a domain generalization method based on model sample matching, which includes the following steps: Figure 1

[0043] Step S1: Obtain the data set, and adapt the prediction of the pre-training model to the target domain through linear label space transformation.

[0044] In the embodiment, step S1 specifically includes the following steps:

[0045] Step S11: Obtain the public domain self-adaptive picture data set, office-31 data set, from the network, and obtain the related annotation of the training data;

[0046] Step S12: Keep the label space shared by the source domain and the target domain consistent through a label space adapter. The adapter is a linear mapping between the two label spaces (pre-training -> source / target). Specifically, given a pre-training model f k ​, and the adaptive model h φ . Through the label adapter, the pre-trained model f k The output of can be transformed and adapted to the target domain as follows:

[0047] y ik =h φ (f k (x i ))

[0048] where x i represents the i-th sample, y ik represents the prediction of the i-th sample output by the k-th model.

[0049] Step S2: Dynamically recommend an appropriate pre-trained model to predict each test sample through the matching network of model characteristics.

[0050] In this embodiment, step S2 specifically includes the following steps:

[0051] Step S21: Define a model scheduler with parameter p, which takes samples x i As input, and determine the assignment to sample x i Model f k The weight w k ,in Here k Represents the model f in the model pool k An estimate of the relative matching degree between the samples. Based on the constructed model pool and model scheduler, the prediction of each test sample is the collection of predictions of the scheduling model. The final prediction y' i The calculation method is:

[0052]

[0053] The final loss function L D The calculation method of (φ,p) is as follows:

[0054]

[0055] Where φ represents the parameters of the adaptive model and D represents the total number of samples.

[0056] Step S22: Using the neural collaborative filtering recommendation algorithm, the matching score m i =[m i1 ,…,m ik ]∈R K , m ik Represents the prediction of the i-th sample output by the k-th model. Sample x i and model It is based on its potential characteristics c={c1,…,cK} is computed, where K denotes the number of models. The sample and model embeddings are first computed and then transformed into a joint latent space. One pre-trained model from the model pool The feature extractor of the model f i generates a sample embedding e i . For each model f k embedding, a learnable embedding e k is introduced to adaptively match the fitness score by mapping to the same space.

[0057] Step S23: A series of ranked models are constructed according to the matching metric. The matching scores are normalized by a Softmax function. The models with the top 5 matching scores are further selected before inference is performed on a given sample.

[0058] Step S3: The stability of model convergence is improved by a loss function.

[0059] In this embodiment, step S3 specifically comprises the following steps:

[0060] The loss L ens of ensemble learning is used to update the matching network and the adapter:

[0061] L ens (φ, p) = l(y i , y i )

[0062] The loss L ada of label space adapter learning is calculated as follows:

[0063]

[0064] where w k represents the weight of the kth model;

[0065] The total loss L to be minimized is defined as follows:

[0066] L = L ens (φ, p) + L ada (φ).

[0067] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems, or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0068] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0069] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0070] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0071] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the present application shall have the common meaning as understood by one of ordinary skill in the art to which this application pertains. The terms "first", "second", and similar terms, used in the present application do not denote any order, quantity, or importance, but are used to distinguish one element from another. The terms "include", "comprise", and similar terms are intended to mean that the elements listed after the terms encompass the elements recited in the terms, and equivalents thereof, and do not exclude other elements. The terms "connected", "coupled", and similar terms are not limited to physical or mechanical connections or couplings, but also include electrical connections or couplings, whether direct or indirect. The terms "upper", "lower", "left", "right", and similar terms are used only to indicate relative positional relationships when the absolute positions of the described objects are changed, and the relative positional relationships can also be changed accordingly.

[0072] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application in other forms. Any person skilled in the art can make changes or modifications to the above disclosed technical contents to obtain equivalent embodiments. However, any simple modification, equivalent change and modification made to the above embodiments without departing from the technical solution of the present application and according to the technical essence of the present application shall still fall within the protection scope of the present application.

[0073] The present application is not limited to the above preferred embodiments, and any person can derive other various forms of a model sample matching based domain generalization method under the inspiration of the present application. Any equivalent change and modification made within the scope of the present application shall fall within the scope of the present application.

Claims

1. A model sample matching based domain generalization method, characterized in that, The method comprises the following steps: Step S1: obtaining a data set and adapting the prediction of a pre-trained model to a target domain through a linear label space transformation; Step S2: dynamically recommending a pre-trained model to predict each test sample through a model characteristic matching network; Step S3: improving the stability of model convergence through a loss function; Step S1 specifically comprises the following steps: Step S11: obtaining a public domain self-adaptive picture data set and obtaining relevant annotations of training data; Step S12: making the label spaces shared by the source domain and the target domain consistent through a label space adapter; the adapter is a linear mapping between the two label spaces: pre-training→source / target; In step S12, a pre-trained model f k , and an adaptive model h φ are given; the output of the pre-trained model f k is converted and adapted to the target domain by a label adapter in the following way: y ik = h φ (f k (x i )) where x i represents the i-th sample, y ik represents the prediction of the i-th sample by the k-th model output; Step S2 specifically comprises the following steps: Step S21: define a model scheduler parameter p as a sample x i as input and determine the weight w i assigned to the model f k for the sample x k , where where w k represents the estimate of the relative match between the model f k in the model pool and the sample; based on the constructed model pool and the model scheduler, the prediction of each test sample is a set of predictions of the scheduled models; the final prediction y ' i The calculation method is: Finally, the loss function L D The (φ, p) is calculated as follows: Wherein φ represents the parameters of the adaptive model, and D represents the total number of samples; Step S22: matching score m i = [m i1 ,…,m ik ] ∈ R K , m ik represents the prediction of the i-th sample output by the k-th model: sample x i and model is calculated according to the latent feature c = {c1,…,c K}; first, the sample and the model are embedded, and then converted into a joint latent space; the feature extractor of a pre-trained model from the model pool generates a sample embedding e i for the sample x i ; for the embedding of each model f k , a learnable embedding e k is introduced to adaptively match the fitness score by mapping to the same space; Step S23: after a series of ranking models are constructed according to the matching metric; the matching scores are standardized through a Softmax function; and before a given sample is inferred, a model with the highest matching score is further selected.

2. The domain generalization method based on model sample matching according to claim 1, characterized in that: Step S3 specifically comprises the following steps: Loss L by ensemble learning ens Updating the matching network and the adapter: L ens (φ,p) = l(y i ,y i ) Loss L learned by the computing label space adapter ada : where w k represents the weight of the kth model; The total loss L that needs to be minimized is defined as follows: L = L ens (φ, p) + L ada (φ).

3. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the domain generalization method based on model sample matching according to claim 1 or 2 when executing the program.

4. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the steps of the domain generalization method based on model sample matching according to claim 1 or 2 when executed by the processor.

Images