Multi-source-domain depth migration image recognition method and system based on evidence theory

Abstract

The invention provides a multi-source domain depth migration image identification method and system based on an evidence theory, and the method comprises the steps: inputting images of a plurality of source domains and a target domain through an initialization module, and setting a loss ratio hyper-parameter, an iteration round number, an uncertainty coefficient and a screening threshold value; then, a transfer learning module executes transfer classification for each source domain-target domain pair, and further performs training to obtain a trained single-source domain transfer recognition model; then, an evidence fusion module converts a result of single-source domain migration identification into basic trust assignments (BBA) through the Dirichlet distribution, and fuses the BBAs by adopting an evidence theory so as to determine the category of a target domain sample; and finally, a pseudo label generation training module generates a high-confidence-coefficient pseudo label according to a fusion result, and the model is optimized through multi-round iteration. According to the method, the inter-domain distribution difference can be effectively reduced, the complementary feature information of each source domain is fully utilized, and the accuracy and robustness of target domain image recognition are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a multi-source domain deep migration image recognition method and system based on evidence theory. Background Art

[0002] In multi-sensor target recognition tasks, the information provided by different sensors is complementary. Fusion of this information helps build more accurate target classification models. Leveraging this advantage, multi-sensor fusion technology has been widely used in fields such as intelligent security and autonomous driving. It can more comprehensively capture target features and effectively improve recognition accuracy in complex scenarios.

[0003] However, in transfer learning scenarios, due to the differences in the physical properties of each sensor and the environments in which it is located, the collected data often has significant distribution differences. This data can be divided into multiple source domains. Despite the heterogeneity between domains, these source domain data still contain discriminative information that is valuable for modeling the target domain. In other words, different source domains complement each other in terms of feature representation. If these cross-domain features can be effectively integrated, it will help significantly improve the recognition performance of the target domain.

[0004] However, current multi-source domain transfer recognition methods generally adopt a centralized fusion strategy, which involves jointly modeling multiple source domain data by constructing a unified objective function to extract globally shared, domain-invariant features. This approach has certain limitations when fusing multi-source domain information at the feature level: due to the large distribution differences between source domains, unified optimization may cause inter-domain interference, which in turn weakens or even loses some useful feature information during the fusion process. This information loss limits the model's ability to fully exploit complementary features in the source domains, hindering further improvements in recognition performance. In particular, in image-based transfer recognition tasks, how to effectively mitigate inter-domain distribution differences and maximize the exploitation of complementary characteristics between source domains becomes a key challenge in improving target recognition accuracy.

[0005] A search of patent literature revealed an invention patent with publication number CN117828520A, which discloses a variable-condition rolling bearing fault diagnosis method based on multi-source domain deep transfer. The method calculates weights based on the correlation between the source and target domains, constructs a highly correlated auxiliary source domain based on the weights, and designs a feature extractor to isolate interference from low-correlation source domain information. The classifier outputs are aligned using the Wasserstein distance, and a weighted decision model is constructed using this method. To address the issue of data distribution misalignment, the method leverages the varying reliability of pseudo-labels in the early and late stages of model training. The MMD distance is used to align the feature distributions of the source and target domains in the early stages of model training. Highly reliable pseudo-labels are generated in the later stages of training, and the LMMD distance and triplet loss are introduced to achieve finer-grained alignment of feature distributions. This patent utilizes only weighted decision making, lacking uncertainty management. Pseudo-label generation relies solely on classifier outputs, without incorporating multiple rounds of iterative optimization and confidence screening, potentially impacting label reliability.

[0006] To sum up, in response to the above-mentioned problems of the existing technology, studying a multi-source domain deep migration image recognition method and system based on evidence theory has become a key task that needs to be solved urgently. Summary of the Invention

[0007] In view of the defects in the prior art, the purpose of the present invention is to provide a multi-source domain deep migration image recognition method and system based on evidence theory.

[0008] According to the present invention, a multi-source domain deep migration image recognition method based on evidence theory includes the following steps:

[0009] Step S1: Input multiple source and target domain images and set hyperparameters, including loss ratio hyperparameter, number of iterations, uncertainty coefficient, and screening threshold.

[0010] Step S2: Based on the loss ratio hyperparameter, transfer learning classification is performed on each source domain-target domain pair to obtain multiple pre-trained single-source domain transfer recognition models;

[0011] In step S3, based on the uncertainty coefficient, the prediction results of each pre-trained single-source domain transfer recognition model are converted into BBA through Dirichlet distribution, and fused using evidence theory to obtain the BBA and category of the target domain sample, which are then converted back into probability, i.e., the final prediction probability;

[0012] In step S4, a pseudo label is generated for the target domain based on the final prediction probability, and a high-confidence pseudo label is obtained by screening according to the screening threshold. The model is trained and the pseudo label generation and training are repeated until the number of iterations is reached to complete the iterative optimization.

[0013] Preferably, step S1 includes the following sub-steps:

[0014] Step S1.1, input P source domains are defined as The corresponding source domain dataset is

[0015] The target domain is defined as D T ={χ T ,P(X T )}, the corresponding target domain dataset is The feature spaces of the source domain and the target domain are consistent, that is, But the data distribution is different, i.e.

[0016] And set the category space of the target recognition task to Ω={ω1,ω2,…,ω K};

[0017] in:

[0018] represents the source domain;

[0019] p represents the p-th source domain;

[0020] S p represents the p-th source domain dataset;

[0021] Represents the feature space of the p-th source domain dataset;

[0022] represents the sample set of the p-th source domain;

[0023] Represents the features of the i-th sample of the p-th source domain dataset;

[0024] represents the label of the i-th sample of the p-th source domain dataset;

[0025] Represents the number of samples in the p-th source domain dataset;

[0026] P(·) represents the distribution of data;

[0027] D T represents the target domain;

[0028] χ T Represent the feature space of the target domain dataset;

[0029] N T Indicates the number of samples in the target domain dataset;

[0030] X T Represents the sample set of the target domain dataset;

[0031] Represents the jth sample of the target domain dataset;

[0032] ω k represents the kth category;

[0033] K represents the total number of categories;

[0034] Step S1.2: Set hyperparameters. Hyperparameters include:

[0035] Loss ratio hyperparameter used to balance the weights of classification cross entropy loss and local maximum mean difference loss in the total loss function.

[0036] The number of iterations is used to indicate the number of iterations of pseudo-label screening and model training.

[0037] Uncertainty coefficient, used to adjust the degree of uncertainty when converting the predicted probability into the parameters of the Dirichlet distribution.

[0038] The screening threshold is used to filter out pseudo-label samples with high confidence, and retain pseudo-label samples with prediction probabilities higher than the screening threshold for subsequent model training.

[0039] Preferably, step S2 includes the following sub-steps:

[0040] In step S2.1, according to the K categories of the object recognition task, the source domain and the target domain are divided into K corresponding category sub-domains. For each category sub-domain, the local maximum average difference loss term between the source domain and the target domain in the reproducing kernel Hilbert space is calculated. The local maximum average difference loss term is defined as the average of the sum of the maximum average differences over all category sub-domains, that is:

[0041]

[0042] in:

[0043] represents the local maximum average difference loss term, which is the average of the sum of the maximum average differences between the source domain data and the target domain data in each category sub-domain;

[0044] m represents the number of samples in the p-th source domain dataset;

[0045] n represents the number of samples in the p-th target domain dataset;

[0046] represents the i-th sample of the p-th source domain;

[0047] represents the jth sample in the target domain;

[0048] f(·) represents the function that transforms from the feature space to the reproducing kernel Hilbert space;

[0049] Represents the weight assigned to the k-th category on the p-th source domain, through Calculation, which represents the fraction of samples belonging to the category among all samples;

[0050] Indicates the label corresponding to the k-th category in the p-th source domain, with a value of 0 or 1;

[0051] Represents the weight assigned to the kth category in the target domain, through calculate;

[0052] Represents the label corresponding to the kth category in the target domain;

[0053] ||·|| means calculating the norm, ||·|| 2 is the square of the norm.

[0054] Step S2.2: For source domain data, calculate the classification cross entropy loss between the model prediction results and the true labels, that is:

[0055]

[0056] in:

[0057] y k Represents the true label of the sample (using one-hot encoding);

[0058] represents the predicted probability distribution of the single-source domain transfer recognition model for sample x;

[0059] This loss term ensures that the model accurately classifies the source domain.

[0060] represents y k and The cross entropy between .

[0061] N represents the number of samples in the corresponding source domain;

[0062] Step S2.3, based on the weighted combination of the local maximum average difference loss term of step S2.1 and the classification cross entropy loss of step S2.2, the total loss function is obtained in, They represent the classification cross entropy loss term and the local maximum average difference loss term respectively; λ represents the loss ratio hyperparameter that adjusts the ratio of the two loss terms, and minimizes the total loss function to train the single-source domain transfer recognition model.

[0063] After the training is completed, the pre-trained single-source domain transfer recognition model can recognize any sample in the target domain. The output prediction result is expressed as That is, the prediction result of the p-th pre-trained single-source domain transfer recognition model is used for the i-th sample in the target domain dataset. The prediction result is expressed as a vector of the number of categories, which is expressed as

[0064] in: Represents the value of the kth dimension of the vector, that is, the predicted probability of the sample belonging to the k+1th category by the pth pre-trained single-source domain transfer recognition model.

[0065] Preferably, step S3 includes the following sub-steps:

[0066] Step S3.1: Convert the prediction results of each pre-trained single-source domain transfer recognition model into BBA through Dirichlet distribution to obtain P pre-trained single-source domain transfer recognition models about the target domain samples. BBA.

[0067] Step S3.2: Use evidence theory combination rules to train P pre-trained single-source domain transfer recognition models on target domain samples. The BBAs are fused to obtain the final fused BBA and the final fused uncertainty.

[0068] In step S3.3, the final fused BBA and the final fused uncertainty are converted into the final prediction probability for each category.

[0069] Preferably, step S3.1 includes: setting the parameters of the Dirichlet distribution

[0070] in Represents the predicted probability of the kth class;

[0071] β represents the parameter that adjusts the proportion of uncertainty in the evidence theory, that is, the uncertainty coefficient.

[0072] Set each category to true BBA: This means that the sample belongs to the BBA of the k+1th class;

[0073] Where S represents the normalized coefficient, and the calculation formula is

[0074] Setting uncertainty Represents the BBA that is uncertain to which category the sample belongs.

[0075] Therefore, the predicted probability vector of the p-th pre-trained single-source domain transfer recognition model is Converted into a set of BBA values ​​{b0,b1,…,b K-1 ,u}.

[0076] Preferably, step S3.2 includes the following sub-steps:

[0077] Step S3.2.1, calculate the conflict coefficient

[0078] in:

[0079] m(·) represents the function for calculating the corresponding BBA;

[0080] m p (·) represents the function of calculating the corresponding BBA of the p-th pre-trained single-source domain transfer recognition model;

[0081] m q (·) represents the function of calculating the corresponding BBA of the qth pre-trained single-source domain transfer recognition model;

[0082] Indicates that the intersection of subset A and subset B in the identification framework is empty.

[0083] A and B represent two subsets in the recognition framework, and their intersection needs to be an empty set. In the scenario defined here, we only need to and Just multiply and sum the corresponding numbers.

[0084] Represents the value of the i-th dimension of the BBA corresponding to the p-th pre-trained single-source domain transfer recognition model; Represents the value of the j-th dimension of the BBA corresponding to the q-th pre-trained single-source domain transfer recognition model.

[0085] Step S3.2.2, calculate the normalization constant That is, 1 minus the conflict coefficient.

[0086] Step S3.2.3, for the p-th source domain and the q-th source domain, calculate the BBA after normalization constant fusion: Among them, k represents the kth dimension of the fused BBA, u q represents the uncertainty corresponding to the qth pre-trained single-source domain transfer recognition model, u p Represents the uncertainty corresponding to the p-th pre-trained single-source domain transfer recognition model.

[0087] Calculate the uncertainty after normalization constant fusion

[0088] Preferably, step S3.3 includes:

[0089] The final fusion uncertainty Assign to each category according to uniform distribution, that is, assuming that in the absence of other information, the prior probability of the sample belonging to each category is Get the target sample The final posterior probability of belonging to the k+1th class Get the predicted probability of each category, and the final predicted probability vector is

[0090] Preferably, step S4 includes the following sub-steps:

[0091] Step S4.1, for each sample in the target domain According to the final predicted probability vector of the sample Take the highest dimension in the final predicted probability vector, that is, the highest predicted probability Get the predicted category k.

[0092] The highest predicted probability Compared with the set screening threshold t, if It is considered that the model has a high confidence in the prediction of the sample, and a high confidence pseudo label in the form of one-hot encoding is generated, which is expressed as Only the kth dimension is 1, and the rest are 0.

[0093] Among them, the screening threshold t is a hyperparameter and is set before training;

[0094] High confidence pseudo labels obtained based on predictions The model is trained using cross entropy loss.

[0095] In step S4.2, all target domain samples with high-confidence pseudo labels are collected to form a training subset, and the model is trained using the training subset by minimizing the cross-entropy loss.

[0096] The cross entropy loss consists of:

[0097] The overall cross entropy loss measures the cross entropy loss between the final prediction result of the model and the high-confidence pseudo-label, expressed as:

[0098]

[0099] in:

[0100] Represents the model's predicted probability that the i-th target domain dataset sample belongs to the k+1-th category;

[0101] Represents the value of the model's k+1th dimension of the pseudo label for the i-th target domain dataset sample;

[0102] T represents the target domain.

[0103] The single-source cross-entropy loss measures the cross-entropy loss between the independent prediction results of the pre-trained single-source domain transfer recognition model and the high-confidence pseudo-label. For the p-th source domain, it is expressed as:

[0104]

[0105] in:

[0106] represents the predicted probability of the p-th pre-trained single-source domain transfer recognition model for the i-th target domain dataset sample belonging to the k+1-th class;

[0107] Represents the value of the k+1th dimension of the pseudo label.

[0108] Combining the overall cross entropy loss and the single-source cross entropy loss, we get the total loss expression: Minimize the total loss through optimization algorithms Update the model's trainable parameters, including the parameters of each pre-trained single-source domain transfer recognition model.

[0109] Step S4.3: Repeat steps S4.1 to S4.2, use the updated model to re-predict target domain samples, generate new high-confidence pseudo labels, and optimize the cross-entropy loss based on the new high-confidence pseudo labels until the preset number of iterations is reached or the model performance reaches saturation on the target domain, and the final optimization result is obtained.

[0110] The present invention also provides a multi-source domain deep migration image recognition system based on evidence theory, comprising:

[0111] The initialization module is used to input multiple source and target domain images and set hyperparameters, including the loss ratio hyperparameter, the number of iterations, the uncertainty coefficient, and the screening threshold.

[0112] The transfer learning module is used to perform transfer learning classification on each source domain-target domain pair based on hyperparameters to obtain multiple pre-trained single-source domain transfer recognition models.

[0113] The evidence fusion module converts the prediction results of each pre-trained single-source domain transfer recognition model into BBA based on the uncertainty coefficient through Dirichlet distribution, and fuses them using evidence theory to obtain the BBA and category of the target domain sample, and converts them back into probability, that is, the final prediction probability.

[0114] The pseudo-label generation training module generates pseudo-labels for the target domain based on the final prediction probability, and filters them according to the screening threshold to obtain high-confidence pseudo-labels, trains the model, and passes multiple rounds of training.

[0115] Repeat pseudo-label generation and training for iterative optimization.

[0116] Preferably, the source domain includes a plurality of source data, and the source data includes images and labels corresponding to the images; the target domain includes target data, and the target data includes images.

[0117] Compared with the prior art, the present invention has the following beneficial effects:

[0118] 1. This paper adopts evidence theory to fuse the migration recognition results of multiple source domains, effectively alleviating the interference problem caused by the distribution differences between domains.

[0119] 2. This paper converts the prediction results into basic trust assignment representation by introducing Dirichlet distribution and combines uncertainty information to make the model more robust when processing target domain samples.

[0120] 3. The present invention adopts pseudo-label screening and self-training mechanism to gradually improve the recognition accuracy of target domain samples and enhance the generalization ability of the model.

[0121] 4. This paper proposes a distributed, multi-source fusion transfer learning architecture, which can better preserve the discriminative features of each source domain and improve the overall recognition performance compared with the traditional centralized method. BRIEF DESCRIPTION OF THE DRAWINGS

[0122] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0123] Figure 1 Schematic diagram of the process of multi-source domain deep migration image recognition method based on evidence theory in an embodiment of the present invention;

[0124] Figure 2 Schematic diagram of a multi-source domain deep migration image recognition system based on evidence theory in an embodiment of the present invention;

[0125] Figure 3 Schematic diagram of a sample Office-Home dataset in an embodiment of the present invention. DETAILED DESCRIPTION

[0126] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0127] The present invention provides a multi-source domain deep migration image recognition method and system based on evidence theory. It considers the migration learning classification problem in which data from multiple source domains and target domains are in the same feature space but have different data distributions. It improves the accuracy of feature alignment through multiple migration learning modules, and uses evidence theory to fuse complementary information from different source domains. Finally, it uses pseudo-label generation technology to improve the accuracy and obtain accurate migration target recognition results. Specifically, it includes: inputting images from multiple source domains and one target domain through an initialization module, and setting the loss ratio hyperparameter, number of iterations, uncertainty coefficient and screening threshold at the same time; then, the migration learning module performs migration classification for each source domain-target domain pair, and then trains to obtain a trained single-source domain migration recognition model; then, the evidence fusion module converts the results of single-source domain migration recognition into basic belief assignment (BBA) through Dirichlet distribution, and then fuses these BBAs using evidence theory to determine the category of the target domain sample; finally, the pseudo-label generation training module generates high-confidence pseudo-labels based on the fusion results, and optimizes the model through multiple rounds of iterations. The present invention can effectively reduce the distribution differences between domains, make full use of the complementary feature information of each source domain, and significantly improve the accuracy and robustness of target domain image recognition.

[0128] Example 1:

[0129] Figure 1 Schematic diagram of the process of multi-source domain deep migration image recognition method based on evidence theory in an embodiment of the present invention.

[0130] like Figure 1 As shown, this embodiment provides a multi-source domain deep migration image recognition method based on evidence theory, including the following steps:

[0131] In step S1, multiple source domain images and one target domain are input, and hyperparameters are set. The hyperparameters include the loss ratio hyperparameter, the number of iterations, the uncertainty coefficient, and the screening threshold.

[0132] In this embodiment, multiple source domain images and labels and one target domain image are input, the above data are preprocessed, and the hyperparameters required for the target recognition task are initialized and configured; the source domain includes multiple source domain data, and the source domain data includes images and labels corresponding to the images; the target domain includes multiple target domain data, and the target domain data includes images.

[0133] Specifically, step S1 includes the following sub-steps:

[0134] Step S1.1, input P source domains are defined as The corresponding source domain dataset is

[0135] The target domain is defined as D T ={χ T ,P(X T )}, the corresponding target domain dataset is

[0136] It is worth noting that the target domain data only contains image samples but no category labels in the initial stage.

[0137] Assume that the feature spaces of all source and target domain images are consistent, that is, However, due to the existence of domain differences, their data distributions are different, i.e. And set the category space of the target recognition task to Ω={ω1,ω2,…,ω K};

[0138] in:

[0139] represents the source domain;

[0140] p represents the p-th source domain;

[0141] S p represents the p-th source domain dataset;

[0142] Represents the feature space of the p-th source domain dataset;

[0143] represents the sample set of the p-th source domain;

[0144] Represents the features of the i-th sample of the p-th source domain dataset;

[0145] represents the label of the i-th sample of the p-th source domain dataset;

[0146] Represents the number of samples in the p-th source domain dataset;

[0147] P(·) represents the distribution of data;

[0148] D T represents the target domain;

[0149] T represents the target domain dataset;

[0150] χ TRepresent the feature space of the target domain dataset;

[0151] N T Indicates the number of samples in the target domain dataset;

[0152] X T Represents the sample set of the target domain dataset;

[0153] Represents the jth sample of the target domain dataset;

[0154] ω k represents the kth category;

[0155] K represents the total number of categories;

[0156] Ω represents the category space;

[0157] Step S1.2: Set hyperparameters. These hyperparameters will guide the subsequent training and fusion processes. The hyperparameters include:

[0158] Loss ratio hyperparameter used to balance the weights of classification cross entropy loss and local maximum mean difference loss in the total loss function.

[0159] The number of iterations is used to indicate the number of iterations of pseudo-label screening and model training. Through multiple rounds of iterative optimization, the performance of the model in the target domain is gradually improved.

[0160] Uncertainty coefficient, used to adjust the degree of uncertainty when converting the predicted probability into the parameters of the Dirichlet distribution.

[0161] The screening threshold is used to filter out pseudo-label samples with high confidence, and retain pseudo-label samples with prediction probabilities higher than the screening threshold for subsequent model training.

[0162] In step S2, based on the loss ratio hyperparameter, transfer learning classification is performed on each source domain-target domain pair to obtain multiple pre-trained single-source domain transfer recognition models.

[0163] In this embodiment, transfer learning is performed separately for each source domain-target domain pair, thereby obtaining P pre-trained single-source domain transfer recognition models. Each pre-trained single-source domain transfer recognition model is responsible for transferring knowledge from a specific source domain to the target domain.

[0164] Specifically, for the p-th source domain-target domain pair, the single-source domain transfer recognition model is trained through the following sub-steps (taking the p-th source domain as an example). Step S2 includes the following sub-steps:

[0165] In step S2.1, based on the category space in the source domain, set K category subdomains, where K is the number of categories. For each category subdomain, construct a local maximum average difference loss term, and combine the local maximum average difference loss terms of each category subdomain.

[0166] Specifically, according to the K categories of the object recognition task, the source domain and the target domain are divided into K corresponding category sub-domains. For each category sub-domain, the local maximum average difference loss term between the source domain and the target domain in the reproducing kernel Hilbert space is calculated. The local maximum average difference loss term is defined as the average of the sum of the maximum average differences over all category sub-domains, that is:

[0167]

[0168] in:

[0169] represents the local maximum average difference loss term, which is the average of the sum of the maximum average differences between the source domain data and the target domain data in each category sub-domain;

[0170] m represents the number of samples in the p-th source domain dataset;

[0171] n represents the number of samples in the p-th target domain dataset;

[0172] represents the i-th sample of the p-th source domain;

[0173] represents the jth sample in the target domain;

[0174] f(·) represents the function that transforms from the feature space to the reproducing kernel Hilbert space;

[0175] Represents the weight assigned to the k-th category on the p-th source domain, through Calculation, which represents the fraction of samples belonging to the category among all samples;

[0176] Indicates the label corresponding to the k-th category in the p-th source domain, with a value of 0 or 1;

[0177] Represents the weight assigned to the kth category in the target domain, through calculate;

[0178] Represents the label corresponding to the kth category in the target domain.

[0179] Since there are no true labels in the target domain initially, this part of the weight is calculated based on the soft labels (i.e., predicted probabilities) predicted by the model for the target domain samples. This part of the label is replaced by the soft label technology, that is, the probability of each category predicted by the single-source domain transfer recognition model for the target domain.

[0180] ||·|| means calculating the norm, ||·|| 2 is the square of the norm.

[0181] In step S2.2, the classification cross entropy loss term of the single-source domain transfer recognition model is set according to the difference between the prediction result of the single-source domain transfer recognition model and the true label.

[0182] Specifically, for the source domain, since it has a true label, the classification cross entropy loss between the model prediction result and the true label is directly calculated, that is:

[0183]

[0184] in:

[0185] represents the classification cross entropy loss;

[0186] y k Represents the true label of the sample (using one-hot encoding);

[0187] represents the predicted probability distribution of the single-source domain transfer recognition model for sample x;

[0188] This loss term ensures that the model accurately classifies the source domain;

[0189] represents y k and The cross entropy between .

[0190] N represents the number of samples in the corresponding source domain.

[0191] Step S2.3, perform a weighted combination of the local maximum average difference loss term of step S2.1 and the classification cross entropy loss of step S2.2 to obtain the total loss function in, They represent the classification cross entropy loss term and the local maximum average difference loss term respectively; λ represents the hyperparameter that adjusts the proportion of the two loss terms, and the total loss function is minimized to train the single-source domain transfer recognition model.

[0192] After the training is completed, the pre-trained single-source domain transfer recognition model can recognize any sample in the target domain. Output a prediction result expressed as That is, the prediction result of the p-th pre-trained single-source domain transfer recognition model is used for the i-th sample in the target domain dataset. The prediction result is expressed as a probability vector of the number of categories (K dimensions), which is expressed as

[0193] in Represents the value of the kth dimension of the vector, that is, the predicted probability of the sample belonging to the k+1th category by the pth pre-trained single-source domain transfer recognition model.

[0194] In step S3, based on the uncertainty coefficient, the prediction results of each pre-trained single-source domain transfer recognition model are converted into BBA (Basic Belief Assignment) through Dirichlet distribution, and fused using evidence theory to obtain the BBA and category of the target domain sample, and then converted back to probability, that is, the final prediction probability.

[0195] In this embodiment, after obtaining independent predictions of the target domain samples from P pre-trained single-source domain transfer recognition models, evidence theory (also known as Dempster-Shafer evidence theory) is used to fuse these prediction results from different information sources in order to obtain a more reliable and robust final recognition result.

[0196] Step S3 includes three sub-steps: converting to BBA through Dirichlet distribution, combining BBAs corresponding to multiple source domains into predicted BBA, and converting predicted BBA back to predicted probability.

[0197] Specifically, step S3 includes the following sub-steps:

[0198] Step S3.1: Convert the prediction results of each pre-trained single-source domain transfer recognition model into BBA (Basic Trust Assignment) through Dirichlet distribution to obtain the corresponding P pre-trained single-source domain transfer recognition models about the target domain samples. BBA.

[0199] For a target domain sample Its predicted probability vector from the p-th pre-trained single-source domain transfer recognition model is In order to transform this probability information into BBA in belief function theory, it is necessary to use Dirichlet distribution.

[0200] Furthermore, step S3.1 includes setting the parameters of the Dirichlet distribution

[0201] in It represents the predicted probability of the kth class. Since it has nothing to do with which domain it is, the superscript p is omitted;

[0202] β represents the parameter that adjusts the proportion of uncertainty in the theory of evidence, i.e., the uncertainty coefficient. A smaller β value means that a greater weight is given to the probability of the model prediction, with less uncertainty, thus making the BBA more concentrated.

[0203] Set each category to true BBA: This means that the sample belongs to the BBA of the k+1th class;

[0204] Where S represents the normalized coefficient, and the calculation formula is

[0205] Setting uncertainty The BBA represents the category to which the uncertain sample belongs, that is, the part that cannot be assigned to a certain category due to incomplete information.

[0206] Therefore, the predicted probability vector of the p-th pre-trained single-source domain transfer recognition model is Converted into a set of BBA values ​​{b0,b1,…,b K-1 ,u}.

[0207] In step S3.2, the BBA of target domain samples from each pre-trained single-source domain transfer recognition model is fused using the evidence theory fusion rule to obtain the total BBA.

[0208] Specifically, the evidence theory combination rule is used to train the P pre-trained single source domain transfer recognition models on the target domain samples. The BBAs are fused to obtain the final fused BBA and the final fused uncertainty.

[0209] To fuse two sources (e.g., BBA from the p-th and q-th pre-trained single-source domain transfer recognition models: m p and m q ) as an example, step S3.2 includes the following sub-steps:

[0210] Step S3.2.1, calculate the conflict coefficient

[0211] in:

[0212] m(·) represents the function for calculating the corresponding BBA;

[0213] m p (·) represents the function of calculating the corresponding BBA of the p-th pre-trained single-source domain transfer recognition model;

[0214] m q (·) represents the function of calculating the corresponding BBA of the qth pre-trained single-source domain transfer recognition model;

[0215] Indicates that the intersection of subset A and subset B in the identification framework is empty.

[0216] A and B represent two subsets in the recognition framework, and their intersection needs to be an empty set. In the scenario defined here, we only need to and Just multiply and sum the corresponding numbers.

[0217] Represents the value of the i-th dimension of the BBA corresponding to the p-th pre-trained single-source domain transfer recognition model;

[0218] Represents the value of the j-th dimension of the BBA corresponding to the q-th pre-trained single-source domain transfer recognition model.

[0219] Step S3.2.2, calculate the normalization constant That is, 1 minus the conflict coefficient.

[0220] Step S3.2.3, for the p-th source domain and the q-th source domain, calculate the BBA after normalization constant fusion: Among them, k represents the kth dimension of the fused BBA, u q represents the uncertainty corresponding to the qth pre-trained single-source domain transfer recognition model, u p Represents the uncertainty corresponding to the p-th pre-trained single-source domain transfer recognition model.

[0221] Calculate the uncertainty after normalization constant fusion

[0222] The above steps yield the fusion result of two BBAs and uncertainty. However, for multi-source domain problems, the BBAs need to be fused pairwise to obtain the final result. In other words, for multiple source domains, the corresponding BBAs need to be iteratively fused pairwise.

[0223] Specifically, for the case of multiple source domains, the BBAs of the first and second pre-trained single-source domain transfer recognition models are fused to obtain an intermediate fusion result, and the intermediate fusion result is fused with the BBA of the third pre-trained single-source domain transfer recognition model, and so on, until the BBAs of all P source domains are fused, and the final fusion result is obtained, that is, the final fusion and the final fusion uncertainty The formula is expressed as the following calculation steps:

[0224]

[0225] in:

[0226] Represents the intermediate variable and the intermediate fusion result.

[0227] u 1 Represents the uncertainty corresponding to the first pre-trained single-source domain transfer recognition model;

[0228] u i+1 represents the uncertainty corresponding to the i+1th pre-trained single-source domain transfer recognition model;

[0229] represents the uncertainty corresponding to the i-1th intermediate variable;

[0230] u 2 Represents the uncertainty corresponding to the second pre-trained single-source domain transfer recognition model;

[0231] represents the kth dimension of the BBA corresponding to the first pre-trained single-source domain transfer recognition model;

[0232] represents the kth dimension of the BBA corresponding to the second pre-trained single-source domain transfer recognition model;

[0233] represents the kth dimension of the BBA corresponding to the Pth pre-trained single-source domain transfer recognition model;

[0234] It represents the kth dimension of the BBA corresponding to the single-source domain transfer recognition model after the i+1th pre-training.

[0235] Indicates the final fusion of BBA.

[0236] In step S3.3, the final fused BBA and final fused uncertainty obtained in step S3.2 are converted back into prediction probability.

[0237] In this embodiment, the final fused BBA and the final fused uncertainty are converted into the final prediction probability of each category to facilitate decision making.

[0238] Specifically, step S3.3 includes:

[0239] The final fusion uncertainty According to a certain prior probability distribution (such as uniform distribution), it is assigned to each category. That is, it is assumed that in the absence of other information, the prior probability of the sample belonging to each category is Get the target sample The final posterior probability of belonging to the k+1th class Get the predicted probability of each category, and the final predicted probability vector is

[0240] In step S4, a pseudo label is generated for the target domain based on the final prediction probability, and a high-confidence pseudo label is obtained by screening according to the screening threshold. The model is trained and the pseudo label generation and training are repeated until the number of iterations is reached to complete the iterative optimization.

[0241] In this embodiment, pseudo labels are generated for some unlabeled samples in the target domain based on the categories of the target domain samples, and these pseudo-labeled samples are used for further iterative optimization and fine-tuning. Specifically, threshold screening is performed based on the categories of the target domain samples obtained in step S3 to select samples with high-confidence labels, and these samples are used for labeled training.

[0242] Specifically, step S4 includes the following sub-steps:

[0243] Step S4.1, for each sample in the target domain According to the final predicted probability vector of the sample Take the highest dimension in the final predicted probability vector, that is, the highest predicted probability Get the predicted category k.

[0244] The highest predicted probability Compared with the set screening threshold t, if It is considered that the model has a high confidence in the prediction of the sample, and a high confidence pseudo label in the form of one-hot encoding is generated, which is expressed as Only the kth dimension is 1, and the rest of the dimensions are 0.

[0245] Among them, the screening threshold t is a hyperparameter and is set before training;

[0246] High confidence pseudo labels obtained based on predictions The model is trained using cross entropy loss.

[0247] In step S4.2, all target domain samples with high-confidence pseudo labels are collected to form a training subset, and the model is trained using the training subset by minimizing the cross-entropy loss.

[0248] The cross entropy loss consists of:

[0249] The overall cross entropy loss measures the cross entropy loss between the final prediction result of the model and the high-confidence pseudo-label, expressed as:

[0250]

[0251] in:

[0252] represents the overall cross entropy loss;

[0253] Represents the model's predicted probability that the i-th target domain dataset sample belongs to the k+1-th class;

[0254] Represents the value of the model's k+1th dimension of the pseudo label for the i-th target domain dataset sample;

[0255] T represents the target domain.

[0256] The single-source cross-entropy loss measures the cross-entropy loss between the independent prediction results of the pre-trained single-source domain transfer recognition model and the high-confidence pseudo-label. For the p-th source domain, it is expressed as:

[0257]

[0258] in:

[0259] represents the single-source cross entropy loss;

[0260] represents the predicted probability of the p-th pre-trained single-source domain transfer recognition model for the i-th target domain dataset sample belonging to the k+1-th class;

[0261] Represents the value of the k+1th dimension of the pseudo label.

[0262] Combining the overall cross entropy loss and the single-source cross entropy loss, we get the total loss expression: Minimize the total loss through optimization algorithms Update the model's trainable parameters, including the parameters of each pre-trained single-source domain transfer recognition model.

[0263] Step S4.3: Repeat steps S4.1 to S4.2, use the updated model to re-predict target domain samples, generate new high-confidence pseudo labels, and optimize the cross-entropy loss based on the new high-confidence pseudo labels until the preset number of iterations is reached or the model performance reaches saturation on the target domain, and the final optimization result is obtained.

[0264] Through multiple rounds of such iterations, the performance of the model in the target domain gradually improves, and the quality of pseudo-labels also improves accordingly, forming a positive self-learning cycle, thereby guiding the model to learn more robust and accurate discriminant features, and ultimately perform well in the target domain.

[0265] Example 2:

[0266] Figure 3 Schematic diagram of a sample Office-Home dataset in an embodiment of the present invention.

[0267] by Figure 3 As a preferred example, the Office-Home dataset was proposed by researchers at Arizona State University in 2017. The samples in the dataset are all pictures of common items in homes and offices. The data includes four fields: Art, Clipart, Product, and RealWorld, and 65 categories such as Alarm Clock, with a total of 15,588 picture samples. Set up migration from three source domains to one target domain, and construct four multi-source domain migration target recognition tasks: {Art, Clipart, Product}→Real World, {Art, Clipart, Real World}→Product, {Art, Product, Real World}→Clipart, {Clipart, Product, Real World}→Art. Example samples of the target recognition dataset are as follows: Figure 3 shown.

[0268] We selected recent state-of-the-art methods, including MFSAN, DJSAN, MIEM, MJD, and CSR, and compared them with the method proposed in this patent. We used the classification accuracy of target domain samples as the algorithm evaluation metric. The test results are shown in Table 1, where each source domain is represented by its initial letter.

[0269] Table 1 Accuracy of different methods on the Office-Home dataset (%)

[0270]

[0271] For the image data from the Office-Home dataset, this method uses evidence theory to fuse complementary information from different source domains, successfully learning knowledge from each source domain. It also utilizes a multi-round pseudo-label generation strategy to further improve object recognition performance. Compared to other single-source and multi-source transfer recognition algorithms, this method achieves the highest accuracy. The results demonstrate that using distributed transfer across source domains and fusing them using evidence theory is an effective approach for accurate object recognition and holds significant application value.

[0272] Example 3:

[0273] The present invention also provides a multi-source domain deep migration image recognition system based on evidence theory. The multi-source domain deep migration image recognition system based on evidence theory is realized by executing the process steps of the multi-source domain deep migration image recognition method based on evidence theory. That is, those skilled in the art can understand the multi-source domain deep migration image recognition method based on evidence theory as a preferred implementation of the multi-source domain deep migration image recognition system based on evidence theory.

[0274] Figure 2Schematic diagram of a multi-source domain deep migration image recognition system based on evidence theory in an embodiment of the present invention.

[0275] like Figure 2 As shown, the multi-source domain deep migration image recognition system based on evidence theory in an embodiment of the present invention includes:

[0276] The initialization module is used to input image data from multiple source and target domains and set hyperparameters, including the loss ratio hyperparameter, number of iterations, uncertainty coefficient, and screening threshold.

[0277] The transfer learning module is used to perform transfer learning classification on each source domain-target domain pair based on hyperparameters to obtain multiple pre-trained single-source domain transfer recognition models.

[0278] The evidence fusion module converts the prediction results of each pre-trained single-source domain transfer recognition model into BBA (Basic Belief Assignment) based on the uncertainty coefficient through Dirichlet distribution, and fuses them using evidence theory to obtain the BBA and category of the target domain sample, and converts them back into probability, that is, the final prediction probability.

[0279] The pseudo-label generation and training module generates pseudo-labels for the target domain based on the final prediction probability, and filters them according to the screening threshold to obtain high-confidence pseudo-labels. The model is trained and iterative optimization is performed through multiple rounds of repeated pseudo-label generation and training.

[0280] Specifically, the source domain includes multiple source data, and the source data includes images and labels corresponding to the images; the target domain includes target data, and the target data includes images.

[0281] The transfer learning module includes single-source transfer learning units with the same number as the number of source domains P, that is, the system contains P parallel single-source transfer learning units. Each single-source transfer learning unit corresponds to a source domain-target domain pair and is responsible for transferring knowledge from a specific source domain to the target domain, thereby obtaining P pre-trained single-source domain transfer recognition models. Each pre-trained single-source domain transfer recognition model traverses all images in the target domain to obtain the corresponding prediction results. The prediction results are input into the evidence fusion module to obtain the categories of all target domain samples.

[0282] Specifically, module M1 includes the following submodules:

[0283] Module M1.1, input P source domains are defined as The corresponding source domain dataset is

[0284] The target domain is defined as D T ={χ T ,P(X T )}, the corresponding target domain dataset is

[0285] It is worth noting that the target domain data only contains image samples but no category labels in the initial stage.

[0286] Assume that the feature spaces of all source and target domain images are consistent, that is, However, due to the existence of domain differences, their data distributions are different, i.e. And set the category space of the target recognition task to Ω={ω1,ω2,…,ω K};

[0287] in:

[0288] represents the source domain;

[0289] p represents the p-th source domain;

[0290] S p represents the p-th source domain dataset;

[0291] Represents the feature space of the p-th source domain dataset;

[0292] represents the sample set of the p-th source domain;

[0293] Represents the features of the i-th sample of the p-th source domain dataset;

[0294] represents the label of the i-th sample of the p-th source domain dataset;

[0295] Represents the number of samples in the p-th source domain dataset;

[0296] P(·) represents the distribution of data;

[0297] D T represents the target domain;

[0298] T represents the target domain dataset;

[0299] χ T Represent the feature space of the target domain dataset;

[0300] N T Indicates the number of samples in the target domain dataset;

[0301] X T Represents the sample set of the target domain dataset;

[0302] Represents the jth sample of the target domain dataset;

[0303] ω k represents the kth category;

[0304] K represents the total number of categories;

[0305] Ω represents the category space;

[0306] Module M1.2 sets hyperparameters, which will guide the subsequent training and fusion processes. Hyperparameters include:

[0307] Loss ratio hyperparameter used to balance the weights of classification cross entropy loss and local maximum mean difference loss in the total loss function.

[0308] The number of iterations is used to indicate the number of iterations of pseudo-label screening and model training. Through multiple rounds of iterative optimization, the performance of the model in the target domain is gradually improved.

[0309] Uncertainty coefficient, used to adjust the degree of uncertainty when converting the predicted probability into the parameters of the Dirichlet distribution.

[0310] The screening threshold is used to filter out pseudo-label samples with high confidence, and retain pseudo-label samples with prediction probabilities higher than the screening threshold for subsequent model training.

[0311] Module M2 performs transfer learning classification on each source-target domain pair based on the loss ratio hyperparameter, and obtains multiple pre-trained single-source domain transfer recognition models.

[0312] In this embodiment, transfer learning is performed separately for each source domain-target domain pair, thereby obtaining P pre-trained single-source domain transfer recognition models. Each pre-trained single-source domain transfer recognition model is responsible for transferring knowledge from a specific source domain to the target domain.

[0313] Specifically, for the p-th source domain-target domain pair, the single-source domain transfer recognition model is trained through the following submodules (taking the p-th source domain as an example). Module M2 includes the following submodules:

[0314] In module M2.1, we set K category subdomains based on the category space of the source domain, where K is the number of categories. We construct a local maximum average difference loss term for each category subdomain and combine the local maximum average difference loss terms of each category subdomain.

[0315] Specifically, according to the K categories of the object recognition task, the source domain and the target domain are divided into K corresponding category sub-domains. For each category sub-domain, the local maximum average difference loss term between the source domain and the target domain in the reproducing kernel Hilbert space is calculated. The local maximum average difference loss term is defined as the average of the sum of the maximum average differences over all category sub-domains, that is:

[0316]

[0317] in:

[0318] represents the local maximum average difference loss term, which is the average of the sum of the maximum average differences between the source domain data and the target domain data in each category sub-domain;

[0319] m represents the number of samples in the p-th source domain dataset;

[0320] n represents the number of samples in the p-th target domain dataset;

[0321] represents the i-th sample of the p-th source domain;

[0322] represents the jth sample in the target domain;

[0323] f(·) represents the function that transforms from the feature space to the reproducing kernel Hilbert space;

[0324] Represents the weight assigned to the k-th category on the p-th source domain, through Calculation, which represents the fraction of samples belonging to the category among all samples;

[0325] Indicates the label corresponding to the k-th category in the p-th source domain, with a value of 0 or 1;

[0326] Represents the weight assigned to the kth category in the target domain, through calculate;

[0327] Represents the label corresponding to the kth category in the target domain.

[0328] Since there are no true labels in the target domain initially, this part of the weight is calculated based on the soft labels (i.e., predicted probabilities) predicted by the model for the target domain samples. This part of the label is replaced by the soft label technology, that is, the probability of each category predicted by the single-source domain transfer recognition model for the target domain.

[0329] ||·|| means calculating the norm, ||·|| 2 is the square of the norm.

[0330] Module M2.2 sets the classification cross entropy loss term of the single-source domain transfer recognition model based on the difference between the prediction results of the single-source domain transfer recognition model and the true label.

[0331] Specifically, for the source domain, since it has a true label, the classification cross entropy loss between the model prediction result and the true label is directly calculated, that is:

[0332]

[0333] in:

[0334] represents the classification cross entropy loss;

[0335] y k Represents the true label of the sample (using one-hot encoding);

[0336] represents the predicted probability distribution of the single-source domain transfer recognition model for sample x;

[0337] This loss term ensures that the model accurately classifies the source domain.

[0338] represents y k and The cross entropy between .

[0339] N represents the number of samples in the corresponding source domain.

[0340] Module M2.3 performs a weighted combination of the local maximum average difference loss term of module M2.1 and the classification cross entropy loss of module M2.2 to obtain the total loss function in, They represent the classification cross entropy loss term and the local maximum average difference loss term respectively; λ represents the hyperparameter that adjusts the proportion of the two loss terms, and the total loss function is minimized to train the single-source domain transfer recognition model.

[0341] After the training is completed, the pre-trained single-source domain transfer recognition model can recognize any sample in the target domain. Output a prediction result expressed as That is, the prediction result of the p-th pre-trained single-source domain transfer recognition model is used for the i-th sample in the target domain dataset. The prediction result is expressed as a probability vector of the number of categories (K dimensions), which is expressed as

[0342] in Represents the value of the kth dimension of the vector, that is, the predicted probability of the sample belonging to the k+1th category by the pth pre-trained single-source domain transfer recognition model.

[0343] Specifically, module M3 includes the following submodules:

[0344] Module M3.1, through Dirichlet distribution, converts the prediction results of each pre-trained single-source domain transfer recognition model into BBA (basic trust assignment), and obtains the corresponding P pre-trained single-source domain transfer recognition models about the target domain samples. BBA.

[0345] For a target domain sample Its predicted probability vector from the p-th pre-trained single-source domain transfer recognition model is In order to transform this probability information into BBA in belief function theory, it is necessary to use Dirichlet distribution.

[0346] Furthermore, module M3.1 includes: setting the parameters of the Dirichlet distribution

[0347] in It represents the predicted probability of the kth class. Since it has nothing to do with which domain it is, the superscript p is omitted;

[0348] β represents the parameter that adjusts the proportion of uncertainty in the theory of evidence, i.e., the uncertainty coefficient. A smaller β value means that a greater weight is given to the probability of the model prediction, with less uncertainty, thus making the BBA more concentrated.

[0349] Set each category to true BBA: This means that the sample belongs to the BBA of the k+1th class;

[0350] Where S represents the normalized coefficient, and the calculation formula is

[0351] Setting uncertainty The BBA represents the category to which the uncertain sample belongs, that is, the part that cannot be assigned to a certain category due to incomplete information.

[0352] Therefore, the predicted probability vector of the p-th pre-trained single-source domain transfer recognition model is Converted into a set of BBA values ​​{b0,b1,…,b K-1 ,u}.

[0353] Module M3.2 uses the evidence theory fusion rule to fuse the BBA of each pre-trained single-source domain transfer recognition model on the target domain samples to obtain the total BBA.

[0354] Specifically, the evidence theory combination rule is used to train the P pre-trained single source domain transfer recognition models on the target domain samples. The BBAs are fused to obtain the final fused BBA and the final fused uncertainty.

[0355] To fuse two sources (e.g., BBA from the p-th and q-th pre-trained single-source domain transfer recognition models: m p and m q ) as an example, module M3.2 includes the following submodules:

[0356] Step S3.2.1, calculate the conflict coefficient

[0357] in:

[0358] m(·) represents the function for calculating the corresponding BBA;

[0359] m p (·) represents the function of calculating the corresponding BBA of the p-th pre-trained single-source domain transfer recognition model;

[0360] m q (·) represents the function of calculating the corresponding BBA of the qth pre-trained single-source domain transfer recognition model;

[0361] Indicates that the intersection of subset A and subset B in the identification framework is empty.

[0362] A and B represent two subsets in the recognition framework, and their intersection needs to be an empty set. In the scenario defined here, we only need to and Just multiply and sum the corresponding numbers.

[0363] Represents the value of the i-th dimension of the BBA corresponding to the p-th pre-trained single-source domain transfer recognition model;

[0364] Represents the value of the j-th dimension of the BBA corresponding to the q-th pre-trained single-source domain transfer recognition model.

[0365] Module M3.2.2, Calculating Normalization Constants That is, 1 minus the conflict coefficient.

[0366] Module M3.2.3, for the p-th source domain and the q-th source domain, calculate the BBA after the normalization constant fusion: Among them, k represents the kth dimension of the fused BBA, u q represents the uncertainty corresponding to the qth pre-trained single-source domain transfer recognition model, u p Represents the uncertainty corresponding to the p-th pre-trained single-source domain transfer recognition model.

[0367] Calculate the uncertainty after normalization constant fusion

[0368] The above modules can be used to obtain the fusion results of two BBAs and uncertainty. However, for multi-source domain problems, the BBAs need to be fused pairwise to obtain the final result. In other words, for multiple source domains, the corresponding BBAs need to be iteratively fused pairwise.

[0369] Specifically, for the case of multiple source domains, the BBAs of the first and second pre-trained single-source domain transfer recognition models are fused to obtain an intermediate fusion result, and the intermediate fusion result is fused with the BBA of the third pre-trained single-source domain transfer recognition model, and so on, until the BBAs of all P source domains are fused, and the final fusion result is obtained, that is, the final fusion and the final fusion uncertainty The formula is expressed as the following calculation module:

[0370]

[0371] in:

[0372] Represents the intermediate variable and the intermediate fusion result.

[0373] u 1 Represents the uncertainty corresponding to the first pre-trained single-source domain transfer recognition model;

[0374] u i+1 represents the uncertainty corresponding to the i+1th pre-trained single-source domain transfer recognition model;

[0375] represents the uncertainty corresponding to the i-1th intermediate variable;

[0376] u 2 Represents the uncertainty corresponding to the second pre-trained single-source domain transfer recognition model;

[0377] represents the kth dimension of the BBA corresponding to the first pre-trained single-source domain transfer recognition model;

[0378] represents the kth dimension of the BBA corresponding to the second pre-trained single-source domain transfer recognition model;

[0379] represents the kth dimension of the BBA corresponding to the Pth pre-trained single-source domain transfer recognition model;

[0380] It represents the kth dimension of the BBA corresponding to the single-source domain transfer recognition model after the i+1th pre-training.

[0381] Indicates the final fusion of BBA.

[0382] Module M3.3 converts the final fused BBA and final fused uncertainty obtained in module M3.2 back into prediction probability.

[0383] In this embodiment, the final fused BBA and the final fused uncertainty are converted into the final prediction probability of each category to facilitate decision making.

[0384] Specifically, module M3.3 includes:

[0385] The final fusion uncertainty According to a certain prior probability distribution (such as uniform distribution), it is assigned to each category. That is, it is assumed that in the absence of other information, the prior probability of the sample belonging to each category is Get the target sample The final posterior probability of belonging to the k+1th class Get the predicted probability of each category, and the final predicted probability vector is

[0386] Specifically, module M4 includes the following submodules:

[0387] Module M4.1, for each sample in the target domain According to the final predicted probability vector of the sample Take the highest dimension in the final predicted probability vector, that is, the highest predicted probability Get the predicted category k.

[0388] The highest predicted probability Compared with the set screening threshold t, if It is considered that the model has a high confidence in the prediction of the sample, and a high confidence pseudo label in the form of one-hot encoding is generated, which is expressed as Only the kth dimension is 1, and the rest are 0.

[0389] Among them, the screening threshold t is a hyperparameter and is set before training;

[0390] High confidence pseudo labels obtained based on predictions The model is trained using cross entropy loss.

[0391] Module M4.2 collects all target domain samples with high-confidence pseudo labels to form a training subset, and uses the training subset to train the model by minimizing the cross-entropy loss.

[0392] The cross entropy loss consists of:

[0393] The overall cross entropy loss measures the cross entropy loss between the final prediction result of the model and the high-confidence pseudo-label, expressed as:

[0394]

[0395] in:

[0396] represents the overall cross entropy loss;

[0397] Represents the model's predicted probability that the i-th target domain dataset sample belongs to the k+1-th class;

[0398] Represents the value of the model's k+1th dimension of the pseudo label for the i-th target domain dataset sample;

[0399] T represents the target domain.

[0400] The single-source cross-entropy loss measures the cross-entropy loss between the independent prediction results of the pre-trained single-source domain transfer recognition model and the high-confidence pseudo-label. For the p-th source domain, it is expressed as:

[0401]

[0402] in:

[0403] represents the single-source cross entropy loss;

[0404] represents the predicted probability of the p-th pre-trained single-source domain transfer recognition model for the i-th target domain dataset sample belonging to the k+1-th class;

[0405] Represents the value of the k+1th dimension of the pseudo label.

[0406] Combining the overall cross entropy loss and the single-source cross entropy loss, we get the total loss expression: Minimize the total loss through optimization algorithms Update the model's trainable parameters, including the parameters of each pre-trained single-source domain transfer recognition model.

[0407] Module M4.3 repeatedly executes modules M4.1 to M4.2, using the updated model to re-predict target domain samples, generate new high-confidence pseudo labels, and optimize the cross-entropy loss based on the new high-confidence pseudo labels until the preset number of iterations is reached or the model performance reaches saturation on the target domain, and the final optimization result is obtained.

[0408] Through multiple rounds of such iterations, the performance of the model in the target domain gradually improves, and the quality of pseudo-labels also improves accordingly, forming a positive self-learning cycle, thereby guiding the model to learn more robust and accurate discriminant features, and ultimately perform well in the target domain.

[0409] Through the collaborative work and iterative optimization of the four modules described above, this paper aims to build a high-performance image recognition system that can effectively integrate information from multiple source domains, overcome inter-domain differences, and gradually adapt to the characteristics of target domain data. This system not only focuses on improving final recognition accuracy but also explicitly models and processes prediction uncertainty through evidence theory, thereby enhancing decision reliability. This approach is particularly suitable for complex image recognition scenarios where source data is highly heterogeneous and target domain data is unavailable or expensive to label.

[0410] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0411] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A multi-source domain deep migration image recognition method based on evidence theory, characterized by: The steps include: Step S1: input multiple source domain and target domain images and set hyperparameters, including loss ratio hyperparameter, number of iterations, uncertainty coefficient, and screening threshold; Step S2: Based on the loss ratio hyperparameter, transfer learning classification is performed on each source domain-target domain pair to obtain multiple pre-trained single-source domain transfer recognition models; Step S3: Based on the uncertainty coefficient, the prediction results of each pre-trained single-source domain transfer recognition model are converted into BBA through Dirichlet distribution, and fused using evidence theory to obtain the BBA and category of the target domain sample, and then converted back into probability, i.e., the final prediction probability; In step S4, a pseudo label is generated for the target domain according to the final prediction probability, and a high-confidence pseudo label is obtained by screening according to the screening threshold. The model is trained, and pseudo label generation and training are repeated until the number of iterations is reached to complete iterative optimization.

2. The multi-source domain deep migration image recognition method based on evidence theory according to claim 1 is characterized in that: The step S1 includes the following sub-steps: Step S1.1, input P source domains are defined as The corresponding source domain dataset is The target domain is defined as D T ={χ T ,P(X T )}, the corresponding target domain dataset is The feature spaces of the source domain and the target domain are consistent, that is, But the data distribution is different, i.e. And set the category space of the target recognition task to Ω={ω1,ω2,…,ω K }; in: represents the source domain; p represents the p-th source domain; S p represents the p-th source domain dataset; Represents the feature space of the p-th source domain dataset; represents the sample set of the p-th source domain; Represents the features of the i-th sample of the p-th source domain dataset; represents the label of the i-th sample of the p-th source domain dataset; Represents the number of samples in the p-th source domain dataset; P(·) represents the distribution of data; D T represents the target domain; χ T Represent the feature space of the target domain dataset; N T Indicates the number of samples in the target domain dataset; X T Represents the sample set of the target domain dataset; Represents the jth sample of the target domain dataset; ω k represents the kth category; K represents the total number of categories; Step S1.2: Setting hyperparameters, including: The loss ratio hyperparameter is used to balance the weights of classification cross entropy loss and local maximum mean difference loss in the total loss function; The number of iterations is used to indicate the number of iterations of pseudo-label screening and model training; Uncertainty coefficient, used to adjust the degree of uncertainty when converting the predicted probability into Dirichlet distribution parameters; The screening threshold is used to screen out pseudo-label samples with high confidence, and retain pseudo-label samples with a prediction probability higher than the screening threshold for subsequent model training.

3. The multi-source domain deep migration image recognition method based on evidence theory according to claim 2 is characterized in that: The step S2 includes the following sub-steps: Step S2.1: Divide the source domain and the target domain into K corresponding category sub-domains according to the K categories of the object recognition task. For each category sub-domain, calculate the local maximum average difference loss term between the source domain and the target domain in the reproducing kernel Hilbert space. The local maximum average difference loss term is defined as the average of the sum of the maximum average differences over all category sub-domains, that is: in: represents the local maximum average difference loss term, which is the average of the sum of the maximum average differences between the source domain data and the target domain data in each category sub-domain; m represents the number of samples in the p-th source domain dataset; n represents the number of samples in the p-th target domain dataset; represents the i-th sample of the p-th source domain; represents the jth sample in the target domain; f(·) represents the function that transforms from the feature space to the reproducing kernel Hilbert space; Represents the weight assigned to the k-th category on the p-th source domain, through Calculate the fraction of samples that belong to the category among all samples; Indicates the label corresponding to the k-th category in the p-th source domain, with a value of 0 or 1; Represents the weight assigned to the kth category in the target domain, through calculate; Represents the label corresponding to the kth category in the target domain; ||·|| means calculating the norm, ||·|| 2 is the square of the norm; Step S2.2: For source domain data, calculate the classification cross entropy loss between the model prediction results and the true labels, that is: in: y k Represents the true label of the sample (using one-hot encoding); represents the predicted probability distribution of the single-source domain transfer recognition model for sample x; represents y k and The cross entropy between N represents the number of samples in the corresponding source domain; Step S2.3, performing a weighted combination of the local maximum average difference loss term of step S2.1 and the classification cross entropy loss of step S2.2 to obtain the total loss function in, They represent the classification cross entropy loss term and the local maximum average difference loss term respectively; λ represents the loss ratio hyperparameter that adjusts the ratio of the two loss terms, and the total loss function is minimized to train the single-source domain transfer recognition model. After the training is completed, the pre-trained single-source domain transfer recognition model can recognize any sample in the target domain. The output prediction result is expressed as That is, the prediction result of the p-th pre-trained single-source domain transfer recognition model is used for the i-th sample in the target domain dataset. The prediction result is expressed as a vector of the number of categories, that is, expressed as in: Represents the value of the kth dimension of the vector, that is, the predicted probability of the sample belonging to the k+1th category by the pth pre-trained single-source domain transfer recognition model.

4. The multi-source domain deep migration image recognition method based on evidence theory according to claim 3 is characterized in that: The step S3 includes the following sub-steps: Step S3.1: Convert the prediction results of each pre-trained single-source domain transfer recognition model into BBA through Dirichlet distribution to obtain the corresponding P pre-trained single-source domain transfer recognition models about the target domain samples. BBA; Step S3.2: Use evidence theory combination rules to analyze the P pre-trained single-source domain transfer recognition models on the target domain samples. The BBAs are fused to obtain the final fusion BBA and the final fusion uncertainty; Step S3.3, converting the final fused BBA and the final fused uncertainty into the final prediction probability for each category.

5. The multi-source domain deep migration image recognition method based on evidence theory according to claim 4 is characterized in that: The step S3.1 includes: setting the parameters of the Dirichlet distribution in Represents the predicted probability of the kth class; β represents the parameter that adjusts the proportion of uncertainty in the evidence theory, that is, the uncertainty coefficient; Set each category to true BBA: This means that the sample belongs to the BBA of the k+1th class; Where S represents the normalized coefficient, and the calculation formula is Setting uncertainty Represents the BBA that is uncertain to which category the sample belongs. Therefore, the predicted probability vector of the p-th pre-trained single-source domain transfer recognition model is Converted into a set of BBA values ​​{b0,b1,…,b K-1 ,u}.

6. The multi-source domain deep migration image recognition method based on evidence theory according to claim 5 is characterized in that: The step S3.2 includes the following sub-steps: Step S3.2.1, calculate the conflict coefficient in: m(·) represents the function for calculating the corresponding BBA; m p (·) represents the function of calculating the corresponding BBA of the p-th pre-trained single-source domain transfer recognition model; m q (·) represents the function of calculating the corresponding BBA of the qth pre-trained single-source domain transfer recognition model; Indicates that the intersection of subset A and subset B in the identification framework is empty; A and B represent two subsets in the recognition framework, and their intersection needs to be an empty set. In the scenario defined here, it is only necessary to set all i≠j. and Just multiply and sum the corresponding numbers. Represents the value of the i-th dimension of the BBA corresponding to the p-th pre-trained single-source domain transfer recognition model; represents the value of the jth dimension of the BBA corresponding to the qth pre-trained single-source domain transfer recognition model; Step S3.2.2, calculate the normalization constant That is, 1 minus the conflict coefficient; Step S3.2.3, for the p-th source domain and the q-th source domain, calculate the BBA after normalization constant fusion: Among them, k represents the kth dimension of the fused BBA, u q represents the uncertainty corresponding to the qth pre-trained single-source domain transfer recognition model, u p represents the uncertainty corresponding to the p-th pre-trained single-source domain transfer recognition model; Calculate the uncertainty after normalization constant fusion 7. The multi-source domain deep migration image recognition method based on evidence theory according to claim 6 is characterized in that: The step S3.3 includes: The final fusion uncertainty Assign to each category according to uniform distribution, that is, assuming that in the absence of other information, the prior probability of the sample belonging to each category is Get the target sample The final posterior probability of belonging to the k+1th class Get the predicted probability of each category, and the final predicted probability vector is 8. The multi-source domain deep migration image recognition method based on evidence theory according to claim 7 is characterized in that: Step S4 includes the following sub-steps: Step S4.1: for each sample in the target domain According to the final predicted probability vector of the sample Take the highest dimension in the final prediction probability vector, that is, the highest prediction probability Get the predicted category k; The highest predicted probability Compared with the set screening threshold t, if It is considered that the model has a high confidence in the prediction of the sample, and a high confidence pseudo label in the form of one-hot encoding is generated, which is expressed as Only the kth dimension is 1, and the rest are 0; Among them, the screening threshold t is a hyperparameter and is set before training; High confidence pseudo labels obtained based on predictions The model is trained using cross entropy loss; Step S4.2: Collect all target domain samples with high-confidence pseudo labels to form a training subset, and use the training subset to train the model by minimizing the cross entropy loss. The cross entropy loss includes: The overall cross entropy loss measures the cross entropy loss between the final prediction result of the model and the high confidence pseudo label, expressed as: in: Represents the model's predicted probability that the i-th target domain dataset sample belongs to the k+1-th category; Represents the value of the model's k+1th dimension of the pseudo label for the i-th target domain dataset sample; T represents the target domain; The single-source cross entropy loss measures the cross entropy loss between the independent prediction results of the pre-trained single-source domain transfer recognition model and the high-confidence pseudo-label. For the p-th source domain, it is expressed as: in: represents the predicted probability of the p-th pre-trained single-source domain transfer recognition model for the i-th target domain dataset sample belonging to the k+1-th class; Represents the value of the k+1th dimension of the pseudo label; The overall cross entropy loss and the single-source cross entropy loss are combined to obtain the total loss expression: Minimize the total loss by an optimization algorithm Updating the trainable parameters of the model, including the parameters of each pre-trained single-source domain transfer recognition model; Step S4.3, repeatedly execute steps S4.1 to S4.2, use the updated model to re-predict the target domain samples, generate new high-confidence pseudo labels, and optimize the cross-entropy loss based on the new high-confidence pseudo labels until the preset number of iterations is reached, or the model performance reaches saturation on the target domain, and the final optimization result can be obtained.

9. A multi-source domain deep migration image recognition system based on evidence theory, characterized by: include: An initialization module is used to input multiple source and target domain images and set hyperparameters, including a loss ratio hyperparameter, the number of iterations, an uncertainty coefficient, and a screening threshold; A transfer learning module is used to perform transfer learning classification on each source domain-target domain pair based on the hyperparameters to obtain multiple pre-trained single-source domain transfer recognition models; The evidence fusion module converts the prediction results of each pre-trained single-source domain transfer recognition model into BBA based on the uncertainty coefficient through Dirichlet distribution, and fuses them using evidence theory to obtain the BBA and category of the target domain sample, and converts them back into probability, i.e., the final prediction probability; The pseudo-label generation and training module generates pseudo-labels for the target domain according to the final prediction probability, filters them according to the screening threshold, obtains high-confidence pseudo-labels, trains the model, and performs iterative optimization through multiple rounds of repeated pseudo-label generation and training.

10. The multi-source domain deep migration image recognition system based on evidence theory according to claim 9 is characterized in that: The source domain includes a plurality of source data, and the source data includes images and labels corresponding to the images; the target domain includes target data, and the target data includes images.

Images